Categories

Pages

Sample vCloud Air CLI Session – Subscription Service

Here is a sample interaction with a vCloud Air Subcription Service using the vCloud Air Command Line Interface.// Install vca in a Python Virtual Env

$ sudo pip install virtualenv
$ virtualenv pythonenv
$ cd pythonenv
$ source bin/activate// Get a specific version of vca-cli, not the production version.
$ pip install vca-cli==10rc8

// check version, should be same as below
# vca –version
vca-cli version 10rc8 (pyvcloud: 12rc4)
$ vca login org1@websterx.com –password mypswd –host vchs.vmware.com –type subscription –version 5.6 –org M933009684-4424
Login successful for profile ‘default’

$ vca status
Status:
| Property        | Value                                                |
|—————–+——————————————————|
| gateway         |                                                      |
| host            | https://vchs.vmware.com                              |
| host_score      | https://score.vca.io                                 |
| instance        |                                                      |
| org             | M933009684-4424                                      |
| org_url         |                                                      |
| password        |                                                      |
| service         |                                                      |
| service_type    | subscription                                         |
| service_version | 5.6                                                  |
| session         | inactive                                             |
| session_token   |                                                      |
| session_uri     |                                                      |
| token           | fe69f2df510846bfaa06d44e2820ae0ece9458c3,30f1f092-.. |
| user            | org1@websterx.com                                    |
| vdc             |                                                      |
| verify          | True                                                 |

$ vca org use –org M933009684-4424 –service  M933009684-4424
Using organization ‘M933009684-4424′ in profile ‘default’
Details for org ‘M933009684-4424′:
| Type       | Name                                 |
|————+————————————–|
| Org Id     | 8688e315-7b8c-47e8-8c5e-2b2abc962ce7 |
| Org Name   | M933009684-4424                      |
| catalog    | cf-catalog                           |
| catalog    | Public Catalog                       |
| catalog    | pivotal_om_4c3986062021e2975053      |
| catalog    | pivotal_om_cf7c2633a5162e3c25ba      |
| orgNetwork | M933009684-4424-default-routed       |
| orgNetwork | M933009684-4424-default-isolated     |
| vdc        | M933009684-4424                      |
vcadevops@precise-amd64:~/chef-repo$

$ vca vdc use –vdc M933009684-4424

$ vca vdc
Available Virtual Data Centers in org ‘M933009684-4424′ for ‘default’ profile:
| Virtual Data Center   | Selected   |
|———————–+————|
| M933009684-4424       |            |
(pippre)bwebster-mbpro:pippre bwebster$

$ vca vdc use –vdc M933009684-4424

$ vca network
Networks available in Virtual Data Center ‘M933009684-4424′:
| Name                             | Mode      | Gateway       | Netmask       | DNS 1   | DNS 2   | Pool IP Range                 |
|———————————-+———–+—————+—————+———+———+——————————-|
| M933009684-4424-default-isolated | isolated  | 192.168.99.1  | 255.255.255.0 | None    | None    | 192.168.99.2-192.168.99.100   |
| M933009684-4424-default-routed   | natRouted | 192.168.109.1 | 255.255.255.0 | None    | None    | 192.168.109.2-192.168.109.100 |

$ vca catalog
Catalogs and items:
| Catalog                         | Item                                     |
|———————————+——————————————|
| Public Catalog                  | CentOS64-32bit                           |
| Public Catalog                  | CentOS63-32bit                           |
| Public Catalog                  | W2K12-STD-64BIT-SQL2K12-STD-SP1          |
| Public Catalog                  | CentOS63-64bit                           |
| Public Catalog                  | W2K12-STD-R2-SQL2K14-WEB                 |
| Public Catalog                  | W2K12-STD-64BIT                          |
| Public Catalog                  | W2K12-STD-R2-64BIT                       |
| Public Catalog                  | CentOS64-64bit                           |
| Public Catalog                  | W2K8-STD-R2-64BIT                        |
| Public Catalog                  | Ubuntu Server 12.04 LTS (amd64 20150127) |
| Public Catalog                  | W2K8-STD-R2-64BIT-SQL2K8-WEB-R2-SP2      |
| Public Catalog                  | W2K12-STD-R2-SQL2K14-STD                 |
| Public Catalog                  | W2K12-STD-64BIT-SQL2K12-WEB-SP1          |
| Public Catalog                  | Ubuntu Server 12.04 LTS (i386 20150127)  |
| Public Catalog                  | W2K8-STD-R2-64BIT-SQL2K8-STD-R2-SP2      |
| cf-catalog                      | Pivotal Ops Manager VApp-2               |
| cf-catalog                      | CentOS64-64bit                           |

// 1 time setup of dhcp pool on vdc gateway
$ vca dhcp add –network M933009684-4424-default-routed  –pool 192.168.109.101-192.168.109.110

$ vca vapp create -a ubu -V ubuChefNode -c ‘Public Catalog’ -t ‘Ubuntu Server 12.04 LTS (amd64 20150127)’ -n M933009684-4424-default-routed -m DHCP

// If no vmware guest customziaton

$ vca vapp power.on –vapp ubu

// if using customization

vca vapp customize –vapp ubu –vm ubuChefNode –file ./add_public_ssh_key.sh

vCloud Air – An inside server cannot reach itself through the gateway public ip

In most cases accessing a server on an internal network through the gateway is a simple matter of adding NAT rules to the Edge Gateway.
For example these rules would allow an external server to access an internal server through the gateway public ip.

DNAT   23.92.200.151 port 4242      >>  192.168.109.3  port 4242
SNAT   192.168.109.0/24                 >>  23.92.200.151

gateway_sm

The Hairpin NAT Problem:

But a confusing issue can occur when an server on the inside network tries to reach itself or another inside server by addressing the public ip on the gateway.

In the diagram above, 192.168.109.3 is a server on a vCloud Air Virtual Data Center
internal network and 23.92.200.151 is a public ip on the Gateway of the VDC.

Given the gateway rules above, let’s test this scenario .

Test

Start Netcat on the inside server and have it listen on 192.168.109.3 port 4242 for connections.

$ nc -l 4242 &

// From the inside server, we can reach the same server using the inside address 192.1.68.109.3
$ telnet 192.168.109.3 4242
— Connected Accepted

// From a server outside vCloud Air 50.4.227.75 we can reach through the gateway public ip to the inside server.
$ telnet 23.92.200.151 4242
— Connection Accepted

// But, from the inside server 192.168.109.3 to public ip outside we don’t get routed back in
$ telnet 23.92.200.151 4242
— Connection Refused

Whats Going on?
Why are all connections to 23.92.200.151 4242 not being routed to 192.168.109.3 4242 ?

Solutions:

Solve with a Local Host File Entry

The simplest solution is do the lookup using a hostname instead of an ip.
Add an entry to the local server hosts file so the local address is used for inside destinations.

For example

In the /etc/hosts file add the line

myserver 192.168.109.3

When this entry exists on the local server, hostname resolution will find the local server address before any DNS resolution finds a the public ip of the gateway.

But sometimes its not possible to know the local address of all servers on the internal network.
This leads us to the second solution using the gateway.

Solve with Gateway Configuration

In this solution we will add NAT rules to both the internal network and the Gateway external network.

gateway_sm

Add identical DNAT and SNAT rules  to both networks.

Explanation:

With the new rules,

  • Calls from an outside server to the gateway public ip – 50.4.227.75   calling   23.92.200.151  4242

When the packet below arrives at the 192.168.109.0 network
Rule #2 – Changes the destination address in the packet to 192.168.109.3 4242
Rule #4 – Not applied change since source is not 192.168.109.x  Src packet stays the same.
IP SRC:  50.4.227.75                       >>          IP SRC:  50.4.227.75
IP DST:   23.92.200.151  4242                       IP DST:  192.168.109.3  4242

  • Calls from the inside server to gateway public ip  – 192.168.109.3   calling   23.92.200.151

When the packet below enters the 23.92.200.0 network

Rule #1 – Changes the Destination address in packet from 23.92.200.151 to 192.168.109.3
The packet never leaves the network.

Rule #3 – Changes the Source address from 192.168.109.3 to 23.92.200.151
Two tricky things here,  the packet looks like it came from outside, so responses will be sent back to the outside address. But DNAT rule #1 will route responses back to the  192.168.109.3)

IP SRC:  192.168.109.3                      >>      IP SRC:  23.92.200.151
IP DST:   23.92.200.151  4242                      IP DST:  192.168.109.3 4242

Additional DNAT rules will be required on both networks for all additional inside servers that will require inside access through the gateway public ip.

SSL Handshake error on vCloud Air configuring Pivotal Cloud Foundry

When installing PCF 1.3.2 on vCloud Air I ran into an “SSL Handshake error”
when attempting to access the Cloud Foundry Ops Mgr setup page.

https:://23.92.99.98/setup
Firefox reported the following error:

“The connection was Interrupted”
The connection was interrupted while the page was loading.
The site could be temporarily unavailable or too busy.
Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy,
make sure that Firefox is permitted to access the Web.

A quick test with openssl verfied the issue

$  openssl s_client  -connect 23.92.99.98:443
CONNECTED(00000003)
140694674667168:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:596:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 0 bytes

New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol  : TLSv1
Cipher    : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg   : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1415297890
Timeout   : 7200 (sec)
Verify return code: 0 (ok)

Resolution:
Each vCloud Air Virtual Data Center includes two public ip addresses on the Gateway Appliance.
During setup for Pivotal Cloud Foundry,
NAT rules are configured to map one of the ips to the PCF Ops Manager
and the other ip to the gateway load balancer.

Apparently the two public ips are not equivalent.
If you encounter the ssl error above, change the Gateway NAT rules to reverse the ip mappings
so that the ip currently mapped to the router is mapped to Ops Manager.

For example:
Given two Gateway Ips of
23.92.99.98
23.92.99.99
And
Ops Manager on 192.168.109.2  and the
Gateway Load Balancer on 192.168.109.10

d2p3-ext DNAT    23.92.99.98       Any 192.168.109.2     Any TCP
d2p3-ext DNAT    23.92.99.99       Any 192.168.109.10   Any TCP

Change the Nat rules to

d2p3-ext DNAT    23.92.99.98       Any 192.168.109.10  Any TCP
d2p3-ext DNAT    23.92.99.99       Any 192.168.109.2    Any TCP