I am not a sysadmin. In fact, I’m every sysadmin’s worst nightmare: I’m a power user that has access to sudo. I run a few machines and when there are problems, I follow the xkcd approach. This leads to me copy-pasting in commands that I found over google, usually on stackexchange. This is how science advances. This post details my experience trying to run a multi-user OpenStack project here. The basic issue is that we have one (1) public IP address for our VMs and many users and instances. There are two possible approaches. The nice route would be to just configure the virtual router on the openstack cloud to do this forwarding automagically. Alternatively, we could make a routing VM where we can address this directly.

Managing the Virtual Router

I can’t find a way to access this functionality without access to the underlying OpenStack interface, i.e., this seems supported but it’s not a user-space option. It does seem possible and preferred.

Creating a VM router

The longer saga of trial and error.

First, I created a new security group that allowed for custom TCP access on a range of ports. Let’s say it’s 2201 to 2222. It isn’t, but that’s good for now.

On the host VM, I edit /etc/ssh/ssh_config to inclue the lines

Port 2201
Port 2202
...
Port 2222

and I do the same thing in /etc/ssh/sshd_config. I then restarted ssh: service ssh restart.

Now, I can ssh into the gateway VM on nonstandard ports. Next up, I’ll try to pick off these ports with an iptables and send them onto another VM. First, I allow port forwarding on the gateway VM by editing /etc/sysctl.conf

net.ipv4.ip_forward=1
# or for a one-off:
sysctl -w net.ipv4.ip_forward=1

Finally, I set up iptables:

iptables -t nat -A PREROUTING -p tcp --dport 2201 -j DNAT --to-destination 192.168.2.22:22
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

One cool thing that came of this was the ability to ssh through to the private machines using ssh-agent. This is fine for me but it doesn’t do anything to help anyone who doesn’t have their ssh keys injected into the server. I could create users and allow administration, but that defeats the whole point of the OpenStack exercise.

ssh-agent
ssh-add /path/to/ssh/privatekey

ssh -A ubuntu@public.ip.address
ssh ubuntu@private.ip.address

Here’s a bunch of stuff that didn’t work.

One solution suggested something like this:

 iptables -t nat -A POSTROUTING -d 192.168.2.0/24 -o eth1 -j SNAT --to-source XXX.XXX.XXX.XXX
 iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 2201 -j DNAT --to 192.168.2.22:22
 iptables -A FORWARD -p tcp -d 192.168.2.22 --dport 2201 -j ACCEPT

I note that the VMs only come with one virtual eth0 NIC. Again, we hit the problem that a lot of the established solutions want the VM to be built like a proper switch/router with an extra eth1 port.

Cybera’s old instructions say that all you need to do is:

iptables -t nat -A PREROUTING -p tcp --dport 2201 -j DNAT --to-destination 192.168.2.22:22

However, they advise using a specific RAC Proxy VM with the software already installaed. It’s not clear if this image has a second NIC in it or not. Another location suggests adding something to the main tables (i.e., not the NAT tables) to allow a forward:

iptables -A FORWARD -p tcp -d 192.168.2.22 --dport 22 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

Maybe the above could use some input port control as well --sport 2201. Or the interface (-i eth0) Some cases suggest including a MASQUERADE flag:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE  

However, this again is a eth0 and eth1 case. Things aren’t looking good for our heros. I’m not sure but it seems like we want to provision VMs with another NIC.