Setup multiple vNICs on an OpenStack-based CentOS VM
Introduction
Sometimes, you need an external Internet-facing floating IP (FIP) for your existing OpenStack-based CentOS VM with an internal network interface. However, since the VM is going to be in use, getting rid of the old internal-facing network interface may not be possible. This can be because data is already flowing to it, or for configuration reasons. In either case, you can fix this issue by adding a second network interface that links the VM to an external Internet-facing network. Check out the steps below to find out how!
Prerequisites
To do all the steps listed below, you will need to have OpenStack installed, as well as a CentOS VM already set up.
Start Here
First thing's first: warn your users that network connectivity may be lost during the configuration and that you will need a maintenance window. Then create a snapshot of the VM in case you bonk the setup.
Add the second, external-facing NIC to the VM (through the Horizon dashboard is fine but you can also use the OpenStack CLI if you are so inclined).
Associate a floating IP with the NIC (again, either through Horizon dashboard or CLI).
Setup a new OpenStack security group to include the incoming ports your users have requested. At this time, it doesn't hurt to add SSH and All ICMP rules for your own IP address for testing. Remember, generally, these requests are to open up the VM to all Internet-facing traffic so exercise discretion when setting allowable ports/port ranges.
Add the new OpenStack security group to the VM.
SSH into the VM on the existing internal interface.
Check the current network configuration:
ifconfig -a
The -a argument will list all detected network interfaces, including those that are down
You should see the second network interface at this point. Copy its MAC address.
CentOS 8 cloud images don't appear to consistently come with /lib/udev/write_net_rules (maybe because they use cloud-init) so we will modify the persistent net rules configuration by hand. Add an entry for the new network interface (eth1 in this example). It doesn't appear from my experience that order matters in this file so eth1 should be able to be after eth0 if you're picky about those types of things:
[centos@opst-centos-vm ~]$ sudo vi /etc/udev/rules.d/70-persistent-net.rules
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?", ATTR{address}=="fa:16:3e:6f:75:c0", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?", ATTR{address}=="fa:16:3e:35:b1:51", NAME="eth0"[centos@opst-centos-vm ~]$
Open /etc/sysconfig/network-scripts/ifcfg-eth0 (or whatever interface name is currently the main interface). We will ignore the file's comment which says "..., do not edit."
[centos@opst-centos-vm ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-eth0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=dhcp
DEVICE=eth0
HWADDR=fa:16:3e:ed:8b:ae
MTU=1450
ONBOOT=yes
STARTMODE=auto
TYPE=Ethernet
USERCTL=no
Add the following line, write the file to disk and exit the file:
DEFROUTE=no
Copy the file you just edited to a new file in the same directory but append the second network interface's name to the command (eth1 in our case):
sudo cp /etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth1
Open the new file, replace the existing MAC address with the new interface's MAC address:
[centos@opst-centos-vm ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-eth1
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=dhcp
DEVICE=eth0
HWADDR=fa:16:3e:ed:8b:ae (replace with new MAC address)
MTU=1450
ONBOOT=yes
STARTMODE=auto
TYPE=Ethernet
USERCTL=no
Add the following line, write the file to disk and exit the file:
DEFROUTE=yes
At this point, you should be ready to reboot to ensure the network configuration comes up properly. Confirm that no users are still logged in. If there are users still logged in send them a shutdown message:
sudo shutdown -r +5 "The system is going down for maintenance, please save all work ASAP"
After reboot, attempt to ping the new IP address. You should receive completed ping responses. Then, attempt to SSH into the new address. If you're successful, confirm that the route table is set up appropriately. If you're unsuccessful skip to step 16:
[centos@opst-centos-vm ~]$ ip r
default via 10.10.2.1 dev eth1 metric 100
default via 10.10.1.1 dev eth0 metric 101
10.10.1.0/24 dev eth0 proto kernel scope link src 10.10.1.120
10.10.2.0/24 dev eth1 proto kernel scope link src 10.10.2.8
[centos@opst-centos-vm ~]$
Finally, confirm that the external-facing NIC is handling Internet-bound traffic. If it is, you will know because the FIP will be the IP address that is returned by the command:
[centos@opst-centos-vm ~]$ curl ifconfig.co
###.###.###.###
[centos@opst-centos-vm ~]$
If you are not able to SSH into the VM through the FIP, then attempt to SSH in through the internal interface. If you are unable to access through the internal interface, you may have to rebuild the VM (VM rebuild using the same IP address tutorial will be published at a later date). If you are able to SSH in through the internal interface, confirm your route table and which NIC is handling outbound Internet traffic:
ip r
curl ifconfig.co
If the internal interface appears as the default interface and/or has a higher listing in the route table than the new interface or the IP address that is returned from your curl command gives a different IP address than your FIP or whatever is expected (your VPN IP, for example), check that DEFROUTE=yes/no in your eth## config files are set appropriately. If this is not the issue, additional debugging beyond the scope of this tutorial will be required.
Once the FIP is up and working, remove the SSH and All ICMP rules from your new security group.
Notify your users that the VM is ready for use.
Conclusion
There you have it! Now your VM can send and receive traffic through an Internet-facing FIP. However, if you struggle through any of the above steps, contact Awnix and let us guide you through it!