My two favorite things in IT are networking and linux. It’s an exciting time for both right now because they’re kind of meshing together. Linux is fully recognized today as a powerful platform for business along with stable distro’s being available for workstation use. It really has come along way since I first installed Fedora Core back in my teenage years. It’s an explosive time for Open Source as a whole. Large businesses are laying profitable paths that include or launch Open Source Projects. This trend is really starting to effect all areas of IT, bringing me to the excitement around Networking.
Linux has for as long as I’ve known it, had some robust abilities for networking. Today however, quite a few big names are spinning their own version with networking itself being the focus. I’ve been particularly interesting in AT&T’s dNOS project, which is really just getting off the ground. They’ve handed it over to the Linux Foundation who has renamed it the DANOS project (Disaggregated Network Operating System). AT&T also purchased Vyatta from Brocade this year. Vyatta is the earliest “Linux in a router” I know of. If you’ve ever used VyOS or Ubiquiti products, you’ve used some Vyatta code. Let’s not leave Pfsense out of the discussion. Yes it’s BSD, but it’s open source. Open Network Linux (ONL) is a switch OS ultilizing the ONIE install environment that’s been created by Cumulus Networks. We could also mention a slew of switch, router, and SDN platforms like OpenSwitch, OpenVswitch, Openwall, OpenWrt, OpenDaylight, Untangle; But I’m not trying to make this about a comprehensive list. This is more to recognize the huge groundswell of innovation coming into the Networking industry.
So there’s an important bit of information that makes all this possible today. It used to be that if you wanted to push traffic and make logic decisions quickly enough for a network, you required ASIC’s. Today, much of this can be accomplished on an x86 processor. This is why home users are installing pfsense on an old dell workstation, throwing in a x4 NIC card and spinning up a decent router (ensure the cpu has AES instruction set). Before I decided to buy a physical box, I ran a VyOS instance virtualized under Proxmox as my home router. I never hit a performance limitation during that use. I’m not saying that an x86 processor can do everything that a big iron DPI firewall device can, but the gap has closed significantly compared to 10 years ago and it continues to shorten.
White Box switching has been here for a couple years. Yes I’m not seeing it in the wild, I’m pretty sure it’s still pretty well isolated to Datacenters, but there’s nothing stopping us from trying it out. Take Cumulous Networks, they allow you to grab their image and spin it up in virtual, try it out. They don’t sell switches, they sell the platform. And, back to the theme of this writing, it’s just a linux OS. Companies like Dell, HPE, and Supermicro are now manufacturing some of their switches to be loaded with 3rd party OS’s (Not to mention plenty of Big Iron brands I don’t encounter). Cisco has an iOS version that can be purchased as software only, no hardware required.
Playing catch up
So I’m not in big enterprise IT and I don’t rub shoulders with developers aside from tech meetups. However, I know enough to see why we’ve gotten here and where we’re going. Dev-Ops has become the standard philosophy for any modern development team. This writing isn’t going to go into what Dev-Ops is. I could point to tools like Ansible, Chef, Puppet, Vagrant, but those are just tools, that’s not what Dev-Ops is and like I said, I never worked in that environment. The point is though, it has made development a streamlined and high output process. Developers are able to produce at rates that other silo’s in IT can’t keep up with. That’s where open source networking comes in. Networking more than ever needs a control plane that scales. Networking needs a network infrastructure that can be virtualized. Networking needs overlay’s than can span Datacenter’s. And of course, queue cliche, it needs Automation. The old big iron companies, the Ciscos, the Junipers, haven’t really be leading the way, so engineers have made their own way. These changes are going to have an effect on small and medium business over time. That’s the part that I’m excited about. SDN and SDWAN have so far manifested in products like Meraki and it seams like every Wifi vendor has some kind of Cloud Platform. I see these as pretty rudimentary and I’m more interested in compatible northbound API’s enabling a multi-vendor SDN solution. I think Open Source is what’s going to get us there.
Making a Router
CentOS7, or pretty well any distro out there has routing capabilities out of the box. This is to point out that Networking is at the core of Linux.
I’m not trying to say this is the best way, but it’s possible. What I’m about to go over comes in handy for me quite a bit. I run Linux on my work laptop and just spin up a Windows VM when needed. With that, I can effectively place a router in front of my VM that’s connected to multiple subnets on a network, maybe a couple VPN’s, and have the ability to setup out of box devices with default IP’s without having to fumble through the Windows UI and lose my current internet connection. The point is, where you have a linux box, you have some robust networking capabilities.
All configuration below is not intended to be persistent. A reboot will clear all settings. /etc/sysconfig/network-scripts will have the configuration files needed to create persistent configuration changes.
Topology above. Starting with basic network configuration, this is going to cover Routing and NAT as well. Services like DHCP, NTP, DNS; can be turned up similar to any linux host as well, I’m not going to bother with that here.
ip is that start of all good things networking in linux. To view all active interfaces, use…
[root@centos7-router ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether d6:f1:28:19:a7:ce brd ff:ff:ff:ff:ff:ff inet 10.17.17.53/27 brd 10.17.17.63 scope global dynamic ens18 valid_lft 84890sec preferred_lft 84890sec inet6 fd10:17:17:10:9d09:62a2:cce8:2af4/64 scope global noprefixroute dynamic valid_lft 2591832sec preferred_lft 604632sec inet6 fe80::d56f:5fcf:f227:7708/64 scope link valid_lft forever preferred_lft forever 3: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 46:a4:b6:b6:4c:74 brd ff:ff:ff:ff:ff:ff inet6 fe80::be88:fd18:5ce7:3da0/64 scope link valid_lft forever preferred_lft forever 4: ens20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:c2:ea:f3:04:59 brd ff:ff:ff:ff:ff:ff inet6 fe80::4f95:2e0a:25fe:d1c3/64 scope link valid_lft forever preferred_lft forever
Or to see physical state use…
[root@centos7-router ~]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether d6:f1:28:19:a7:ce brd ff:ff:ff:ff:ff:ff 3: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 46:a4:b6:b6:4c:74 brd ff:ff:ff:ff:ff:ff 4: ens20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:c2:ea:f3:04:59 brd ff:ff:ff:ff:ff:ff
To turn down an interface,
ip link set ens18 down
and to turn up respectively.
ip link set ens18 up
Adding IP’s to an interface is straight forward, and you can add more than one like any good router allowing you to be part of multiple networks at once. ens19 is CentOS7-Router’s LAN interface. We’ll configure it and Client01’s interface so they can start talking.
[root@centos7-router ~]# ip addr add 192.168.56.1/29 dev ens19 [root@client01~]# ip addr add 192.168.56.2/29 dev ens18
The two hosts are up and reachable on the same LAN now. That just about covers basic interface management on a Linux host, but there’s a couple other commands I’m going to be using later on that fit in here.
To reach out and get a DHCP address, dhclient is the tool that’s used. By itself will cause all interface to send a DHCP discovery packet. To declare a specific interface, just follow the command with the interface name
dhclient ens18
Lastly, deleting an IP address from an interface.
ip addr del X.X.X.X dev ensXX
The above will delete an IP whether it was statically assigned or acquired from DHCP.
VLANS
Just a quick section here. VLAN’s on are pretty simple to configure. The only difference is that the interface must be turned up after creation. Create a VLAN with…
[root@centos7-router ~]# ip link add link ens19 name ens19.5 type vlan id 5
ip link will show the link as down until it’s turned on via…
[root@centos7-router ~]# ip link set ens19.5 up
I’ll do the same over on Client01, and assign some IP’s.
[root@client01 ~]# ip link add link ens18 name ens18.5 type vlan id 5 [root@client01 ~]# ip link set ens18.5 up [root@client01 ~]# ip addr add 192.168.5.2/29 dev ens18.5 [root@client01 ~]# [root@centos7-router ~]# ip addr add 192.168.5.1/29 dev ens19.5
Now both host can communicate both tagged and untagged traffic. Lastly, to delete a virtual interface, use…
[root@client01 ~]# ip link delete ens18.5
ip routing
That magic command that turns a capable Cisco switch into a Layer 3 switch. The same needs done for CentOS7-Router. Before any routing is truly enabled, any Linux host can build a routing table. Again this comes in handy when troubleshooting. Having multiple router’s on a network and a table to match prevents the need to disconnect from one network to join another. The commands here are pretty intuitive.
Show routes, both learned and connected with…
[root@centos7-router ~]# ip route default via 10.17.17.33 dev ens18 default via 10.17.17.33 dev ens18 proto static metric 100 10.17.17.32/27 dev ens18 proto kernel scope link src 10.17.17.53 metric 100 192.168.5.0/29 dev ens19.5 proto kernel scope link src 192.168.5.1 192.168.56.0/29 dev ens19 proto kernel scope link src 192.168.56.1
Add a route with… (note that it’s possible to declare the desired interface by appending <dev interface.name> to the command below, but unless there’s two existing routes to the “via” destination, it’s not necessary and will be appended automatically.
[root@centos7-router ~]# ip route add 192.168.50.0/24 via 10.17.17.33
Now ip route shows…
[root@centos7-router ~]# ip route default via 10.17.17.33 dev ens18 default via 10.17.17.33 dev ens18 proto static metric 100 10.17.17.32/27 dev ens18 proto kernel scope link src 10.17.17.53 metric 100 192.168.5.0/29 dev ens19.5 proto kernel scope link src 192.168.5.1 192.168.50.0/24 via 10.17.17.33 dev ens18 192.168.56.0/29 dev ens19 proto kernel scope link src 192.168.56.1
As expected, deleting a route can be done with…
[root@centos7-router ~]# ip route del 192.168.50.0/24
Or with more specific parameters with multiple routes have the same information.
Enable Routing
Nothing special is needed to be able to route packets for other hosts on a network, just entering the following will allow it..
sysctl -w net.ipv4.ip_forward=1
On Client01, all that’s needed to talk to hosts on the LAN & WAN is a route…
ip route add 0.0.0.0/0 via 192.168.56.1 ##### or ##### [root@client01 ~]# ip route add default via 192.168.56.1
And an appropriate route in this case on both the layer 3 switch and a firewall not included in the network diagram for return traffic.
Enable NAT
firewalld is needed to enable NAT. So install, enable at boot, and start now with…
[root@centos7-router ~]# yum install firewalld [root@centos7-router ~]# systemctl enable firewalld [root@centos7-router ~]# systemctl start firewalld
Once done there, establish a zone policy where…
[root@centos7-router ~]# firewall-cmd --zone=external --add-interface=ens18 The interface is under control of NetworkManager, setting zone to 'external'. success [root@centos7-router ~]# firewall-cmd --zone=internal --add-interface=ens19 The interface is under control of NetworkManager, setting zone to 'internal'. success
And configure NAT masquerading…
[root@centos7-router ~]# firewall-cmd --zone=external --add-masquerade
NAT’s now active and masquerading all traffic from Client01.
When it comes to firewall rules, iptables or firewalld is the answer. Learning either tool is a write-up in itself so I won’t be doing that here. I can say that I never much cared for iptables and I favor firewalld.
Wrap-up
There’s some better solutions out there than configuring things the way I have here. One of the easiest is to just install VyOS where needed. If you want a traditional router look and feel on an existing box though, take a look at Quagga. It effectively turns a Linux host into a router with non of the leg work and without taking anything away from the host. Expect a Cisco like CLI and advanced features like OSPF, BGP, IP-SEC, even OpenVPN. It’s available from most default repo’s and weighs in under 50M installed. For something that feels familiar to iron routers, it’s a great choice. I imagine any CLI router junky will find themselves right at home in it.
I’m interested in seeing what switch OS’s are out there so I’ll be looking at Cumulous here soon. I didn’t mention above, but you can also create a switch or what’s called a Linux Bridge on a linux host too, I just don’t see many use cases as most host these days are getting virtualized anyway. The goal here was just to show that many of the functions of those router/firewall boxes in our racks aren’t so unique. Open Source is arguably more secure than some Black Box with publicly vetted code. IMO, today it makes more and more sense to explore Open Source solutions for network infrastructure needs. At the very least, as someone who works with networks I find the readily available tools help me do my job more effectively. Not just like here with routing, but port scanning with nmap, having ssh, telnet, ftp all ready to run in a terminal without the need for a third party application.