Tag Archives: Linux

More IPv6 Networking

At the end of a previous post I left myself with two systems talking to each other using IPv6 link-local addresses.  In this post I’m going to describe a couple of methods of applying wider scope IPv6 addresses.

Basic network with link local addresses

Basic network with link local addresses

The above diagram shows the basic network I am using.  Alpha and beta are identical Fedora 19 virtual machines, the only difference is their Ethernet MAC addresses, on both systems the Ethernet interface is enp0s3.  The systems were configured as described in my previous blog post.  I can ping from system to system:

[root@alpha ~]# ping6 -I enp0s3 fe80::2
PING fe80::2(fe80::2) from fe80::1 enp0s3: 56 data bytes
64 bytes from fe80::2: icmp_seq=1 ttl=64 time=0.279 ms
64 bytes from fe80::2: icmp_seq=2 ttl=64 time=0.246 ms
64 bytes from fe80::2: icmp_seq=3 ttl=64 time=0.357 ms
^C
--- fe80::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.246/0.294/0.357/0.046 ms

[root@beta ~]# ping6 -I enp0s3 fe80::1
PING fe80::1(fe80::1) from fe80::2 enp0s3: 56 data bytes
64 bytes from fe80::1: icmp_seq=1 ttl=64 time=0.223 ms
64 bytes from fe80::1: icmp_seq=2 ttl=64 time=0.317 ms
64 bytes from fe80::1: icmp_seq=3 ttl=64 time=0.277 ms
^C
--- fe80::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.223/0.272/0.317/0.040 ms

Note the use of the -I enp0s3 argument to ping6, this is because the addresses I am using are link-local scope only as discussed previously.  I can also connect from system to system using SSH:

[root@alpha ~]# ssh fe80::2%enp0s3
The authenticity of host 'fe80::2%enp0s3 (fe80::2%enp0s3)' can't be established.
RSA key fingerprint is 67:d7:20:6e:f2:e9:25:de:d8:4c:e6:8b:e1:71:f9:4e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'fe80::2%enp0s3' (RSA) to the list of known hosts.
root@fe80::2%enp0s3's password: 
Last login: Fri Sep 6 13:53:08 2013
[root@beta ~]#

One point to note, I had to use the %enp0s3 scoping suffix to the address so that SSH knew which interface to use for the connection.

Prefixes

IPv6 works on the basis of allocated prefixes, these are similar in idea to CIDR subnets in IPv4.  You would normally obtain your prefix from your ISP.  According to the IETF recommendations ISPs should be handing customers /56 prefixes.  This means that out of the 128 bit IPv6 address you would have 72 bits for use in creating your own subnets and host addresses.

Best practice is that you shouldn’t use prefixes longer that 64 bits as this starts to interfere with the automatic address allocation that I’ll discuss later.  The exception to this rule is for point-to-point links where you should use 127 bit prefixes as per RFC 6164.

Given that IPv6 addresses are designed to be global it’s not a good idea to use your own prefixes for examples like those contained in this blog as this could lead to duplicate addresses and prefixes existing.  To solve this problem RFC 3849 defines the prefix 2001:db8::/32 as reserved for use in documentation.  This is the prefix I’ll be using in the rest of this blog post.

Manual Addressing

Allocating an IPv6 address manually to an interface is very simple:

[root@alpha ~]# ip addr add dev enp0s3 2001:db8::1/64
[root@alpha ~]# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether 08:00:27:11:14:ff brd ff:ff:ff:ff:ff:ff
 inet6 2001:db8::1/64 scope global 
 valid_lft forever preferred_lft forever
 inet6 fe80::1/64 scope link 
 valid_lft forever preferred_lft forever

With the address assigned I can now ping it:

[root@alpha ~]# ping6 2001:db8::1
PING 2001:db8::1(2001:db8::1) 56 data bytes
64 bytes from 2001:db8::1: icmp_seq=1 ttl=64 time=0.017 ms
64 bytes from 2001:db8::1: icmp_seq=2 ttl=64 time=0.033 ms
64 bytes from 2001:db8::1: icmp_seq=3 ttl=64 time=0.038 ms
^C
--- 2001:db8::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.017/0.029/0.038/0.010 ms

Note that I don’t need to provide the -I enp0s3 argument, nor do I need to provide the %enp0s3 interface suffix to the address.  This is because the 2001:db8::1/64 address has global scope.  Contrast this with the link-local address fe80::1/64:

[root@alpha ~]# ping6 fe80::1
connect: Invalid argument
[root@alpha ~]# ping6 -I enp0s3 fe80::1
PING fe80::1(fe80::1) from fe80::1 enp0s3: 56 data bytes
64 bytes from fe80::1: icmp_seq=1 ttl=64 time=0.017 ms
64 bytes from fe80::1: icmp_seq=2 ttl=64 time=0.030 ms
64 bytes from fe80::1: icmp_seq=3 ttl=64 time=0.032 ms
^C
--- fe80::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.017/0.026/0.032/0.007 ms

I configured an address (2001:db8::2/64) on the other system which I can also ping, and connect to using SSH:

[root@alpha ~]# ping6 2001:db8::2
PING 2001:db8::2(2001:db8::2) 56 data bytes
64 bytes from 2001:db8::2: icmp_seq=1 ttl=64 time=0.296 ms
64 bytes from 2001:db8::2: icmp_seq=2 ttl=64 time=0.298 ms
64 bytes from 2001:db8::2: icmp_seq=3 ttl=64 time=0.311 ms
^C
--- 2001:db8::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.296/0.301/0.311/0.021 ms
[root@alpha ~]# ssh 2001:db8::2
The authenticity of host '2001:db8::2 (2001:db8::2)' can't be established.
RSA key fingerprint is 67:d7:20:6e:f2:e9:25:de:d8:4c:e6:8b:e1:71:f9:4e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '2001:db8::2' (RSA) to the list of known hosts.
root@2001:db8::2's password: 
Last login: Wed Sep 11 11:03:02 2013 from fe80::1%enp0s3
[root@beta ~]# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether 08:00:27:84:41:4d brd ff:ff:ff:ff:ff:ff
 inet6 2001:db8::2/64 scope global 
 valid_lft forever preferred_lft forever
 inet6 fe80::2/64 scope link 
 valid_lft forever preferred_lft forever

Again note that I don’t need to add the %enp0s3 scope suffix to the ssh command.

Stateless Address Auto-Configuration

Stateless Address Auto-Configuration (SLAAC) is the process by which an IPv6 host can automatically obtain it’s prefix, default gateway, mtu, and other information from a router on the network.  For a Linux system to provide SLAAC services on a network it needs to have the radvd (Router Advertisement Daemon) installed.  I did this on alpha with the command yum install radvd.

Radvd is controlled by the configuration file /etc/radvd.conf.  In this file you specify the interfaces you want radvd to connect to and the options it should use, for example I am using:

interface enp0s3
{
    AdvSendAdvert on;
    prefix 2001:db8::1/64
    {
        AdvOnLink on;
        AdvAutonomous on;
    };
};

Using systemctl I can start radvd:

[root@alpha radvd-1.9.2]# systemctl start radvd.service

I can then connect to the console of beta and bring the Ethernet interface up:

[root@beta ~]# ip link set dev enp0s3 up

And checking the addresses assigned to this interface shows:

[root@beta ~]# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether 08:00:27:84:41:4d brd ff:ff:ff:ff:ff:ff
 inet6 2001:db8::a00:27ff:fe84:414d/64 scope global dynamic 
    valid_lft 86168sec preferred_lft 14168sec
 inet6 fe80::a00:27ff:fe84:414d/64 scope link 
    valid_lft forever preferred_lft forever

From this I can see that the interface has been assigned a link-local address, fe80::a00:27ff:fe84:414d/64, based on the standard link-local prefix of fe80::/64 and the MAC address of the interface converted to an modified EUI-64 address.  I can also see that the interface has a second address which is based on the prefix, 2001:db8::/64, we supplied to radvd and the MAC address of the interface converted to an modified EUI-64 address.  I can use this address to access beta with ping and ssh:

[root@alpha ~]# ping6 2001:db8::a00:27ff:fe84:414d
PING 2001:db8::a00:27ff:fe84:414d(2001:db8::a00:27ff:fe84:414d) 56 data bytes
64 bytes from 2001:db8::a00:27ff:fe84:414d: icmp_seq=1 ttl=64 time=0.797 ms
64 bytes from 2001:db8::a00:27ff:fe84:414d: icmp_seq=2 ttl=64 time=0.325 ms
64 bytes from 2001:db8::a00:27ff:fe84:414d: icmp_seq=3 ttl=64 time=0.357 ms
^C
--- 2001:db8::a00:27ff:fe84:414d ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.325/0.493/0.797/0.215 ms
[root@alpha ~]# ssh 2001:db8::a00:27ff:fe84:414d
root@2001:db8::a00:27ff:fe84:414d's password: 
Last login: Wed Sep 11 12:30:19 2013 from 2001:db8::a00:27ff:fe11:14ff
[root@beta ~]#

If I connected a third host to this network segment and brought up it’s Ethernet interface it would also automatically obtain an IPv6 address with the 2001:db8::/64 prefix and be able to use this to communicate with other systems.

Next Steps

There’s much more to SLAAC than I’ve described above.  It also interacts with DHCPv6 to configure additional information on systems such as NTP.  I’ll cover these topics in a subsequent blog post.

IPv6 Networking with Fedora

Now that I’ve got a better understanding of the recent changes to the Fedora networking stack I have been experimenting with IPv6 networking.  Over the next few years this is going to be a hot topic as we completely exhaust the IPv4 address space, become more aware of the limitations of NAT based solutions, and the Internet of Things starts to take shape.

To get started I did a minimal install of Fedora 19 into a VirtualBox VM which had a single, host only, network interface.  Once the install was complete the first thing I did was to disable and remove both NetworkManager and Firewalld so that I could control the system manually:

systemctl stop NetworkManager.service
systemctl disable NetworkManager.service
systemctl stop firewalld.service
systemctl disable firewalld.service
yum remove firewalld NetworkManager NetworkManager-glib

So that I got consistent device names for my Ethernet devices I also removed the biosdevname package and rebooted:

yum remove biosdevname
shutdown -r now

Once the system came back up the only network interface that was up and running was the loopback interface which had both an IPv4 and an IPv6 address automatically:

[root@alpha ~]# ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc no queue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever

You can see that device lo has the usual IPv4 address of 127.0.0.1/8 and that there is also an IPv6 address of ::1/128 assigned.  IPv6 addresses are 128 bits in length, usually when written down the addresses are broken down into groups of 4 bits which are represented by a single hexadecimal digit.  For example the four bit value 0000 is 0 and the value 0001 is 1.  Groups of 4 hexadecimal digits, representing 16 bits of the IPv6 address, are separated by colons when written down, for example our loopback address is: 0000:0000:0000:0000:0000:0000:0000:0001.  These 4 digit groups are sometimes referred to as hextets.

As you can see writing addresses in this manner is quite unwieldy so there are two mechanisms that allow us to compress the address.  The first is that within a hextet we can strip all of the leading zeros.  For our loopback address this reduces the last hextet from 0001 to 1, which is a slight improvement.  The second mechanism is that the largest, contiguous, group of hextets with all zeros can be replaced by a double colon (::).  In our loopback example that means we can replace all of the zeros with the double colon to get the final address of ::1.

Note that you can only have one occurrence of :: within an address.  If you want to know more, Wikipedia has some more detail.

With this in mind we can then ping both loopback addresses:

[root@alpha ~]# ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time= 0.035 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time= 0.049 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time= 0.049 ms

--- 127.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtf min/avg/max/mdev = 0.035/0.044/0.049/0.008 ms
[root@alpha ~]# ping6 ::1
PING ::1(::1) 56 data bytes
64 bytes from ::1: icmp_seq=1 ttl=64 time= 0.051 ms
64 bytes from ::1: icmp_seq=2 ttl=64 time= 0.053 ms
64 bytes from ::1: icmp_seq=3 ttl=64 time= 0.044 ms

--- ::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtf min/avg/max/mdev = 0.044/0.049/0.053/0.006 ms

Note that for IPv6 you need to use the ping6 command.  It’s a bit unfortunate but only some commands are dual stack, i.e. they take either IPv4 or IPv6 addresses and work as expected – ping isn’t one of these!

The next step is to bring the actual Ethernet interface up and start configuring IPv6 so that we can communicate between hosts and access services.  I can use the ip link show command to list all of the interfaces on my host, in my case I have a single interface called enp0s3 which is currently in the down state. I can bring the interface up with the command ip link set enp0s3 up.

I can then check the state of the interface with the following:

[root@alpha ~]# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 disc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:11:14:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:fe11:14ff/64 scope link
       valid_lft forever preferred_lft forever

The interface is up and interestingly it’s been assigned an IPv6 address.  This is one of the nice features of IPv6, every interface has what’s called a link-local address.  The link-local address is only valid on the particular physical network segment that the interface is connected to, packets to and from this address will not be routed.

Link-local addresses can be manually assigned or automatically generated.  In this case the address has been automatically generated.  To generate the address the host used the assigned link-local prefix fe80::/64  and then created an EUI-64 host address from the MAC address of the interface, these were combined to give the address fe80::a00:27ff:fe11:14ff/64.

It is possible to manually configure the link-local address.  It’s a two step process where first you remove the automatically assigned address, then you add the address you want:

[root@alpha ~]# ip addr del dev enp0s3 fe80::a00:27ff:fe11:14ff/64
[root@alpha ~]# ip addr add dev enp0s3 scope link fe80::1/64
[root@alpha ~]# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 disc pfifo_fast state UP qlen 1000
    link/ether 08:00:27:11:14:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::1/64 scope link
       valid_lft forever preferred_lft forever

Now that the interface has an address I should be able to ping it much like I did with the loopback interface:

[root@alpha ~]# ping6 fe80::1
connect: Invalid argument

The reason that this fails is that the address is link-local, if a host had two interfaces it would be valid for them both to have the same link-local address, so the above command is to ambiguous. The solution is to supply the -I <interface> argument:

[root@alpha ~]# ping6 -I enp0s3 fe80::1
PING fe80::1(fe80::1) from fe80::1 enp0s3: 56 data bytes
64 bytes from fe80::1: icmp_seq=1 ttl=64 time=0.038 ms
64 bytes from fe80::1: icmp_seq=2 ttl=64 time=0.057 ms
64 bytes from fe80::1: icmp_seq=3 ttl=64 time=0.047 ms

--- fe80::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtf min/avg/max/mdev = 0.038/0.047/0.057/0.009 ms

You can also specify  the link/interface by scoping the address.  This is done with by appending a suffix to the address:

[root@alpha ~]# ping6 fe80::1%enp0s3

At this point I’ve got a single host with a single Ethernet interface which has a link-local IPv6 address either manually or automatically configured.  I’ll publish another blog post shortly which will add a second host to the network and hopefully I’ll get them communicating using IPv6.

More on Fedora Networking

Since my first post on the subject I’ve done a bit more digging and acquired a more complete understanding of how devices are named.  It turns out that biosdevname and NetworkManager make up only two-thirds of the story, you also need to factor in the influence of systemd!

NetworkManager is a bit of a red herring, the actual cause of the different names is a conflict between biosdevname and systemd as you can see discussed in Fedora Bugzilla 965718.

You can see this in action if you do a clean, minimal install of Fedora 19 and then do the following:

[root@localhost ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: p2p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 08:00:27:b7:0b:d5 brd ff:ff:ff:ff:ff:ff
[root@localhost ~]# ls /etc/sysconfig/network-scripts/ifcfg-*
/etc/sysconfig/network-scripts/ifcfg-enp0s3
/etc/sysconfig/network-scripts/ifcfg-lo
[root@localhost ~]# yum remove biosdevname
Resolving Dependencies
--> Running transaction check
---> Package biosdevname.x86_64 0:0.4.1-4.fc19 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Removing:
biosdevname x86_64 0.4.1-4.fc19 @anaconda 58 k

Transaction Summary
================================================================================
Remove 1 Package

Installed size: 58 k
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : biosdevname-0.4.1-4.fc19.x86_64 1/1
Verifying : biosdevname-0.4.1-4.fc19.x86_64 1/1

Removed:
biosdevname.x86_64 0:0.4.1-4.fc19

Complete!

Now reboot the system, and when it comes back up:

[root@localhost ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 08:00:27:b7:0b:d5 brd ff:ff:ff:ff:ff:ff

Note that the interface is now named enp0s3 which matches the name of the configuration file in /etc/sysconfig/network-scripts/.

On a server you probably don’t want NetworkManager, though it may be impossible to stem the tide long term (as an aside, I wonder what Red Hat plan for RHEL 7?).  You can remove NetworkManager and the system will just work:

root@localhost ~]# systemctl stop NetworkManager.service
[root@localhost ~]# systemctl disable NetworkManager.service
rm '/etc/systemd/system/multi-user.target.wants/NetworkManager.service'
rm '/etc/systemd/system/dbus-org.freedesktop.NetworkManager.service'
[root@localhost ~]# systemctl enable network.service
network.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig network on
[root@localhost ~]# systemctl start network.service

Between disabling NetworkManger and enabling network.service you might want to manually down all of your interfaces, in my case, ifdown enp0s3.  Or, once you’ve enabled network.service, you could just reboot.

Finally you can remove the NetworkManager and NetworkManager-glib packages using yum.

Fedora Firewall Changes

There’s another networking change that I’ve discovered in Fedora, the replacement of /etc/sysconfig/iptables (and the IPv6 equivalent) with Firewalld.  From reading the Firewalld documentation I can appreciate the use case for laptop and desktop users, however my main interest is servers (not that I’d recommend running Fedora as an OS for production servers) so I’d rather revert back to a static firewall system.  Here’s how to make that change:

yum install iptables-services
systemctl mask firewalld.service
systemctl enable iptables.service
systemctl enable ip6tables.service

At this point you should edit /etc/sysconfig/iptables-config and /etc/systconfig/ip6tables-config to lock down your system.  Once you’re happy with the static firewall rules:

systemctl stop firewalld.service
systemctl start iptables.service
systemctl start ip6tables.service
yum remove firewalld

Notice that there is a small window of vulnerability between stopping Firewalld and starting the iptables services.

Fedora Networking Changes

I’ve been away from Fedora for a few  years now, basically since I got fed up with the maintenance burden of running Fedora on my laptop and switched to a Macbook.  Recently I’ve been playing around again and it seems there’s been a large number of changes over the last few releases.  Of these it’s the changes to networking that have been testing my muscle memory the most.

The first thing that got me was the removal of the ifconfig command.  This command has been replaced by the more powerful, and correspondingly more complex, ip command.  There are a couple of commands I used to run all the time to query the state of the system:

  1. ifconfig -a to show all interfaces and their IPv4 addresses.  The equivalent is ip -4 addr.
  2. ifconfig ethX to show the config of that interface.  The equivalent is ip link show ethX (though I’ll come onto the fact that ethX no longer exists a bit later).
  3. netstat -rn to show the IPv4 route table.  The equivalent is ip route.

You’ll have noticed that I referred to ethX above, this naming convention is no longer used in Fedora.  The system has switched to consistent network device naming which means that devices are now named for their location, for example my virtual machine has two ethernet interfaces which are p2p1 and p7p1.  So the ip command in the example above would actually be ip link show p2p1.  You can disable this feature and get the ethX style naming back by passing biosdevname=0 on the kernel command line.

There’s one more networking change that had me confused for a while.  Previously on a server I would use the ifup and ifdown scripts to bring interfaces up and down, the IPv4 configuration of these interfaces would be controlled by files in /etc/sysconfig/network-scripts/ which had a naming convention of ifcfg-ethX.  When I went looking for these files I expected to find the equivalent, but using the new naming convention, instead I found ifcfg-enp0s3 and ifcfg-enp0s8!

It turns out that even on a minimal Fedora install NetworkManager is configured to control the networking.  NetworkManager has the concept of connections being separate to devices, so the files are used to configure a connection that is assigned to an interface.  You can see this in action by using the command nmcli dev list iface p2p1 and looking for the CONNECTIONS.AVAILABLE-CONNECTIONS line in the output, in my case this refers to enp0s3.  You can use the nmcli command to control the properties of a connection and it’s association with an interface.