Anycast: The Interface

5433595964_0e491e7cfc_b

In pre­vi­ous parts of this series, I dis­cussed the basics of packet net­work­ing and how IP rout­ing can be hacked to pro­vide a geo­graph­i­cally redun­dant IP. In this install­ment, I’ll talk about one way to con­fig­ure an any­cast IP on a server, and why you want to do it that way.

Let’s say you want to cre­ate an any­cast IP at 10.10.10.10. You want to have a server in two dif­fer­ent cities that each “own” that IP. The best way to do this is to con­fig­ure the Anycast IP on a loop­back alias, instead of on a reg­u­lar interface.

You do this on RHEL-based sys­tems fairly triv­ially, by cre­at­ing a new file:

$ cat /etc/sysconfig/network-scripts/ifcfg-lo:10
DEVICE=lo:10
ONBOOT=yes
IPADDR=10.10.10.10
NETMASK=255.255.255.255

So, once you’ve got­ten that inter­face con­fig­ured, just ensure your ser­vice is lis­ten­ing to it. One thing to keep in mind here is that we’re cre­at­ing an alias to the typ­i­cal lo loop­back inter­face. This inter­face is the one that has the 127.0.0.1 address on it. This is inter­est­ing because it brings up another corol­lary with other things: there’s noth­ing spe­cial about any par­tic­u­lar IP address. 192.168.*.*, and 127.0.0.1 are only spe­cial by con­ven­tion. Similarly, there’s noth­ing spe­cial about any given Layer 3-capable inter­face, includ­ing the loop­back device. You can just as eas­ily add an alias to the loop­back inter­face with a non-local IP (as we’ve done here).

A good anal­ogy is ASCII encod­ing: there’s noth­ing inher­ently spe­cial about the num­ber 65 that ties it to “A”, it’s just a stan­dard­ized con­ven­tion we agree use. Likewise, there’s noth­ing spe­cial about any given char that pre­vents it from hold­ing any given num­ber between –128 and 127.

At any rate, the next step is to add a sta­tic route to your router directed at that IP. On a Cisco router, this would look like this:

RouterA# ip route 10.10.10.10 255.255.255.255 10.20.20.20 name dns.example.org

…assum­ing your “real” net­work card (that is, eth0) has 10.20.20.20 as it’s IP address. That’s it, you should now be able to reach your ser­vice from the Anycast IP.

So why do you want to do it that way, rather than just give the 10.10.10.10 address directly to a sec­ondary net­work inter­face (i.e. dif­fer­ent card)? Turns out there are some oper­a­tional rea­sons for this:

  1. The router will keep it’s route to your Anycast IP as long as there’s an active inter­face on the right VLAN. What this means in prac­tice is that it’s often impos­si­ble to remove the route to 10.10.10.10 with­out totally shut­ting down the net­work card (that is, not just run ifdown on the inter­face, but modprobe -r on the dri­ver mod­ule). Depending on your router/switch topol­ogy, even that may not work to remove the route.
  2. You don’t need to “waste” another port on your switch. A decent 48-port, 10/100/1000, man­aged, Cisco switch costs about $2,500 online. That means each port costs $50, so don’t use them if you don’t have to.

You may have noticed the prob­lem with oper­a­tional rea­son­ing #1: namely, how do you remove a sta­tic route once you’ve con­fig­ured it on your router?

Some ways to do that are cov­ered in the next installment:

  1. Anycast: Networking Introduction
  2. Anycast: The Loophole
  3. Anycast: The Interface
  4. Anycast: Handling Routes
  5. Anycast: DGRAM vs. STREAM
  6. Anycast: IP-SLA HOWTO

Leave a Reply

*