Anycast: The Loophole

As I men­tioned in my last post, there’s a loop­hole inher­ent in how routers for­ward your pack­ets around the Internet — specif­i­cally, routers are next-hop for­ward­ing: your data is sent as a stream of pack­ets to a router, and it looks at its rout­ing table and decides where the packet should go next.

A route is just an entry in a table that says “pack­ets for 192.168.250.1 should be sent to this IP address, which will send it to the right place for you.” On the last router before the host, the route says “192.168.22.8 is right here, so just send it to the MAC address asso­ci­ated with that IP straight away.”

So here’s the first half of the loop­hole: Each router decides for itself what the rout­ing table looks like. There’s noth­ing say­ing that two dif­fer­ent routers can’t *both* have totally dif­fer­ent routes to a par­tic­u­lar IP. Whichever route you take depends entirely on which router your pack­ets go through on the way to the des­ti­na­tion. This is inher­ent in how IP net­works work.

Here’s the sec­ond half of the loop­hole: there’s noth­ing that says that an IP address has to be unique to a par­tic­u­lar machine or net­work interface.

Put those two together, and you’ve got the basics of any­cast­ing: two dif­fer­ent servers, in two dif­fer­ent cities, both own 192.168.250.1. The router each server is plugged into knows that IP is right there, and because routers need to know what the other router does in order to be use­ful, each router knows that there is *another* route to that same IP via the router in the other city.

The routers talk either talk to each other via an Interior Gateway Protocol (basi­cally, they each say “these are all the routes I have”), or they’re sta­t­i­cally con­fig­ured to know about each other. You actu­ally have to do this for any­thing to work.

And if your net­work is sane, any over­lap­ping routes have a con­cept of met­rics: that is, how much does it “cost” to go from here to there. The cost of a con­nected route is (by default) 1. The cost of any­thing else is going to be higher — because there’s the other side’s “1”, plus the cost of the link (in this case, the link between the two cities). So whichever router has a bet­ter met­ric is the one that will be used.

But all this is trans­par­ent. You’re still just talk­ing to 192.168.250.1, but that IP is “owned” by lots of servers, and the routers decide which way to send the traf­fic (that is, they do their jobs ).

Now, it’s not quite that sim­ple to imple­ment this. You’ve got to have a sane net­work — one with a prop­erly con­fig­ured IGP — and you’ve got to have some way to “watch” the service.

More on that in the next installment:

1. Anycast: Networking Introduction
2. Anycast: The Loophole
3. Anycast: The Interface
4. Anycast: Handling Routes
5. Anycast: DGRAM vs. STREAM
6. Anycast: IP-SLA HOWTO