|

I’m set­ting up an iso­lated net­work for peo­ple to test inter­nal appli­ca­tions on, since the devel­op­ers all have Sun work­sta­tions with a dual-port Gigabit NIC on the moth­er­board, and we’ve got a bunch of older net­work equip­ment that we haven’t got­ten around to eBay­ing yet. What I’m doing is link­ing the sec­ond NICs together with some vir­tual machines and the older net­work equip­ment to cre­ate a sep­a­rate devel­op­ment network.

The devel­op­ment net­work is a full Layer-3 net­work run­ning an IGP between mul­ti­ple nodes with attached client boxes. This allows me to play around with a decent lab net­work, and pro­vides devel­op­ers with a way to dis­cover that Linux sets the TTL of mul­ti­cast pack­ets to “1” well before they are called to explain why their appli­ca­tion didn’t work even after loads of test­ing, spend 8 hours play­ing head-desk, and finally start ques­tion­ing me about fire­walls on our inter­nal net­work, forc­ing me to claw it out of them that they are dri­ving mul­ti­cast with­out a license and explain how to use tcpdump.

Not that I’ve had to do that a dozen times now, or any­thing…
Read the rest of this entry »

Aside from the laugh­able idea of “mil­i­tantly” sup­port­ing any­thing with a blog post, Miguel sim­ply noted that these peo­ple exist, have writ­ten a book, and will be doing the speaking-tour-thing near him. Does he agree with the con­tents? (shakes eight-ball) Signs point to Yes.

Is he free to do so? Also yes.

Are you free to ignore him? Still yes.

Does His Chomskiness actu­ally take the chal­lenge and pro­vide a bet­ter rebut­tal to the under­ly­ing book than politely demand­ing Miguel STFU? Yep.

Oh, and here’s a patch that will let you do some­thing cool with XEN 3.0.3:

--- network-bridge      2007-02-08 09:21:12.000000000 -0600
+++ network-vlans       2007-09-14 09:55:20.000000000 -0500
@@ -26,6 +26,7 @@
 # bridge     The bridge to use (default xenbr${vifnum}).
 # netdev     The interface to add to the bridge (default eth${vifnum}).
 # antispoof  Whether to use iptables to prevent spoofing (default no).
+# vlans      VLANs to add on top of the bridge
 #
 # Internal Vars:
 # pdev="p${netdev}"
@@ -64,18 +65,27 @@
 bridge=${bridge:-xenbr${vifnum}}
 netdev=${netdev:-eth${vifnum}}
 antispoof=${antispoof:-no}
+vlans=$(echo $vlans | sed -e 's/,/ /g')

 pdev="p${netdev}"
 vdev="veth${vifnum}"
 vif0="vif0.${vifnum}"

 get_ip_info() {
-    addr_pfx=`ip addr show dev $1 | egrep '^ *inet' | sed -e 's/ *inet //' -e 's/ .*//'`
+    addr_pfx=`ip addr show dev $1 | sed -n 's/^ *inet \(.*\) [^ ]*$/\1/p'`
     gateway=`ip route show dev $1 | fgrep default | sed 's/default via //'`
 }
+
+is_bonding() {
+    [ -f "/sys/class/net/$1/bonding/slaves" ]
+}
+
+is_ifup() {
+    ip link show dev $1 | awk '{ exit $3 !~ /[< ,]UP[,>]/ }'
+}

 do_ifup() {
-    if ! ifup $1 ; then
+    if ! ifup $1 || ! is_ifup $1 ; then
         if [ ${addr_pfx} ] ; then
             # use the info from get_ip_info()
             ip addr flush $1
@@ -206,8 +216,8 @@
        mac=`ip link show ${netdev} | grep 'link\/ether' | sed -e 's/.*ether \(..:..:..:..:..:..\).*/\1/'`
        preiftransfer ${netdev}
        transfer_addrs ${netdev} ${vdev}
-       if ! ifdown ${netdev}; then
-           # If ifdown fails, remember the IP details.
+       if is_bonding ${netdev} || ! ifdown ${netdev}; then
+           # Remember the IP details if necessary.
            get_ip_info ${netdev}
            ip link set ${netdev} down
            ip addr flush ${netdev}
@@ -223,6 +233,18 @@
        add_to_bridge  ${bridge} ${vif0}
        add_to_bridge2 ${bridge} ${pdev}
        do_ifup ${netdev}
+
+       if [ -n "$vlans" ]; then
+               vconfig set_name_type VLAN_PLUS_VID_NO_PAD
+
+               for vlan in $vlans; do
+                       create_bridge xenbr${vlan}
+
+                       vconfig add ${bridge} ${vlan}
+                       setup_bridge_port vlan${vlan}
+                       add_to_bridge xenbr${vlan} vlan${vlan}
+               done
+       fi
     else
        # old style without ${vdev}
        transfer_addrs  ${netdev} ${bridge}
@@ -262,6 +284,20 @@
        ip link set ${netdev} name ${vdev}
        ip link set ${pdev} name ${netdev}
        do_ifup ${netdev}
+
+       if [ -n "$vlans" ]; then
+               for vlan in $vlans; do
+                       if [ -n `ip link show vlan${vlan} | grep '${bridge}\:'` ]; then
+                               ip link delif ${bridge} xenbr${vlan}
+                               ip link set ${bridge} down
+
+                               ip link set vlan${vlan} down
+                               vconfig rem ${bridge} ${vlan}
+                       fi
+               done
+
+               vconfig set_name_type DEV_PLUS_VID_NO_PAD
+       fi
     else
        transfer_routes ${bridge} ${netdev}
        ip link set ${bridge} down

It may be buggy, since I haven’t tested it in pro­duc­tion. What it does is this: allows you to run an 802.1Q trunk into your XEN server, then put your vir­tual machines on any VLAN you want with a cou­ple con­fig­u­ra­tion stanzas.

So, your xend-config.sxp will have:

(network-script 'network-vlans netdev=eth0 vlans=8,9,10,11,13,121,14,15')

Which trans­lates to “cre­ate bridges for VLAN 8, 9, 11, 13, 121, 14, and 15 with a xenbr pre­fix”. Then you set your DomU vif stanza to be “bridge=xenbr13” and bam! your DomU exists on the VLAN13. The pri­mary lim­i­ta­tion of this is that it keeps your Dom0 on the untagged/native VLAN, which isn’t best practice.

The stack of mod­ules a packet tra­verses to get to a DomU will look like this (with rel­e­vant modules):

[network] -->
dom0: peth0 (dev) -->
dom0: xenbr0 (bridge) -->
dom0: vlan13 (dot1q attached to xenbr0) -->
dom0: xenbr13 (bridge) -->
dom0: vifX.0 (netloop) -->
domU: xen0 (xennet)