• MississippiNetworkDiagram

Mississippi Network Diagram and Configuration

Buick serves DHCP, and does round-robin load-balancing between 2 DSL lines.

All of the metrix boxes run ["STP"]. (We should probably pick a root bridge, and configure it. The SW NAYA metrix looks good. The commons metrix got elected by default.)

The whole network is bridged, and in the 10.11.104.0/22 network.

NEWS

November 11, 2005: I think I've figured out the "received packet with own address as source address" messages. They are only appearing on metrix-west and metrix-naya-sw. I think they are a consequence of having bridges on nodes in managed-mode. The master-mode node rebroadcasts frames sent via it, and bridging puts the interfaces in promiscuous mode, so the sender is hearing the rebroadcast. The messages are therefore, presumably, innocuous. --RussellSenior

November 10, 2005: RussellSenior rebooted metrix-commons and metrix-naya-sw to the new kernel, and magically, traffic started to flow between metrix-west and metrix-naya-sw for the first time. However, oddly, connectivity from buick (10.11.104.1) to metrix-west was still severely lossy. Log messages "received packet with own address as source address" are appearing on metrix-naya-sw's /var/log/messages. Some progress, but some bugs still need straightening out.

November 9, 2005: RussellSenior got into the basement and was able to recover metrix-west via ethernet. The problem had to do with modules not loading. Patched that problem in a somewhat kludgy way by adding "pre-up modprobe ath-pci" to the athN stanzas in /etc/network/interfaces. The ath-pci module should have loaded from /etc/modules, but wasn't for some reason. Applied the same fix to metrix-commons and metrix-naya-sw, but haven't rebooted them. Can ping metrix-commons from metrix-west, but not all the way to metrix-naya-sw. Hoping a reboot to the new kernel will correct that.

November 8, 2005: RussellSenior has copied a new kernel, modules and utilities for use with the madwifi-ng drivers over to metrix-west, metrix-commons, and metrix-naya-sw. The /boot/grub/menu.lst file is modified but still pointing at the 2.6.12.3-metrix kernel. Except in the case of metrix-west, which because it was already not connected, we decided to use it as a test case. It rebooted to 2.6.14-metrix (with the madwifi-ng drivers) and is associated with metrix-commons with a nice strong signal on 802.11a, but for some reason its network is not functioning. Same thing with the 802.11b/g radio, association and a nice strong signal from the street, but no network. It isn't pingable from either radio. Going to try to get inside to test from the ethernet tomorrow.

November 7, 2005: RussellSenior is hacking on a metrix image with a new kernel and madwifi-ng drivers, using the metrix we pulled off of Cecily's as a testbed.

• I built a serial-console cable for the metrixes. It consists of a standard serial cable with one end cut off and spliced with three small wires with female connectors on the tips to slip over the male DB9 pins. It is slightly tricky to install, requiring tweezers, some light and a little persistance, but it beats the hell out of disassembling the thing to get at the serial port. The communications parameters for talking to the metrix console are 19200 baud, N81, no flow control. Using this console cable, the three wires are placed as follows (taking care that bending loads on the wires don't cause the conductive parts to touch... probably should insulate the connectors better with some heat-shrink tubing):
  • pin 2 - blue
  • pin 3 - white
  • pin 5 - black
• Observed that wds clients can't associate with non-wds masters, but any client can associate with wds masters.
• I apt-get update ; apt-get upgrade'd the metrix, which installed about half a dozen new versions of things, but not too overwhelming.
• I also apt-get install'd tcpdump and ntpdate.
• I have compiled a 2.6.14 kernel and current svn madwifi-ng drivers and loaded them onto the metrix with rsync. It still can't see my 802.11a AP, presumably because I don't have the madwifi-ng drivers on the AP yet. Grub is not configured to boot the new kernel, so the metrix falls back to the old kernel without manual intervention on the serial console.
• The eth0 interface isn't coming up on boot. But /etc/network/interfaces is a mess right now, so maybe no surprise! An ifup eth0 cures it, but you obviously need a console to do that.
• The madwifi-ng drivers employ a new method of defining interfaces. Physical devices are named wifi0, wifi1, ... wifiN. Interfaces are created with the wlanconfig command, e.g.: "wlanconfig ath0 create wlandev wifi0 wlanmode ap". The associated utilities are installed in /usr/local/bin. Some /etc/network/interfaces changes need thinking through.

October 28, 2005: CalebPhillips, RussellSenior, and MichaelWeinberg replaced the metrix on Cecily's rooftop and re-attached the equipment with real chimney mounting hardware. This node seems to be working just fine now, with good connectivity to Commons. However, currently there seems to be a problem with the switch at Naya that is preventing the network from handing out DHCP or access to the intarweb. The todo list below has the current todo.

Russell and I came back to fix the switch issue and found Buick DOA. We also replaced the switch with one Russell had on hand. Everything seems to work now...except, Cecily's cannot connect to anything past commons in the direction of naya. Specifically, it seems like clients of the commons 802.11a radio (with the omni) cannot see each other (naya-sw and metrix-west are both clients to commons). We are working on an explanation. At this point the network is entirely functional everywhere except metrix-west. - CalebPhillips

October 27, 2005: CalebPhillips and RussellSenior managed to get Ed's roof online. Without access to a ladder we had only one option to make progress, and that was to see if the apparently non-functioning metrix on Ed's roof was actually powered on and accessible from the ethernet. Ed graciously let us in to check. This possibility was suggested by Russell's experience with metrix-naya-sw, where the radios did not initially come up after a reboot. Turns out, the metrix was on. Russell was able to connect via the ethernet, and got a weak signal radio signal on 11g, roughly 7 dB SNR. So the problem wasn't a bad POE connection and it was not a failure to load ath_pci either. Caleb suggested that we might have the antennas backwards. Twice, trying to "ifdown ath0" and the "ifdown ath1" froze the metrix. Russell tried swapping ath0 and ath1 in /etc/network/interfaces, rebooted and bingo, 11b/g started working! SNR in the attic jumped to about 30. However, the 11a backhaul was weak. Pinging to Commons worked, but with about a 40% packet loss. Maybe need to repoint the antenna (isn't there a distance tweak for 11a having to do with an ACK timeout or something, but I thought it was for further than we're talking about here). Caleb and Russell retreated to FreshPot to report success and think. Russell, looking out the window at the backfire already pointing at Ed's from NAYA NW, realized there was a chance that Ed's radio might be able to hit NAYA NW, even though it wasn't pointed directly at it, because it was only 600 or so feet away instead of 1500 feet to Commons. Russell changed the metrix-naya-nw ath0 radio to ESSID backhaul-nw and master mode on channel 161 and turned it on, then walked up the block near Ed's, logged in via 802.11g and reconfigured its ath1 (connected to the 11a antenna) to backhaul-nw and rebooted. Bingo! SNR of about 30 dB. Kind of a chewing gum and bailing wire solution, but it is up and passing traffic. - RussellSenior

TODO

• Why can't clients of commons' 802.11a radio see each other? In other words, why can't naya-sw ping metrix-west or vice versa.
• Ed's is mounted on the chimney, plugged in, talking to metrix-naya-nw. Need to try repointing antenna at Commons roof.
• Metrix-naya-sw currently does not have line-of-sight from Commons mast due to vegetation, but still a reasonable signal, probably want to use metrix-naya-nw for 11a backhaul instead for better line-of-sight, perhaps even replacing metrix-naya-sw with a single-radio device. Another option is to leave both metrixes pointing at Commons. The spanning-tree protocol (STP) should pick one to use and self-heal if one goes down.
• Metrix-naya-nw's ath0 is configured as a master for Ed's on ESSID backhaul-nw, channel 161. If we can get Ed's metrix connected to metrix-commons, then we should change metrix-naya-nw's ath0 into client on the 11a backhaul network and point its backfire antenna at metrix-commons.

• The cisco at 10.11.104.4, supposedly in the SE corner of the NAYA roof, is not reachable from buick. MichaelWeinberg says it is unplugged.

• Add ciscos to the MAC address table below.
• Add access point in Commons courtyard to MAC address table below.
• When last rebooted, metrix-naya-sw failed to bring up its radios automatically. Manual remediation of modprobe'ing ath_pci and ifdown/ifup'ing the ath0 and ath1 interfaces got it working. Not sure why ath_pci didn't load initially. Needs investigating.
• The Sector antennas on Naya might benefit from being repointed...
• Install link to second DSL line (in bookstore) and setup round-robin load balancing
• Need to install a "repeater" at Cecily's location, attached to the ethernet port on the PoE injector.
• Need to install a "repeater" at Amnesia brewing inside and behind the corrogated steel.

Access points

See ChannelFrequencyChart for reasoning about channels, and channel choices.

MAC addresses

Firmware Image

The image for the Metrix boxen is at http://cornerstone.personaltelco.net/~brj/metrix-missnet.img It is setup with all 3 interfaces bridged, and br0 set to 10.11.104.5/22.

To flash:

• Setup a sarge installer pxeboot environment.

  • Instructions at http://www.debian.org/releases/stable/i386/ch04s06.html.en

  • The pxelinux included with debian tries to use both the VGA emulation and the serial console. This causes problems. Use pxelinux.0 from http://centerclick.org/net4801/pxelinux/ instead.

  • Use the serial console pxeboot config.
  • Change all the 9600 to 19200 in the config.
  • You probably want to add DEBIAN_FRONTEND=text to the boot options of the kernel you'll use. Use expert26.
• Netboot the metrix into the installer
  • Attach a serial terminal.
  • 19200 baud, 1 bit parity, no flow control
  • Press ctrl+p at the prompt to get to the BIOS

  • boot f0

  • Use expert26. You don't need any extra options.
• Go through the installer up through step 9 (Download installer components). This is where device drivers are loaded.
• After the drivers are loaded, go to a shell.

• wget -O - http://somewhere/foo/metrix-missnet.img | dd of=/dev/discs/disc0/disc bs=1M (don't get it straight from cornerstone, copy it to your machine first)

• Reboot

Note: as of 2005-10-11 this process isn't working for me. The Metrix stalls during the PXELINUX stuff. I think it's probably just an issue with my setup but I've got it working fine using the documentation from Metrix. (- KeeganQuinn)

The way the Metrix website says to do it is actually better. When I was working on the metrixes, though, that documentation didn't exist, to my memory. The debian netboot was simply the easiest way I could find to get a shell netbooted. (- BenjaminJencks)

Can KeeganQuinn or BenjaminJencks please point at or reiterate the referred-to Metrix documentation here? --RussellSenior

[CategoryDocumentation]