WORK-IN-PROGRESS! FOLLOW WITH CAUTION!

This page provides a description, suitable for following along at home, of how we have been creating the firmware image for the AlixCab and WgtCab and AcctonCab devices that we've been working on.

Historical Context

• The AlixCab and WgtCab are named after the CloneArmyBox and subsequent NewCloneArmyBox (or NuCab) they are meant to replace. These predecessors were based on recycled PCs that we obtained for free and, running Debian GNU/Linux, repurposed as routers/gateway devices for our network installations. The AlixCab and WgtCab are based on the Alix single-board computer (from PC Engines in Zurich Switzerland) and the Netgear WGT634U, respectively. PersonalTelco is (as of late June 2009) part way through a purchase of 30 Alix boards. We already own or have deployed a number of Netgear WGT634Us. PersonalTelco has some Soekris board (typically as MetrixKits) deployed about as well, running OpenWrt firmware. The same firmware (or slight permutations) as described here can also run on the Soekris devices.

  Note that the Netgear WGT634U is a somewhat special device. The device has 8 megabytes of flash storage and has 32 megabytes of RAM, which is rare these days. The image produced with this HowTo will not fit on a 4 megabyte flash device.

OpenWrt

• The respective firmware images for the AlixCab and WgtCab are both based on OpenWrt.

  In order to take advantage of the newest and shiniest features and bugfixes, as well as to add our own packages to images, we build the firmware ourselves rather than relying on specific binary image releases of OpenWrt. If you have (or can get easily) a Linux box with a few development tools, you can do this yourself without much difficulty. You will also need an internet connection.

  OpenWrt is built using a Makefile system called a buildroot. The Makefile infrastructure handles all the details of downloading source code, building the cross-compiling toolchain, compiling the packages and composing image files suitable for flashing (or otherwise writing) onto the device. The buildroot system you obtain from OpenWrt is itself not very large. Instead, it has pointers to the source packages for everything it is going to build, and construction implies retrieving them to a local archive if you haven't already.

  We are typically building the most recent (or very nearly) version from OpenWrt's subversion (svn) tree. We are currently using a git version that synchronizes with the svn tree every 10 minutes.

Bleeding Edge

• Building the current version of a developing code base can sometimes not work. Don't be afraid to report non-workingness, using the Trac ticket system or on the freenode Irc channel #openwrt-devel. Over time, bit by bit, you learn about how the buildroot works and can begin to repair problems yourself and submit patches. Don't be afraid. Try, learn, develop. The advantage of building from the main development trunk over one of the "more stable" trees (such as Backfire) is that we can help ensure that we find bugs in the main development branch so that our firmware continues to work in the future. If we hug too closely to "safety" we risk a longer delay between breakage and detection of that breakage, meaning more work to correct the breakage. So far, this has worked out well. We do sometimes encounter breakage, but if we notice and report it, and work to help fix it, it gets fixed faster and everyone is happy.

Checkout

• There are often many ways of doing things and this is no exception. We'll show how we do it. If you like another way better, and it works for you, feel free to deviate. First thing to do is to check out OpenWrt's current "trunk" buildroot verion. In a directory where you ordinarily build software (we'll use /src in the examples here) and where you have at least several gigabytes of free disk space:

  • cd /src
    git clone git://nbd.name/openwrt.git

  Then, change directories into your shiney new buildroot, what is known in OpenWrt as $(TOPDIR), or in our case, /src/openwrt:

  • cd openwrt

Configuration

• The first thing to do after checking out the buildroot is to set up any feeds you want to use. A feed is basically a URL to a tree of packages that can be built using the buildroot. The primary feed is one called "packages". You almost certainly want to enable that one. Feeds are configured in a file called $(TOPDIR)/feeds.conf. The newly checked out tree will contain a file called feeds.conf.default, which looks like this:

  • src-svn packages svn://svn.openwrt.org/openwrt/packages
    src-svn xwrt http://x-wrt.googlecode.com/svn/trunk/package
    src-svn luci http://svn.luci.subsignal.org/luci/branches/luci-0.8/contrib/package
    #src-svn phone svn://svn.openwrt.org/openwrt/feeds/phone
    #src-svn efl svn://svn.openwrt.org/openwrt/feeds/efl
    #src-svn desktop svn://svn.openwrt.org/openwrt/feeds/desktop
    #src-svn xfce svn://svn.openwrt.org/openwrt/feeds/xfce

  We aren't using anything from the xwrt or luci feeds currently, and we'll change the packages feed to use git instead of svn. So, create a file feeds.conf containing this:

  • src-git packages git://nbd.name/packages.git

  Install the feeds, thusly:

  • scripts/feeds update -a
    scripts/feeds install -a

  The OpenWrt buildroot uses a linux kconfig-style configuration interface and stores the package configuration in a file called .config. Here is a minimal example for the Netgear WGT634U.

  • wget http://www.personaltelco.net/~russell/wgtcab-2011-08-19.config
    mv wgtcab-2011-08-19.config .config
    make oldconfig

  If you want to look around or add/remove packages, do:

  • make menuconfig

  Note that selecting <M> for a package will build the package but not include the package in the image. Selecting <*> for a package builds the package and includes it in the image. Non-included packages can be added later using the opkg utility from the running image. Lastly, custom files can be added to the image by placing them in $(TOPDIR)/files/. These files are copied in to the filesystem tree before the image is constructed, but after any package files are copied. This allows you to override package default configurations. There is not a mechanism at the present time for *removing* files. To generate a suitable tree of files, check out the configurator:

  • cd /src
    git clone git://git.personaltelco.net/files-master
    cd files-master

Read the README. If it is necessary to add your node's information to the nodedb.txt (tab-delimited flat file ... make certain you use real tabs!), it is a good idea to create a git branch for your changes:

• git checkout -b my-new-node
  vi nodedb.txt
  git commit -a 

If you make changes to the nodedb.txt file for your node, be sure to submit those back to the main tree so that your network allocation is reserved for you. Either prepare a patch using git's excellent tools, or arrange for write access to the git repository and push your changes back.

Then run the configurator:

• perl FOOCAB.pl --node <mynodename>

or

• perl FOOCAB.pl --host <myhostname>

Then move the generated tree into the OpenWrt buildroot tree:

• mv output $(TOPDIR)/files/

If you want to be part of the PersonalTelcoVPN (and why wouldn't you?), arrange to get an OpenVpn key from a NetworkOperationsTeam member and add that to your $(TOPDIR)/files tree. To create an openvpn key, log into donk, and run:

• mkvpnclient <myhostname>

Answer the default to everything except "Organizational Unit Name (eg, section) []:" (say "WgtCab" or "AlixCab" as appropriate), and for "Common Name" use the hostname with no domain. Then "y" for "Sign the certificate?" and "commit?" This results in two files: myhostname.key and myhostname.crt, which should be copied to $(TOPDIR)/files/etc/openvpn/keys/.

Finally, because the WgtCabs lack a realtime clock, it is quite possible that the clock won't get sync'd up at boot. In that case, you'll find that OpenVpn will gag on its key if the clock is wrong. The tunnel won't come up until the time range of the key becomes valid. The way we've dealt with this problem is with an init script (/etc/init.d/initclock) that sets to time to within the time range where the key is valid. The convention has been to use the first of the month following the one that the key was created to ensure validity. This will allow the tunnel to come up so that we can log in remotely and fix the clock manually.

Building

• To build the cross-compiling toolchain, packages and image, execute the following command (adjust the -j value as appropriate, 17 works for me on a hyperthreaded quad-core):

  • make BUILD_LOG=1 IGNORE_ERRORS=m V=99 -j17

  If things go reasonably well, the last thing you'll see is a list of:

  • [...]
    Generating index for package ./wireless-tools_29-4_mipsel.ipk
    Generating index for package ./zlib_1.2.3-5_mipsel.ipk
    make[2]: Leaving directory `/src/openwrt'
    make[1]: Leaving directory `/src/openwrt'

  On a quad-core Intel Core i7, the WgtCab build finishes in about 17 minutes, your mileage may vary. You can look in $(TOPDIR)/bin/brcm47xx/ and find the images that were built, and in $(TOPDIR)/bin/brcm47xx/packages/ you'll find the package files you just built.

Flashing

(MORE DETAIL TO FOLLOW SHORTLY)