|
Size: 16782
Comment: removal of needless complaints
|
Size: 12554
Comment: added dryrot
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 8: | Line 8: |
|
This is a place to tie together all of the information about the ["VPN"] aspects of ["PTPnet"]. In Summer of 2006, JimmySchmierbach and KeeganQuinn spent several weeks planning and testing a design for a system which could allow all PersonalTelco nodes to be logically interconnected. The design allows us to provide a structure that is flexible enough to take advantage of any type of connection, although we focused mostly on IP-over-IP tunnels created with software such as ["OpenVPN"] with glue provided by ad-hoc IP routing protocols like OptimizedLinkStateRouting. |
This is a project which aims to integrate ["VPN"] technology with ["PTPnet"], focusing primarily on permanent IP-over-IP tunnels created with ["OpenVPN"]. Some history and background are available, as are references to configuration data. Keep in mind the instructions here are only geared towards systems running DebianLinux; someone should probably add notes for other types of node. |
| Line 13: | Line 11: |
| == The Good == | == Goals == |
| Line 15: | Line 13: |
| Aside from being a cool idea and a fascinating problem domain from a technical perspective, there are a couple of practical benefits to this: | Aside from being a cool idea and a fascinating problem domain from a technical perspective, there are a couple of practical benefits to the project. |
| Line 17: | Line 15: |
| 1. '''Maintenance''' - Some nodes are trapped behind unfriendly routers doing cone NAT, which prevents the NetworkOperationsTeam from working on them in the usual way. One example is NodeLuckyLab, but there are more than a couple of these out there. (Is there a list of these?) | 1. '''Simplified maintenance''' - Some nodes are trapped behind unfriendly routers doing cone NAT, which prevents the NetworkOperationsTeam from working on them in the usual way. One example is NodeLuckyLab, but there are more than a couple of these out there: NodesBehindNat |
| Line 26: | Line 24: |
| 4. '''IPv6 deployment''' - Tunnel brokers are effectively the only way that most people in this area can obtain significant IPv6 connectivity. It's better than none at all but these tunnels tend to be unreliable and often suffer from high latency. In contrast, we actually have something of a bandwidth surplus, especially when it comes to just getting around town. The conclusion which follows is that if we were to put a bit of serious effort into it, we could easily end up with a faster and more useful IPv6 network of our own, especially if we establish a BGP peer relationship or two at the Internet border rather than falling back to another broker tunnel. | 4. '''IPv6 deployment''' - Tunnel brokers are effectively the only way that most people in this area can obtain significant IPv6 connectivity. It's better than none at all but these tunnels tend to be unreliable and often suffer from high latency. We have especially if we establish a BGP peer relationship or two at the Internet border rather than falling back to another broker tunnel. |
| Line 28: | Line 26: |
| * Similarly, we could potentially get a head start on exploring and building potential applications to run on these networks. | * Similarly, we could potentially get a head start on exploring and building potential applications to run on these networks. |
| Line 31: | Line 29: |
| == Reference == | == Status == |
| Line 33: | Line 31: |
| Here are the places on this wiki where there is currently information on VPNs: | The OLSR `httpinfo` plugin is running on the server: http://donk.personaltelco.net/olsr/ |
| Line 35: | Line 33: |
|
* OpenvpnIpSchemeProposal * OpenvpnIpScheme * OpenvpnNamingScheme * OpenvpnPortScheme * NodesBehindNat * NetworkAddressAllocations |
Also, the current OLSR topology from the server's perspective can be viewed [http://donk.personaltelco.net/olsr_topology.png here]. This image is regenerated every 15 minutes. Currently, the following clients are configured with tunnels: || '''Host''' || '''Node''' || '''["PTPnet"] DNS name''' || || ballista || NodeHawthorne || `ballista.hawthorne.ptp` || || beast || ["NodeTB151"] || `beast.tb.ptp` || || bone || CoreServer || `bone.ptp` || || buffalogap || NodeBuffaloGap || `buffalogap.buffalogap.ptp` || || cantos || NodePowellsTech || `cantos.powellstech.ptp` || || chevy || NodeMississippi || `chevy.mississippi.ptp` || || cornerstone || CoreServer || `cornerstone.ptp` || || cycle || NodeUglyMug || `cycle.uglymug.ptp` || || dryrot || NodeCommunitecture || `dryrot.communitecture.ptp` || || filth || NodeCedarHillsCrossing || `filth.cedarhills.ptp` || || frick || NodeAnnaBannanas || `frick.annabannanas.ptp` || || kong || NodeAnnaBannanasStJohns || none yet || || lester || NodeDivision34 || `lester.division34.ptp` || || loki || NodeCrowBar || `loki.crowbar.ptp` || || number-one || NodeEcotrust || `number-one.ecotrust.ptp` || || serv0 || ["NodeTB"] || none yet || || thehut || NodeOldTownPizza || `thehut.oldtownpizza.ptp` || || vinge || NodePowellsBooks || `vinge.powellsbooks.ptp` || || yeast || NodeRedWing || `yeast.redwing.ptp` || || zeus || NodeBasementPub || `zeus.basementpub.ptp` || Other hosts were also working recently but need minor configuration updates to match the current server configuration; see the `client.conf` example below. |
| Line 43: | Line 62: |
| == Goals == | == How to Help == |
| Line 45: | Line 64: |
| === Vague and not too ambitious === | These are a few of the things that still need to be done: |
| Line 47: | Line 66: |
| The near-term goal would be to actually complete the implementation that has been attempted so many times, then look into applying Jimmy's ideas to whatever extent is possible, and finally document everything here so that it can be maintained and expanded with relative ease. |
* Compatible OLSR configuration for clients is simple but still needs to be documented. * We still need to configure some of the NodesBehindNat as clients. * Operating systems other than DebianLinux should be supported, with documented configuration instructions. * Eventually it would be nice to have all of the systems accounted for by the NodeAudit configured as clients. * People who are not members of the NetworkOperationsTeam should be able to configure tunnel clients. * Some of the information on this page should be referenced from or moved to other pages. |
| Line 49: | Line 73: |
|
=== The polished brass version === Every time it comes up, the idea is always that we should start with NodesBehindNat. It's an easy decision to reach by committee, since it allows you to completely overrule any naysayer Scrooge types by playing the security card, and the folks who just really want a network get told it'll happen sometime soon. Everyone's happy. It's happened that way probably a dozen times with a different group of people each time; rather than listing all of the names or even the most recent batch, just give yourself a pat on the back if you've ever been one of them. The rationale is generally that those systems stand to gain most significantly at first. While this is completely true, and very noble, I'm afraid it has actually slowed down the progress of VPN deployment overall, as a result of the very same factor which is always thought will speed it along: these nodes aren't accessible except to a person with a laptop who must physically travel to each one. So it's not just a matter of a couple of hours with a terminal - it's a couple of days or more traveling all over town trying to make the right things happen. They're always the first ones, so there are always problems which of course don't manifest until later that night, and so these poor nodes get visited over and over by these equally poor guys who are doing their best to make things work. Anyway, I'm not saying you should refrain from treading that well-traveled path, if you're upwardly mobile (read: car owner) and have the will to get out there and do it. Go for it. Send me (KeeganQuinn) an email; I'll help out. What I am saying is that there is really no reason for all of the nodes that have good connectivity to wait patiently on the back burner while the second-class citizen nodes get emancipated. In fact, it seems to me that it makes more sense if the hard-to-reach nodes get their wings later on in the process; if something doesn't go quite right in the beginning, which has happened every single time, it's no trouble to fix it if the node was accessible anyway. If instead that botch means someone has to drive all the way across town again, the amount of time spent goes way out of proportion to the benefit realized, as does the frustration level of our good friend the example volunteer. === The skinny === We've got a whole bunch of nodes that need to get hooked up. A list needs to be made; NodeAudit is probably a good starting point. To my knowledge, there is not a single node anywhere in the city that doesn't need at least one of these things done to it, although relatively few will need the full-service deal. With that in mind, this is a rough description of things that need to happen to each node before that new tunnel goes hot. I will start doing these tasks myself and fill in more details as I go. Be patient - we'll be lucky if we get them all done before 2008. Don't even think about doing any of this just yet unless you are prepared to fix whatever you break. Actually, it's fine by me if your answer to that is going down to the location, ripping the box out of the dusty hole (nearly all nodes live in dusty holes), bringing it to someone who can fix it, watching them do it, then bringing it back to the dusty hole. It's a great learning experience, and you're not likely to make the same mistake twice after it costs you. They don't need to be done in the NodeAudit list order or the middle of the night or any specific thing. A fair percentage tend to break from time to time, because we're neglecting them, because their operational environment is partially submerged and the water is moonlighting as part of a 220V circuit or just because some of them are really crappy old computers - you'd be amazed. Anyway, if you're going to attempt the process at all, don't be timid, just beat the thing up by running through the steps until you're pretty sure there are no more steps - then it'll either be really properly broken or working perfectly. Either way, you'll get three cheers from me. Computers are pretty good at remembering stuff; if you end up on a node that's been done already, it's just going to tell you that, and usually with this type of stuff, it will refuse to do it again. It will also refuse to do stuff at all if it's something that doesn't seem like a good idea to it. As a final word of warning, some of these systems were not configured with major upgrades in mind, and some were only barely adequate to begin with, so keep on the look-out for completely filled hard disks and partitions. If you get stuck with a full disk, do the best you can to get the system to at least keep serving up Internet access and make a note of it on this page. A few of them could use new disks... Anyway, just dive in and have fun, and you'll have the quirks figured out in no time. 1. Most NuCabs will need to be upgraded to etch first. * Some of them will need to be upgraded to sarge before they can be upgraded to etch. Believe it. * Read the upgrade docs in the release notes and just follow the instructions. Seriously. 2. Install the etch kernel after all of the other software updates and get it running. * Technically this is part of the etch upgrade, but it is worth repeating. It's important. * Some nodes will appear to be upgraded to etch already but are actually running sarge or woody kernels. Always check. * We really want our shiny new VPN to work properly after we set it up, and having a dozen different kernel versions trying to interact sanely does not help at all. It will seem like it works fine at first, but eventually things will start acting weird, so just take care of it before it's a problem. The same goes for OpenVPN and olsrd versions; use only the stuff in etch, if it's a NuCab, or you'll wish you had. Last but not least, it would be really nice to get them hooked into the Osiris server. It doesn't all have to happen at once but it will all need to get done eventually, and it's really pretty quick and easy to do it all in one session. You can move quickly; nodes don't get upset or anything if every detail isn't checked and double-checked. We spend a little more time now to save a lot of time later. After those nodes are connected, work should be continued on interconnecting other nodes, as time allows. These are some pretty mundane goals, and they rather make it sound like someone is paying us or something. Some more interesting (although also more long-term) ideas are described in the section about benefits, above. |
Also, the "Goals" section above might provide some ideas for more ambitious projects. |
| Line 87: | Line 78: |
| For now, we are using one central server with several clients connecting to the server, in a classic hub pattern. At some point, we're going to reach an upper limit with this design and we will have to re-evaluate our options given the technology available at that point. | === Design === |
| Line 89: | Line 80: |
| JimmySchmierbach has done some fairly significant planning in anticipation of this eventuality, including some detailed documentation of a design based on a hierarchy with two tiers, referred to as supernodes and nodes. The basic idea was that all of the supernodes would be connected together with tunnels to form a full mesh pattern, then each node would connect to one or more of the supernodes. The original drawings specified the three core servers as the supernodes: cornerstone, bone and alitheia. | For now, we are using one central server (donk) and allowing any number of clients to connect directly over TCP, in a classic hub pattern. |
| Line 91: | Line 82: |
| Someone once wrote here that Jimmy's plan involved the supernodes each being connected to a master node (eg. donk), resulting in a hierarchy with three tiers. That is not correct. donk was never a functional part in the original design. It didn't have to be; the idea was that with three supernodes in a mesh, all bases were covered as long as only one server was ever down at a time. All of the routes would still work because everything was supposed to be redundant. | At some point in the future, we will likely to reach an upper limit with this design and we will have to re-evaluate our approach given the options which are available at that point. Given the nature of the system, the resource that is most likely to be exhausted first is server bandwidth. |
| Line 93: | Line 84: |
|
It has been roughly one year since that design was dreamed up; unfortunately, during that time, alitheia has been completely removed from service and the Subversion repository (which was our most important organizational tool) has been destroyed. bone has also seen some hard times; although it ended up with an upgraded mainboard and some other components, it is now hosted virtually in the same location as cornerstone, which unfortunately forces us quickly to the conclusion that, of the three supernodes in the original design, only cornerstone retains even a chance of being effective in that role. Jimmy's design is really impressive in theory, but I don't recall that we ever actually got it to work with all of the good parts. The VPN clients kept taking naps and the dynamic routing daemons got confused about the fact that we were running a mesh on a layer over their heads. Sometimes machines on the same switch, side by side, would decide that they'd prefer to talk to each other through a big chunk of Internet. It might work better with current software but I'm not really in a big hurry to try it; there's a lot to be said for simplicity, like for example a big fat server that handles everything and actually just works. You want redundancy? Get another big fat server, and just double everything or let it do round robin failover. Simple. Works. Doesn't confuse the software or the people configuring it. This brings us right back around to the first paragraph in this section: we're running the whole show from donk, until it starts to break, at which point we need to look at our options. I propose we add capacity the same way I suggested we add redundancy: more big fat servers. However, that's probably a moot point, since we're nowhere near any kind of capacity limit. I don't expect we will even catch a glimpse of a limit until real users start generating traffic on the tunnel network, which is not likely given our modest selection of near-term goals, outlined in the previous section. At that point, my bet is that the pipe hits a red line before the box does, anyway. |
IP addresses are allocated dynamically by OpenVPN on the server; routing is configured dynamically by OLSR. |
| Line 103: | Line 88: |
| ==== Just checking ==== | Currently, these directions assume that you are a member of the NetworkOperationsTeam, or at least that you have `sudo` access on donk, and that the client system you are configuring is running DebianLinux. |
| Line 105: | Line 90: |
|
Oh, you actually want to set up a link? Are you absolutely positive that everything has been upgraded and the new kernel is running? ==== OpenVPN ==== To generate a new keypair for a client do something like this: |
First you must generate a new private key and certificate. The following series of commands should do the trick; be sure to replace `you` with your username and `thenode` with the hostname of the client. |
| Line 115: | Line 93: |
| ssh you@donk | ssh you@donk.personaltelco.net |
| Line 121: | Line 99: |
| cp keys/thenode.key ~ | cp keys/thenode.crt ~ |
| Line 123: | Line 101: |
| chown you:you ~/thenode.crt ~/thenode.key | |
| Line 126: | Line 105: |
|
Then, do the configuration on the server side - add a file in `/etc/openvpn/ccd` with a name like thenode.personaltelco.net. The contents should be something like (replacing 10.11.255.X with an unused IP within 10.11.255.0/24 from the NetworkAddressAllocations page): {{{ ifconfig-push 10.11.255.X 255.255.255.0 }}} Finally, you must configure the client. Do something like: |
After this initial setup is done on the server, you're ready to configure the client. Note that current versions of the `personal-telco-router` package depend on `openvpn`, so it may not be necessary to install it if you're dealing with a node managed by the NetworkOperationsTeam and everything is up-to-date. In this case, you can skip both of the steps which call `aptitude`. |
| Line 138: | Line 109: |
|
sudo apt-get update sudo apt-get install openvpn cd /etc/openvpn sudo scp you@donk:thenode.* . sudo scp you@donk:/etc/openvpn/keys/ca.crt . |
scp donk.personaltelco.net:thenode.* . scp donk.personaltelco.net:/etc/ssl/certs/ca.crt . sudo aptitude update sudo aptitude install openvpn sudo mv thenode.key thenode.crt ca.crt /etc/openvpn sudo chown root:root /etc/openvpn/* |
| Line 145: | Line 117: |
| Create the clients configuration file at `/etc/openvpn/client.conf`: |
After the key and certificate have been copied to the client, you should delete them from the server. Create the client configuration file at `/etc/openvpn/client.conf`: |
| Line 161: | Line 135: |
| /etc/init.d/openvpn restart | sudo /etc/init.d/openvpn start client |
| Line 164: | Line 138: |
| Now, you should be able to goto 10.11.255.1 from the client and get to donk, or 10.11.255.X (where X is whatever you assigned it) on donk to get to the client. | At this point, if `avahi-daemon` is installed and running, you should be able to reach `donk.local` from the client, or `thenode.local` from donk. |
| Line 166: | Line 140: |
| == Address Allocation == | If you like, you can now install the `olsrd-plugins` package, configure OLSR and join ["PTPnet"]. |
| Line 168: | Line 142: |
| === Servers === | |
| Line 170: | Line 143: |
|
|| Server || 10.11.255.? || Port || Proto || Compression || Dev || || donk || 1 || 1195/udp || OpenVPN || lzo || tap0 || |
== History == |
| Line 173: | Line 145: |
| === Clients === | I'd appreciate if others with different perspectives would take some time to add their own background to this section. (KeeganQuinn) |
| Line 175: | Line 147: |
|
|| Node || Client || Tunnel To || 10.11.255.? || || NodeLuckyLab || luckylab || donk || 5 || || NodeMississippi || chevy || donk || 6 || || NodeCostellos || afterthought || donk || 7 || || NodeCommunitecture || dryrot || donk || 8 || || NodeNorthstar || star || donk || 9 || || NodePowellsTech || cantos || donk || 10 || || NodeTB151 || beast || donk || 11 || |
=== Origins === |
| Line 184: | Line 149: |
| === DNS === | It is difficult, if not impossible, to give credit to any specific source for the idea; many WirelessCommunities have done it before. |
| Line 186: | Line 151: |
|
Each client/server should have an entry in DNS for their VPN IP as a subdomain of vpn.ptp (i.e. donk.vpn.ptp). But, this isn't always as uptodate as it should be... |
Prior to 2006, several attempts were made at interconnecting PersonalTelco nodes over the Internet, although details are not easily found - and perhaps not particularly relevant, anyway. To be sure, the current implementation is not the first to enjoy at least a marginal degree of success. Statically configured tunnels and routes have been used in the past, as well as OSPF dynamic routing implemented by means of Zebra and Quagga. === 2006 === In Summer of 2006, JimmySchmierbach and KeeganQuinn spent several weeks planning and testing a design for a virtual network with the potential to scale to serve the entire city. The results of this project were inconclusive due to problems with software stability, but a complete theory for the design was formulated. The central idea is based on a hierarchy with two tiers, referred to as supernodes and nodes. The basic idea was that all of the supernodes would be connected together with tunnels in a full mesh pattern, then each node would maintain a connection with two or more of the supernodes at all times. Fault tolerance is a central element in this design; actual potential for operation on a large scale remains untested. Jimmy's original drawings specified the three core servers as the supernodes: cornerstone, bone and alitheia. This page previously stated that the design included the idea of supernodes each being connected to a master node (eg. donk), resulting in a hierarchy with three tiers. That is not correct: donk was never a functional part in the original design, and was never connected to the other systems during this project. It would not have provided any notable benefit in acting as an independent tier; with three supernodes in a mesh in the center and a minimum of two supernode connections per node, all of the routes supported by the system could be maintained if any one server failed. The design is really impressive in theory, but when the time came to test it out, things didn't work out so well. The VPN clients kept taking naps and the dynamic routing daemons often got confused about the fact that we were running a mesh on a layer over their heads. Sometimes machines on the same switch, side by side, would decide that they'd prefer to talk to each other through a big chunk of Internet. This type of erratic behavior made it difficult to take the project seriously at the time. === 2007 === Thanks largely to the success of JasonMcArthur and AaronBaer with their wireless mesh network at ArborLodge, KeeganQuinn took another look at deploying a city-wide virtual network in Summer of 2007. It was now evident that the relevant software had reached a degree of maturity which was so clearly lacking a year before. In addition, the topology was simplified to avoid a myriad of potential issues and to speed the process of initial deployment, although a central point of failure was introduced as a consequence. It is difficult to say with certainty which of these factors had the most effect, but the result was that a reasonably stable virtual network could finally be built. |
| Line 191: | Line 172: |
|
Related notes: |
|
| Line 193: | Line 176: |
|
* NodesBehindNat * NetworkAddressAllocations Technical details about the proposed VPN configuration: (not currently being used) * OpenvpnIpSchemeProposal * OpenvpnIpScheme * OpenvpnNamingScheme * OpenvpnPortScheme |
|
| Line 195: | Line 188: |
| CategoryDocumentation CategoryDamnYouKeegan | CategoryDocumentation CategoryDamnYouKeegan CategoryEducation CategoryMan CategoryNetwork |
Personal Telco VPN
Overview
This is a project which aims to integrate ["VPN"] technology with ["PTPnet"], focusing primarily on permanent IP-over-IP tunnels created with ["OpenVPN"]. Some history and background are available, as are references to configuration data. Keep in mind the instructions here are only geared towards systems running DebianLinux; someone should probably add notes for other types of node.
Goals
Aside from being a cool idea and a fascinating problem domain from a technical perspective, there are a couple of practical benefits to the project.
Simplified maintenance - Some nodes are trapped behind unfriendly routers doing cone NAT, which prevents the NetworkOperationsTeam from working on them in the usual way. One example is NodeLuckyLab, but there are more than a couple of these out there: NodesBehindNat
Universal connectivity - it would be ideal if all of our different locations were connected, via radio, laser, fiber, Ethernet, frame relay circuit or whatever you like, forming one big ["PTPnet"] cloud. Unfortunately, the fiber-backed wireless dream mesh isn't quite blanketing the world yet. However, in the meantime we can achieve a similar effect with tunnels.
One related idea would be allowing more users from outside our network to tunnel in, participating as VPN clients. Think PicoPeer.
- Technically, quite easy to do, with the foundation that is already in place. It could be set up a number of ways.
- Several tunnels exist but currently they are all between nodes and specific designated servers; anyone can access the network but only if they are physically present at one of the integrated nodes, which needlessly limits the potential usefulness of the network.
- What about connecting networks that are not nodes? For example, a home with no wireless network, or a block of servers at a company..
Redundancy - More connections mean more bandwidth for everyone. Even tunnels have the potential to supplement direct links. For example:
- Additional bandwidth could be gained in situations where multiple routes through different interfaces are available.
- Fault tolerance is also possible; traffic can be redirected to another path if one interface fails, reducing or even eliminating service interruptions..
IPv6 deployment - Tunnel brokers are effectively the only way that most people in this area can obtain significant IPv6 connectivity. It's better than none at all but these tunnels tend to be unreliable and often suffer from high latency. We have especially if we establish a BGP peer relationship or two at the Internet border rather than falling back to another broker tunnel.
Education - Tunnels give us an opportunity to start acquiring practical knowledge about how to deal with increasing scale in a wide area network, which is going to be invaluable as we begin facing those problems with physical networks.
- Similarly, we could potentially get a head start on exploring and building potential applications to run on these networks.
Status
The OLSR httpinfo plugin is running on the server: http://donk.personaltelco.net/olsr/
Also, the current OLSR topology from the server's perspective can be viewed [http://donk.personaltelco.net/olsr_topology.png here]. This image is regenerated every 15 minutes.
Currently, the following clients are configured with tunnels:
Host |
Node |
["PTPnet"] DNS name |
ballista |
ballista.hawthorne.ptp |
|
beast |
["NodeTB151"] |
beast.tb.ptp |
bone |
bone.ptp |
|
buffalogap |
buffalogap.buffalogap.ptp |
|
cantos |
cantos.powellstech.ptp |
|
chevy |
chevy.mississippi.ptp |
|
cornerstone |
cornerstone.ptp |
|
cycle |
cycle.uglymug.ptp |
|
dryrot |
dryrot.communitecture.ptp |
|
filth |
filth.cedarhills.ptp |
|
frick |
frick.annabannanas.ptp |
|
kong |
none yet |
|
lester |
lester.division34.ptp |
|
loki |
loki.crowbar.ptp |
|
number-one |
number-one.ecotrust.ptp |
|
serv0 |
["NodeTB"] |
none yet |
thehut |
thehut.oldtownpizza.ptp |
|
vinge |
vinge.powellsbooks.ptp |
|
yeast |
yeast.redwing.ptp |
|
zeus |
zeus.basementpub.ptp |
Other hosts were also working recently but need minor configuration updates to match the current server configuration; see the client.conf example below.
How to Help
These are a few of the things that still need to be done:
- Compatible OLSR configuration for clients is simple but still needs to be documented.
We still need to configure some of the NodesBehindNat as clients.
Operating systems other than DebianLinux should be supported, with documented configuration instructions.
Eventually it would be nice to have all of the systems accounted for by the NodeAudit configured as clients.
People who are not members of the NetworkOperationsTeam should be able to configure tunnel clients.
- Some of the information on this page should be referenced from or moved to other pages.
Also, the "Goals" section above might provide some ideas for more ambitious projects.
Methodology
Design
For now, we are using one central server (donk) and allowing any number of clients to connect directly over TCP, in a classic hub pattern.
At some point in the future, we will likely to reach an upper limit with this design and we will have to re-evaluate our approach given the options which are available at that point. Given the nature of the system, the resource that is most likely to be exhausted first is server bandwidth.
IP addresses are allocated dynamically by OpenVPN on the server; routing is configured dynamically by OLSR.
Configuration
Currently, these directions assume that you are a member of the NetworkOperationsTeam, or at least that you have sudo access on donk, and that the client system you are configuring is running DebianLinux.
First you must generate a new private key and certificate. The following series of commands should do the trick; be sure to replace you with your username and thenode with the hostname of the client.
ssh you@donk.personaltelco.net sudo -s cd /etc/ssl/easy-rsa . vars ./build-key thenode cp keys/thenode.crt /etc/openvpn/keys/ cp keys/thenode.crt ~ mv keys/thenode.key ~ chown you:you ~/thenode.crt ~/thenode.key exit
After this initial setup is done on the server, you're ready to configure the client. Note that current versions of the personal-telco-router package depend on openvpn, so it may not be necessary to install it if you're dealing with a node managed by the NetworkOperationsTeam and everything is up-to-date. In this case, you can skip both of the steps which call aptitude.
ssh you@thenode scp donk.personaltelco.net:thenode.* . scp donk.personaltelco.net:/etc/ssl/certs/ca.crt . sudo aptitude update sudo aptitude install openvpn sudo mv thenode.key thenode.crt ca.crt /etc/openvpn sudo chown root:root /etc/openvpn/*
After the key and certificate have been copied to the client, you should delete them from the server.
Create the client configuration file at /etc/openvpn/client.conf:
client remote donk.personaltelco.net 1195 proto tcp-client dev tap ca /etc/openvpn/ca.crt cert /etc/openvpn/thenode.crt key /etc/openvpn/thenode.key comp-lzo
And finally, start OpenVPN on the client-side:
sudo /etc/init.d/openvpn start client
At this point, if avahi-daemon is installed and running, you should be able to reach donk.local from the client, or thenode.local from donk.
If you like, you can now install the olsrd-plugins package, configure OLSR and join ["PTPnet"].
History
I'd appreciate if others with different perspectives would take some time to add their own background to this section. (KeeganQuinn)
Origins
It is difficult, if not impossible, to give credit to any specific source for the idea; many WirelessCommunities have done it before.
Prior to 2006, several attempts were made at interconnecting PersonalTelco nodes over the Internet, although details are not easily found - and perhaps not particularly relevant, anyway. To be sure, the current implementation is not the first to enjoy at least a marginal degree of success.
Statically configured tunnels and routes have been used in the past, as well as OSPF dynamic routing implemented by means of Zebra and Quagga.
2006
In Summer of 2006, JimmySchmierbach and KeeganQuinn spent several weeks planning and testing a design for a virtual network with the potential to scale to serve the entire city. The results of this project were inconclusive due to problems with software stability, but a complete theory for the design was formulated. The central idea is based on a hierarchy with two tiers, referred to as supernodes and nodes. The basic idea was that all of the supernodes would be connected together with tunnels in a full mesh pattern, then each node would maintain a connection with two or more of the supernodes at all times. Fault tolerance is a central element in this design; actual potential for operation on a large scale remains untested.
Jimmy's original drawings specified the three core servers as the supernodes: cornerstone, bone and alitheia. This page previously stated that the design included the idea of supernodes each being connected to a master node (eg. donk), resulting in a hierarchy with three tiers. That is not correct: donk was never a functional part in the original design, and was never connected to the other systems during this project. It would not have provided any notable benefit in acting as an independent tier; with three supernodes in a mesh in the center and a minimum of two supernode connections per node, all of the routes supported by the system could be maintained if any one server failed.
The design is really impressive in theory, but when the time came to test it out, things didn't work out so well. The VPN clients kept taking naps and the dynamic routing daemons often got confused about the fact that we were running a mesh on a layer over their heads. Sometimes machines on the same switch, side by side, would decide that they'd prefer to talk to each other through a big chunk of Internet. This type of erratic behavior made it difficult to take the project seriously at the time.
2007
Thanks largely to the success of JasonMcArthur and AaronBaer with their wireless mesh network at ArborLodge, KeeganQuinn took another look at deploying a city-wide virtual network in Summer of 2007.
It was now evident that the relevant software had reached a degree of maturity which was so clearly lacking a year before. In addition, the topology was simplified to avoid a myriad of potential issues and to speed the process of initial deployment, although a central point of failure was introduced as a consequence. It is difficult to say with certainty which of these factors had the most effect, but the result was that a reasonably stable virtual network could finally be built.
References
Related notes:
Technical details about the proposed VPN configuration: (not currently being used)
CategoryDocumentation CategoryDamnYouKeegan CategoryEducation CategoryMan CategoryNetwork