Differences between revisions 12 and 13
← Revision 12 as of 2006-01-24 14:49:50 →
Size: 5090
Editor: CalebPhillips
Comment: updated with the latest
← Revision 13 as of 2006-01-30 18:48:03 →
Size: 5118
Editor: CalebPhillips
Comment: linkified kristin
Deletions are marked like this. Additions are marked like this.
Line 45: Line 45:
 * BottleNeck code by KristinTufte  * BottleNeck code by [http://cs.pdx.edu/~tufte/ KristinTufte]

Introduction

As a small research project at PSU for a course in [http://www.cs.pdx.edu/~tufte/410-510DS/ Data Streams], I am going to write some proof-of-concept software that could be used to collect real-time PTPNet-wide statistics using a combination of:

  • A Data Streams Management Server (DSMS)

  • A "Parasite" program that "plugs into" NoCat to report auths and evictions back to the Server

  • A Client that queries that Server and displays the information in a useful way.

The completed project will be licensed under an open-source license for any or all to use. Also, emphasis will be put on generalization, as such an application could certainly be useful for other distributed systems that are similar to the PTP Network.

Initially I will focus on getting the parasite to run on an AP running OpenWRT with NoCatSplash, simply because of the cheapness and ubiquity of this combination.

Progress

  • 2005-11-03: Today I am turning in the project proposal to my professors: [http://smallwhitecube.com/toil/programming/files/datastreams/proposal.pdf (PDF)] [http://smallwhitecube.com/toil/programming/files/datastreams/proposal.dvi (DVI)] [http://smallwhitecube.com/toil/programming/files/datastreams/proposal.tex (TEX)]. I also created this Wiki page today.

  • 2005-11-15: I have finished the code for the first parasite (in 'C' - wow I forgot how much I love 'C'), which is meant to listen to the stderr output of NoCatSplash and send tuples downstream to the DSMS. I will ultimately compile it into an OpenWRT package that can optionally be installed on top of the NoCatSplash package. On the server side of things, my Prof. is adding some network code to the server that will accept connections from clients and feed the input into the server for processing. My next goal is to develop a "node simulator" that should stochastically simulate the output of a bunch of imaginary parisites. This simulator will help with the development of the client.

  • 2005-11-18: Finished the simulator. It is written in ruby and is pretty simple. It will simulate an arbitrary number of nodes with an (initially) random number of clients. Clients come, leases expire. A pretty simple approximation, but it will do for preliminary testing. Now I must wait for my Prof. to finish the network code so I can start connecting the pieces together.

  • 2005-11-30: The network code is finished. I expect to do some serious hacking in the next week or so.

  • 2005-12-02: The parasite code is essentially done. I have made OpenWRT packages for the parasite and a modified (fixed) nocatsplash. I have written some plumbing so that the parasite can plug into ["WiCap-PHP"] too. I have formalized the schema and the queries. I have tested the all the parts from the parasites up to the DSMS. All that's left is the client side of the DSMS.

  • 2005-12-04: The client side of things is mostly done. I have put together a small webpage that displays realtime information to the user. This information is gleaned from an actively running query against the DSMS. Periodically a meta-client saves the stream signature to a datafile that is used (synchronously) by the web-based application. The proof-of-concept website is [https://cs410ds.smallwhitecube.com here] (although the DSMS isn't always running so it doesn't necissarily show "live" data).

  • 2006-01-24: Finally found some time to update this page with the latest content. The final project writeup can be had here: [http://smallwhitecube.com/toil/programming/files/datastreams/final.pdf (PDF)] [http://smallwhitecube.com/toil/programming/files/datastreams/final.dvi (DVI)] [http://smallwhitecube.com/toil/programming/files/datastreams/final.tex (TEX)]. Also the source code can be downloaded [http://smallwhitecube.com/toil/programming/files/datastreams/project-alpha.tar.gz here]. This tarball contains everything except the source for NiagaraST, the datastreams server. The code for NiagaraST was developed under a grant, and it will be released as Free (as in freedom) Software in the not-too-distant future. For the time being, I cannot legally distribute it.

Download

Demo

There is a demo of the proof of concept [https://cs410ds.smallwhitecube.com here]

Credits

SystemWideStatistics (last edited 2007-11-23 18:02:19 by localhost)