[Interop-dev] a crazy little thing called yaffmap

[Dennis Bartsch]

Hello Community,

I am from Freifunk Berlin and would like to introduce to you an implementation of a node database I started with a friend we call yaffmap. 
It got its name because at that time we have had many mapservers in Berlin and so it started as yet another approach for a Freifunk map.

The intention was to make a server that not just produces points and lines (nodes and their links) but to gather all information that might 
help to understand why a link is as bad as it is. This includes to gather wireless scan results, the effective rate chosen/calculated by the 
wireless driver to a specific neighbour and so on. Furthermore it had to be independent from the routing-protocol and its daemons (but 
needs it to gather useful info) and the IP version (or even no IP version for RPs like batman) and had to be able to upload and store data 
from multiple routing protocols on the same node. In order to sample so much information we went the route of scripting an agent for the 
map-server which runs on the nodes gathering the information and uploading it through a JSON interface to the server. For link-state-protocols 
like OLSR we even implemented the upload of the global topology to the server, which gave us some headache. From the beginning on 
I stressed the need for decentralized operation, so replication between the servers was implemented and any community which wants can 
have their own map data server. Moreover I cleary wanted the datacollection/storage to be independent from the frontend the map user is 
presented. In Berlin everytime a new map came along and an old one was gone we saw huge ammounts of data simply disappear. So the 
server only provides a SOAP interface for UIs and other services to use to get map data. The representation of a node in the database ist 
best seen on a graphic. I uploaded it to imageshack, see [1]. A little bit of documentation is found under [2], but wasn't thoroughly updated 
after we left concept stage.

We got to the point where the agent (bunch of shell/AWK) for the nodes runs on openWRT (best with madwifi and ath9k/ath5k) with olsrd 
(other routing daemons simply need scripts for data gathering, agent is held modular), uploades many useful information to the server (or 
another if the first does not respond; multiple can be configured) using JSON, where it gets stored correctly into a SQL database and replicated 
to other servers and provided through a SOAP interface. The agent even is already provided as a package in the respected PBerg Freifunk 
Firmware. There also exists a 'proof-of-concept' implementation for a map frontend, see [3]. The code is hosted on github as seen und [4]. 
Because of the frontend development got stuck the frontend SOAP interface is not very sophisticated nor much tested as is the frontend itself.

The result is a database which can answer more complex question to the mesh engineer or software developer than every I took a look at. 
Maybe you want to know the average effective rate on all 11n interfaces or all 5GHz interfaces? Or you might be interested in the average 
ETX on different channels in a certain geographic area. Or you want to compare how different routing daemons evaluate the quality of a 
certain link. Maybe you even want to draw a noise map. Or wouldn't it be interesting to see the correlation between the effective rate of a 
wifi link and the metrics which result in the routing daemons?

Because the node data includes a 'misc' field through which any kind of node id or statistics could be stored/sent this is can even be integrated 
into existing community portals.

Hope you find this interesting
Dennis

[1] http://imageshack.us/f/21/bildschirmfotovom201204.png/
[2] http://wurststulle.dyndns.org/yaffmap/trac/wiki/DocumentationIndex  (Be patient. It is dripping through an ADSL upload)
[3] http://yaffmap.basicinside.de/vendor/yaffmap-map/
[4] https://github.com/wurststulle/yaffmap-backend 
    https://github.com/wurststulle/yaffmap-agent 
    https://github.com/wurststulle/yaffmap-map
[5] some statistics: http://wurststulle.dyndns.org/yaffmap/
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.funkfeuer.at/pipermail/interop-dev/attachments/20120425/47b23954/attachment.html>