[Interop-dev] hello new subscribers

[Nemesis]

Hey Matthias,

first of all, thank you for your precious feedback.

On 05/31/2015 06:16 PM, Matthias Schiffer wrote:
> On 05/31/2015 03:09 PM, Nemesis wrote:
>> On 05/26/2015 02:16 AM, Matthias Schiffer wrote:
>>> On 05/19/2015 06:18 PM, L. Aaron Kaplan wrote:
>> [...]
>>>> But... this being said, now it's your turn for the new members to this list to speak up!
>>>> Do you want to briefly introduce yourselves?
>>>> What brought you here? What motivates you? What do you want to see done? What do you want to contribute?
>>>>
>>>>
>>>> Thanks,
>>>> Aaron.
>>> Hello interop-dev!
>>>
>>> I'm Matthias (NeoRaider on Github / in IRC) from the Freifunk project of
>>> Lübeck, Germany. Frederico suggested me to join the list at the Wireless
>>> Community Weekend in Berlin last week.
>>>
>>> I'm one of the core developers of the Gluon firmware framework used by
>>> many German Freifunk communities. As we're also developing visualization
>>> systems in and around Gluon, we're in the constant process of inventing,
>>> refining and extending information formats like those NetJSON tries to
>>> define. The other Gluon core developer Nils, who has joined this list as
>>> well, is the original author of the ffmap and MeshViewer projects used
>>> by many Freifunk communities for visualization. Long story short, we'd
>>> love to make our tools more interoperable by making Gluon emit NetJSON
>>> and add NetJSON support to ffmap/MeshViewer :)
>> Welcome you all, I think we are all very happy to have you on board.
>>
>>> You can see parts of our current data formats at [1] and [2] (look for
>>> the entry "68:72:51:20:db:cc" to see the current version, most nodes
>>> aren't updated to the latest Gluon version yet). Some information in our
>>> current format is redundant as we're trying to provide backwards
>>> compatiblity for at least a few Gluon versions. We're also currently
>>> replacing the alfred/batadv-vis-internal topology information format
>>> with an own extensible JSON format (to make it independent of alfred and
>>> generally more flexible).
>>>
>>> As you can see, many parts of our "nodeinfo" and "statistics" are
>>> already similar to the "Network Device Configuration" and "Device
>>> Monitoring" NetJSON objects. For the parts that aren't, I hope we'll
>>> find time soon to think up proposals how to extend NetJSON to support
>>> all of our usecases :)
>>>
>>> Matthias
>>>
>>>
>>> [1] http://metameute.de/~freifunk/alfred/nodeinfo.json
>>> [2] http://metameute.de/~freifunk/alfred/statistics.json
>> Amazing to notice you came up with a format that is not too different
>> from the NetworkDevice, which is a synthesis of the data one can get
>> from ubus, nodewatcher-agent, openwifi map and the nodeshot API.
>>
>> Well the next thing is to start using it in the real world. We'll be
>> doing that in the next couple of months, hopefully if other people do it
>> we can gather feedback and talk about it in person at the battlemesh and
>> CCC.
>>
>> Federico
>>
> We now have the first few nodes using our new topology format online.
> The current output can be found at
>
> http://metameute.de/~freifunk/alfred/neighbours.json
>
> As you can see, our format has some important differences from the
> Network Graph format in NetJSON: each "record" only shows the topology
> as seen from a single node, all nodes' records together are required to
> see the whole topology. This difference is natural as in the metrics we
> use (batman-adv TQ, WLAN signal/noise levels), only information about a
> node's direct neighbourhood is available. It doesn't match the Network
> Routes format well either, as our format doesn't contain information
> about the chosen routes to distant nodes either.

By looking at that JSON document it seems to me that it could be
converted to a NetworkGraph object. So it could be either be returned by
the firmware as an alternative format or it could be converted by a parser.
But I might be wrong, I should try implementing it to be sure.

> Some other notes about differences between our formats and the NetJSON
> draft:

One premise: I see that many points are about missing info, or info that
is not interesting for you.
That should not be a big issue, because you can add custom properties if
needed, or you might omit optional data if it's not interesting. I'll
explain more in detail below.

> * Our formats always contain a node_id attribute, so for
> statistics/neighbours the corresponding nodeinfo can be found

What does the format represent exactly? The topology as seen by some nodes?

In order to understand thoroughly I also need to ask: what kind of id is
node_id exactly? A database id or some kind of hash? How is it generated?

> * The Network Graph format doesn't contain any information about the
> specific interfaces a link belongs to on nodes with multiple interfaces,
> that might be especially interesting when there are different interface
> types (different WLAN bands/channels, wired, VPN tunnel, ...)

This data might be added in custom members or in the properties object.

If more people find this useful, we can have a chat with routing
protocol devs to know if their software would be able to output a
complete NetworkGraph object including this type of data, so that we
might define this data as part of the spec.

> * I find it a bit strange that Network Routes can be included in a
> Device Configuration. Is this supposed to contain static routes only?

Yes.

At the moment this is also the only part of DeviceConfiguration I'm
notreally satisfied with and I would like to ammend.

> Otherwise, including this into Device Monitoring would be more appropriate

Definitely

> * I won't go much into detail about differences between nodeinfo and
> Device Configuration. Much information in the Device Configuration
> example looks much too verbose in my eyes. Especially the interface
> configuration parts contain many attributes that aren't very useful
> (have just been copied from the netifd output?)

It has been inspired by ubus and nodewatcher-agent.

But now that you pointed it out you helped me realize something really
important.

If we go back in the archive (i'm actually looking for the emails but
can't find them), we'll find out that we wrote that in the
DeviceConfiguration and DeviceMonitoring all (or most of) the blocks
should be optional.

But that is not written in the spec so I should definitely add it right
now to clarify, what do you think?
I left out the details of DeviceConfiguration and DeviceMonitoring from
the spec because I wanted to get some more feedback before writing them
down.

> * The traffic information in our statistics format is supplied by
> batman-adv over the ethtool interface. It contains some interesting data
> like the amount of management traffic, and discerns forwarded and
> locally transmitted/received traffic, but no interface-specific information

I see, is the statistic format one of the previous files you linked?

> * The monitoring format shouldn't contain any information about
> interfaces not used for meshing. This is important so "monitoring"
> doesn't turn into "surveillance" when someone uses a single OpenWrt
> system both as a community mesh node and a private router (which is
> something we plan to support with Gluon some time in the future)

I think the implementation of the format is in charge of deciding what
to do with it.

For example: I work in a municipal wifi setting, we mostly don't use
mesh on our nodes, but we do need a format that describes this data.

If your firmware doesn't need this it can simply omit it, but I believe
this misunderstanding is caused by the fact that the specification
doesn't clearly states that all the blocks are optional.

What do you think?

Just to clarify: the format should not dictate/impose to
expose/send/collect sensitive data. It just describes how to represent
data, each implementation will decide how to use it. The important thing
is that we find a way to output and parse this data in a standard (and
possibly easy) way.

Keep up the good work
Federico

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.funkfeuer.at/pipermail/interop-dev/attachments/20150531/1573b5e5/attachment.sig>