[Interop-dev] Improving NetworkGraph and NetworkRoutes [was: hello new subscribers]

[Nemesis]

I had a quick chat with Henning to point out a few minor issues.

I think he already updated his implementation.

Well I have to say thank you very much for being so fast.

If we could find someone that would be willing to develop a simple
prototype of a visualization tool we might gain momentum.

Federico

PS: and sorry to simon for the tons of emails


On 05/31/2015 07:14 PM, Henning Rogge wrote:
> Hi,
>
> I updated the json_for_networks plugin for olsrd2 according to this discussion.
>
> I have attached the output for a fully-meshed three node (VM) mesh
> with node 1 being a IPv4 gateway to the internet.
>
> Henning
>
> On Sun, May 31, 2015 at 5:48 PM, Nemesis <(spam-protected)> wrote:
>> On 05/31/2015 05:07 PM, Henning Rogge wrote:
>>> On Sun, May 31, 2015 at 4:39 PM, Nemesis <(spam-protected)> wrote:
>>>> The goal in my opinions are:
>>>>
>>>> visualization: develop awesome visualization tools that work with any
>>>> routing protocol.
>>>> monitor changes in topology: to develop (hopefully) one (or two in case some
>>>> people are not happy with the first attempt) reusable parser implementation
>>>> to update the links in the node-databases
>>> Visualization, monitoring and debugging (what the hell happens with
>>> our traffic).
>> +1
>>>> NetworkRoutes
>>>> For NetworkRoutes, what kind of tools can we achieve with it? What is
>>>> pushing us to define it?
>>>> I would say that having a standard structure is a good idea, we can also
>>>> reuse this structure in DeviceConfiguration to describe static routes (as we
>>>> are already doing).
>>> NetworkRoutes show us the "output" of the routing protocol generated
>>> by the data inside the NetworkGraph.
>>>
>>> Without the routes it is hard to see what is really going on.
>> Excellent.
>>
>> If you would have to write a proper definition to add on the spec, would
>> "NetworkRoutes show us the "output" of the routing protocol generated by
>> the data inside the NetworkGraph" or would you improve it somehow?
>>
>>>>> On 05/28/2015 11:19 AM, Henning Rogge wrote:
>>>>> olsrd2 is "multitopology capable", which means it can run multiple
>>>>> topologies at the same time.
>>>>>
>>>>> How do we resolve this for the "routes" and "graph" output? Do we need
>>>>> to add another layer which allows multiple metrics, each with
>>>>> graph/routes?
>>>> I remember you mentioned this issue a while ago, now is the right time to
>>>> think about it and try to get it right.
>>>>
>>>> One of the existing initiatives that I've found solves this in a similar way
>>>> to what you proposed (in the later email), that is a list of graphs:
>>>> https://github.com/jsongraph/json-graph-specification/blob/master/examples/car_graphs.json
>>> This looks similar to my approach...
>>>
>>>> What I like most though, is the way GeoJSON solved this. In GeoJSON you can
>>>> have a Feature (geographic object) or a FeatureCollection (a collection of
>>>> geographic objects).
>>>>
>>>> I would prefer a similar approach because that would allow us to have a
>>>> collection of topologies of different routing protocols in a single JSON
>>>> document.
>>> Instead of going into the "level of indirection", maybe it is really
>>> easier to create multiple "network-graph" and "network-routes" JSON
>>> objects.
>>>
>>> Just repeat the header with the routing protocol name/version and add
>>> a different metric name (and maybe originator).
>>>
>>>> I will explain this further with a pull request on the github repository.
>>> Okay.
>> Here: https://github.com/interop-dev/netjson/pull/12/files
>>
>> In that example we have a router running both olsr1 and olsr2
>> simultaneously. Does it make sense?
>>
>>>>> On 05/28/2015 03:08 PM, Henning Rogge wrote:
>>>>> first implementation is finished (is available as the
>>>>> "json_for_networks" plugin in the OONF repository), but I think we
>>>>> need a few changes in our JSON definition.
>>>> First of all, thank you very much for doing this! This will surely speed up
>>>> adoption once we are ready.
>>>> The problem is exactly that we aren't ready yet, so.. I hope everyone of the
>>>> developers following this discussion is aware that as we improve the spec
>>>> and we try to do better we'll have to change the existing implementations,
>>>> like this one henning just wrote or the netdiff library.
>>>> Everyone ok with this?
>>> I hope so...
>>>
>>>> Second, "json for networks" doesn't sound so good for a proposed standard so
>>>> while I was in Madrid and Berlin I talked about netjson (which also has a
>>>> registered domain netjson.org), so now I just renamed the github repository
>>>> for consistency: https://github.com/interop-dev/netjson
>>>> I think netjson is an acceptable name, as geojson is an acceptable name to
>>>> describe simple geographic data.
>>>>
>>>> What do you think?
>>> netjson is fine.
>>>
>>>>> I have attached both "routes" and "graph" output of my plugin, that
>>>>> should make it easier to explain the changes.
>>>> Just checked them out.
>>>>
>>>> Curiosity: is ff_dat_metric an olsr2 metric?
>>> Yes... "freifunk/funkfeuer directional airtime metric". It will become
>>> an experimental IETF RFC soon!
>> Great news!
>>
>>>> Pay attention because in the graph.json file there is  "type":
>>>> "NetworkRoutes" but according to the spec it should be:
>>>>  "type": "NetworkGraph"
>>> Yes, thats a copy/paste error. ^^
>> I noticed many (required) commas are missing, resulting in an invalid
>> JSON just wanted to let you now.
>>
>>>> Although we should resolve this ambiguity (having two similar objects).
>>>>
>>>>> The major change is a new level of indirection, both for routes and
>>>>> graph objects to allow multiple topologies for both of them. This is
>>>>> relevant for dualstack support, because olsrv2 has a different
>>>>> "originator ID" for IPv4 and IPv6. It would also become very useful
>>>>> for running multiple routing metrics at the same time.
>>>>>
>>>>> The second major change is an array of "endpoints" in the graph object.
>>>>>
>>>>> Links are connections between Nodes, but Endpoints are connections of
>>>>> a (Router) Node with an external prefix.
>>>> Let's see if I get it:
>>>>
>>>> for a node with an external prefix we mean a connection with another
>>>> network? Any real world scenario example to help me understand?
>>> The most common external prefix is 0.0.0.0/0... a gateway to the internet.
>>>
>>> It is what HNAs did for OLSRv1.
>>>
>>> A connection of a "mesh router" to a different device that does NOT
>>> run the mesh routing protocol. The router announces "just send traffic
>>> for this to me!"
>> Got it. Thank you very much.
>>
>>>>> Similar data than Links, I
>>>>> just renamed "target" into "prefix" to make sure you don't mix up
>>>>> Links and Endpoints. Maybe they should have the same key.
>>>> If these are still links perhaps it would be better to keep consistency in
>>>> order to avoid complicating the implementations.
>>> No problem with this.
>>>
>>>> I suppose the list of endpoints could be optional?
>>> Sure, not all nodes will have endpoints.
>> Issue: https://github.com/interop-dev/netjson/issues/10
>>
>>>>> The last change was in the Routes object... I have added a "next-id"
>>>>> field, which will contain the router_id of the next hop. This is VERY
>>>>> useful when you work with linklocal IPs, because you cannot recognize
>>>>> the next-hop router by the IP!
>>>> You suggest this to be as standard attribute?
>>> Yes.
>>>
>>>> If yes, you suggest it to be mandatory or optional?
>>> Mandatory... every routing protocol should know it anyways... its just
>>> something that gets lost if you use linklocal IPs.
>>>
>>>> For consistency it should be "next_id".
>>> next_id would be fine.
>> Would you prefer us to add it on the spec and then ask for feedback or
>> rather the other way around (asking for feeback before adding it)?
>>
>> Issue: https://github.com/interop-dev/netjson/issues/9
>>
>>>>> On 05/28/2015 04:23 PM, Henning Rogge wrote:
>>>>> two more questions... one for "routes" and one for "graph".
>>>>>
>>>>> a) do we maybe want to add the fixed field "hops" to the routes? The
>>>>> number of intermediate stations might be interesting to judge a route
>>>>> in addition to the total cost.
>>>> Going back to the motivations and goals, if this is helpful and we need it
>>>> to develop something in the near future, I would add it, otherwise we can
>>>> leave this to be discussed a later phase, in which I hope we'll meet again
>>>> to discuss points like this.
>>>>
>>>> We should keep track of this somehow, I'll be opening some issues on github.
>>> The "hops" field would give you an additional way to judge what kind
>>> of route this is. But I have to admit I don't know if Batman is
>>> counting the hops towards the target.
>> Issue: https://github.com/interop-dev/netjson/issues/11
>>
>>>>> b) what do we do about "links" that are not used for routing? Maybe
>>>>> because they are still asymmetric or they are marked as "lost"... or
>>>>> they are just too bad?
>>>> According to the goal in the premise of this mail (assuming you agree with
>>>> that premise btw) it seems clear to me that we still want the bad links to
>>>> show up in the NetworkGraph object in order to visualize them or monitor
>>>> their health, we just have to be able to understand that they're in very bad
>>>> health conditions.
>>> Maybe something like a "state" field?
>>>
>>> "active" for links that are used for routing.
>>> "transitional" for links that are known to the routing protocol but
>>> are not used (for one reason or the other) for routing.
>>> "lost" for links that are not used for routing anymore, but are still
>>> in a database of the routing protocol.
>> Regarding "lost": currently I beleieve that node-databases /
>> visualization tools rely on the fact that links disappear from the
>> topology when nodes go down, so they update their database (or
>> visualization) accordingly. Showing lost links would probably change how
>> we do things so perhaps we should carefully think about it before
>> proceeding.
>> Also, are we sure other routing protocols keep a database of lost links?
>>
>> Regarding "transitional", this looks interesting, if the 2 linked
>> devices are still able to communicate, it's nice to know that the link
>> is there, but the protocol has chosen to not use it.
>> I think we could find a better word than "transitional" though.
>> One question, wouldn't be the weight parameter be enough to know this?
>> Wouldn't the weight be infinite or something like that?
>>
>> Federico
>>