[Interop-dev] Improving NetworkGraph and NetworkRoutes [was: hello new subscribers]

[Henning Rogge]

Hi,

I updated the json_for_networks plugin for olsrd2 according to this discussion.

I have attached the output for a fully-meshed three node (VM) mesh
with node 1 being a IPv4 gateway to the internet.

Henning

On Sun, May 31, 2015 at 5:48 PM, Nemesis <(spam-protected)> wrote:
> On 05/31/2015 05:07 PM, Henning Rogge wrote:
>> On Sun, May 31, 2015 at 4:39 PM, Nemesis <(spam-protected)> wrote:
>>> The goal in my opinions are:
>>>
>>> visualization: develop awesome visualization tools that work with any
>>> routing protocol.
>>> monitor changes in topology: to develop (hopefully) one (or two in case some
>>> people are not happy with the first attempt) reusable parser implementation
>>> to update the links in the node-databases
>> Visualization, monitoring and debugging (what the hell happens with
>> our traffic).
> +1
>>> NetworkRoutes
>>> For NetworkRoutes, what kind of tools can we achieve with it? What is
>>> pushing us to define it?
>>> I would say that having a standard structure is a good idea, we can also
>>> reuse this structure in DeviceConfiguration to describe static routes (as we
>>> are already doing).
>> NetworkRoutes show us the "output" of the routing protocol generated
>> by the data inside the NetworkGraph.
>>
>> Without the routes it is hard to see what is really going on.
>
> Excellent.
>
> If you would have to write a proper definition to add on the spec, would
> "NetworkRoutes show us the "output" of the routing protocol generated by
> the data inside the NetworkGraph" or would you improve it somehow?
>
>>>> On 05/28/2015 11:19 AM, Henning Rogge wrote:
>>>> olsrd2 is "multitopology capable", which means it can run multiple
>>>> topologies at the same time.
>>>>
>>>> How do we resolve this for the "routes" and "graph" output? Do we need
>>>> to add another layer which allows multiple metrics, each with
>>>> graph/routes?
>>> I remember you mentioned this issue a while ago, now is the right time to
>>> think about it and try to get it right.
>>>
>>> One of the existing initiatives that I've found solves this in a similar way
>>> to what you proposed (in the later email), that is a list of graphs:
>>> https://github.com/jsongraph/json-graph-specification/blob/master/examples/car_graphs.json
>> This looks similar to my approach...
>>
>>> What I like most though, is the way GeoJSON solved this. In GeoJSON you can
>>> have a Feature (geographic object) or a FeatureCollection (a collection of
>>> geographic objects).
>>>
>>> I would prefer a similar approach because that would allow us to have a
>>> collection of topologies of different routing protocols in a single JSON
>>> document.
>> Instead of going into the "level of indirection", maybe it is really
>> easier to create multiple "network-graph" and "network-routes" JSON
>> objects.
>>
>> Just repeat the header with the routing protocol name/version and add
>> a different metric name (and maybe originator).
>>
>>> I will explain this further with a pull request on the github repository.
>> Okay.
>
> Here: https://github.com/interop-dev/netjson/pull/12/files
>
> In that example we have a router running both olsr1 and olsr2
> simultaneously. Does it make sense?
>
>>
>>>> On 05/28/2015 03:08 PM, Henning Rogge wrote:
>>>> first implementation is finished (is available as the
>>>> "json_for_networks" plugin in the OONF repository), but I think we
>>>> need a few changes in our JSON definition.
>>> First of all, thank you very much for doing this! This will surely speed up
>>> adoption once we are ready.
>>> The problem is exactly that we aren't ready yet, so.. I hope everyone of the
>>> developers following this discussion is aware that as we improve the spec
>>> and we try to do better we'll have to change the existing implementations,
>>> like this one henning just wrote or the netdiff library.
>>> Everyone ok with this?
>> I hope so...
>>
>>> Second, "json for networks" doesn't sound so good for a proposed standard so
>>> while I was in Madrid and Berlin I talked about netjson (which also has a
>>> registered domain netjson.org), so now I just renamed the github repository
>>> for consistency: https://github.com/interop-dev/netjson
>>> I think netjson is an acceptable name, as geojson is an acceptable name to
>>> describe simple geographic data.
>>>
>>> What do you think?
>> netjson is fine.
>>
>>>> I have attached both "routes" and "graph" output of my plugin, that
>>>> should make it easier to explain the changes.
>>> Just checked them out.
>>>
>>> Curiosity: is ff_dat_metric an olsr2 metric?
>> Yes... "freifunk/funkfeuer directional airtime metric". It will become
>> an experimental IETF RFC soon!
>
> Great news!
>
>>> Pay attention because in the graph.json file there is  "type":
>>> "NetworkRoutes" but according to the spec it should be:
>>>  "type": "NetworkGraph"
>> Yes, thats a copy/paste error. ^^
>
> I noticed many (required) commas are missing, resulting in an invalid
> JSON just wanted to let you now.
>
>>> Although we should resolve this ambiguity (having two similar objects).
>>>
>>>> The major change is a new level of indirection, both for routes and
>>>> graph objects to allow multiple topologies for both of them. This is
>>>> relevant for dualstack support, because olsrv2 has a different
>>>> "originator ID" for IPv4 and IPv6. It would also become very useful
>>>> for running multiple routing metrics at the same time.
>>>>
>>>> The second major change is an array of "endpoints" in the graph object.
>>>>
>>>> Links are connections between Nodes, but Endpoints are connections of
>>>> a (Router) Node with an external prefix.
>>> Let's see if I get it:
>>>
>>> for a node with an external prefix we mean a connection with another
>>> network? Any real world scenario example to help me understand?
>> The most common external prefix is 0.0.0.0/0... a gateway to the internet.
>>
>> It is what HNAs did for OLSRv1.
>>
>> A connection of a "mesh router" to a different device that does NOT
>> run the mesh routing protocol. The router announces "just send traffic
>> for this to me!"
>
> Got it. Thank you very much.
>
>>>> Similar data than Links, I
>>>> just renamed "target" into "prefix" to make sure you don't mix up
>>>> Links and Endpoints. Maybe they should have the same key.
>>> If these are still links perhaps it would be better to keep consistency in
>>> order to avoid complicating the implementations.
>> No problem with this.
>>
>>> I suppose the list of endpoints could be optional?
>> Sure, not all nodes will have endpoints.
>
> Issue: https://github.com/interop-dev/netjson/issues/10
>
>>
>>>> The last change was in the Routes object... I have added a "next-id"
>>>> field, which will contain the router_id of the next hop. This is VERY
>>>> useful when you work with linklocal IPs, because you cannot recognize
>>>> the next-hop router by the IP!
>>> You suggest this to be as standard attribute?
>> Yes.
>>
>>> If yes, you suggest it to be mandatory or optional?
>> Mandatory... every routing protocol should know it anyways... its just
>> something that gets lost if you use linklocal IPs.
>>
>>> For consistency it should be "next_id".
>> next_id would be fine.
>
> Would you prefer us to add it on the spec and then ask for feedback or
> rather the other way around (asking for feeback before adding it)?
>
> Issue: https://github.com/interop-dev/netjson/issues/9
>
>>>> On 05/28/2015 04:23 PM, Henning Rogge wrote:
>>>> two more questions... one for "routes" and one for "graph".
>>>>
>>>> a) do we maybe want to add the fixed field "hops" to the routes? The
>>>> number of intermediate stations might be interesting to judge a route
>>>> in addition to the total cost.
>>> Going back to the motivations and goals, if this is helpful and we need it
>>> to develop something in the near future, I would add it, otherwise we can
>>> leave this to be discussed a later phase, in which I hope we'll meet again
>>> to discuss points like this.
>>>
>>> We should keep track of this somehow, I'll be opening some issues on github.
>> The "hops" field would give you an additional way to judge what kind
>> of route this is. But I have to admit I don't know if Batman is
>> counting the hops towards the target.
>
> Issue: https://github.com/interop-dev/netjson/issues/11
>
>>>> b) what do we do about "links" that are not used for routing? Maybe
>>>> because they are still asymmetric or they are marked as "lost"... or
>>>> they are just too bad?
>>> According to the goal in the premise of this mail (assuming you agree with
>>> that premise btw) it seems clear to me that we still want the bad links to
>>> show up in the NetworkGraph object in order to visualize them or monitor
>>> their health, we just have to be able to understand that they're in very bad
>>> health conditions.
>> Maybe something like a "state" field?
>>
>> "active" for links that are used for routing.
>> "transitional" for links that are known to the routing protocol but
>> are not used (for one reason or the other) for routing.
>> "lost" for links that are not used for routing anymore, but are still
>> in a database of the routing protocol.
>
> Regarding "lost": currently I beleieve that node-databases /
> visualization tools rely on the fact that links disappear from the
> topology when nodes go down, so they update their database (or
> visualization) accordingly. Showing lost links would probably change how
> we do things so perhaps we should carefully think about it before
> proceeding.
> Also, are we sure other routing protocols keep a database of lost links?
>
> Regarding "transitional", this looks interesting, if the 2 linked
> devices are still able to communicate, it's nice to know that the link
> is there, but the protocol has chosen to not use it.
> I think we could find a better word than "transitional" though.
> One question, wouldn't be the weight parameter be enough to know this?
> Wouldn't the weight be infinite or something like that?
>
> Federico
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graph.json
Type: application/json
Size: 7003 bytes
Desc: not available
URL: <http://lists.funkfeuer.at/pipermail/interop-dev/attachments/20150531/055ea51c/attachment.json>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: routes.json
Type: application/json
Size: 3102 bytes
Desc: not available
URL: <http://lists.funkfeuer.at/pipermail/interop-dev/attachments/20150531/055ea51c/attachment-0001.json>