From (spam-protected) Sat Jun 11 10:31:54 2011 From: (spam-protected) (Clemens John) Date: Sat, 11 Jun 2011 10:31:54 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher Message-ID: <201106111031.58483.clemens-john@gmx.de> Hi, where can I find the database shema of Nodewatcher (WLAN-SI)? I know ofcourse that the shema is explained in the interop wiki but I can´t find the pure code. Any hint? Regards Clemens -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From (spam-protected) Sat Jun 11 11:29:12 2011 From: (spam-protected) (Mitar) Date: Sat, 11 Jun 2011 11:29:12 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <201106111031.58483.clemens-john@gmx.de> References: <201106111031.58483.clemens-john@gmx.de> Message-ID: <4DF33568.7070503@tnode.com> Hi! > where can I find the database shema of Nodewatcher (WLAN-SI)? > I know ofcourse that the shema is explained in the interop wiki but I can´t > find the pure code. Any hint? http://dev.wlan-si.net/browser/nodewatcher/nodewatcher/web/nodes/models.py But there is work on progress on completely changing this into an pluggable and extensible schema. A work in progress core part of this schema can be found here: http://dev.wlan-si.net/wiki/Nodewatcher/Registry/NodeConfiguration The main difference is that schema becomes a tree structure, where plugins can attach new field anywhere in the tree structure. Then other plugins can use those new fields or stick to the core ones. I have no idea how you could implement something like this in anything else than Django (or some other system with good database abstraction) as we are heavily relying on its abstraction and thus automation. Because writing such SQL queries by hand is probably impossible. The main motivation is that we have decided that a better approach than one-suits-all is to make a system, which can be heavily customized by everybody. What would probably useful is to to name our fields and define the content the same as you would have in your common schema definition. And maybe define some automatic mapping between our tree-structure path and (probably?) your flat schema table names. Mitar From (spam-protected) Sat Jun 11 12:38:34 2011 From: (spam-protected) (Jernej Kos) Date: Sat, 11 Jun 2011 12:38:34 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <201106111031.58483.clemens-john@gmx.de> References: <201106111031.58483.clemens-john@gmx.de> Message-ID: <4DF345AA.4080900@unimatrix-one.org> Hello! On 06/11/2011 10:31 AM, Clemens John wrote: > where can I find the database shema of Nodewatcher (WLAN-SI)? > I know ofcourse that the shema is explained in the interop wiki but I can´t > find the pure code. Any hint? That schema (on the interop site) is considered obsolete as we are currently transitioning to a new, more modular schema (as Mitar already explained). Since its implementation is modular it is spread among many modules: * Registry framework that enables construction of such a schema and can also automatically generate forms for configuration of this via the frontend. Docs: http://dev.wlan-si.net/wiki/Nodewatcher/Registry Source: http://dev.wlan-si.net/browser/nodewatcher/nodewatcher/web/registry * Core & CGM schema Docs: http://dev.wlan-si.net/wiki/Nodewatcher/Registry/NodeConfiguration Source: http://dev.wlan-si.net/browser/nodewatcher/nodewatcher/web/core http://dev.wlan-si.net/browser/nodewatcher/nodewatcher/web/core/cgm Many of the old Django models are going to be replaced as they are being ported to the new registry framework. Also all monitoring-related things will be ported to the registry framework as well (schema is not yet defined). If you have any further questions, feel free to ask. Jernej -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 294 bytes Desc: OpenPGP digital signature URL: From (spam-protected) Mon Jun 13 10:18:10 2011 From: (spam-protected) (L. Aaron Kaplan) Date: Mon, 13 Jun 2011 10:18:10 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <4DF33568.7070503@tnode.com> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> Message-ID: On Jun 11, 2011, at 11:29 AM, Mitar wrote: > Hi! > >> where can I find the database shema of Nodewatcher (WLAN-SI)? >> I know ofcourse that the shema is explained in the interop wiki but I can´t >> find the pure code. Any hint? > > http://dev.wlan-si.net/browser/nodewatcher/nodewatcher/web/nodes/models.py > > But there is work on progress on completely changing this into an > pluggable and extensible schema. A work in progress core part of this > schema can be found here: > > http://dev.wlan-si.net/wiki/Nodewatcher/Registry/NodeConfiguration > > The main difference is that schema becomes a tree structure, where > plugins can attach new field anywhere in the tree structure. Then other > plugins can use those new fields or stick to the core ones. Can you explain "tree structure" a bit please? I know tree based / hierarchical DBs. But I am not sure if you mean the same thing. > Thanks, a. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 243 bytes Desc: This is a digitally signed message part URL: From (spam-protected) Mon Jun 13 10:20:38 2011 From: (spam-protected) (Mitar) Date: Mon, 13 Jun 2011 10:20:38 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> Message-ID: <4DF5C856.4000501@tnode.com> Hi! > Can you explain "tree structure" a bit please? Sure. There are in fact two trees. One is a tree of structures of modules/addons/plugins announcing/adding/registering some (sub)schema in the tree. For example, you have a router, which has one or more interfaces, some of them are WiFi interfaces, which can have one or more network configurations (for each SSID or something). So you have a tree of features of this node. This could be seen simply as a node table, having multiple other tables (like basic features: interfaces and so on) using it as a foreign key, then another tables (like subfeatures) using them and so on. But it is more naturally explained with a tree. Then there is a tree of inheritance. The idea is, that you have a basic schema (for each part of the above mentioned tree) defined, which can be inherited, new fields added, some hidden (and automatically filled in) and so on. You should see this as an object-oriented classes and inheritance. So maybe forgot about databases, but see this as a tree of classes, where classes could have their own inheritance, so you can replace one class in a tree with one inheriting from it. For such tree to be useful, you have to have some predefined tree structure, so that you know where you can find classes you want, and also to have those basic classes predefined, so that you can extend them with new features. So there are two dimensions in which you can extend the schema by plugins/addons. Some documentation on this tree representation of ours can be seen here: http://dev.wlan-si.net/wiki/Nodewatcher/Registry http://dev.wlan-si.net/wiki/Nodewatcher/Registry/NodeConfiguration Mitar From (spam-protected) Mon Jun 13 13:30:20 2011 From: (spam-protected) (L. Aaron Kaplan) Date: Mon, 13 Jun 2011 13:30:20 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <4DF5C856.4000501@tnode.com> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> Message-ID: Thanks for the explanation. I understand it a little bit now. But I am not sure if I got it 100%. This all sounds very very much like LDAP to me now. What is the difference to LDAP? Would it be possible to define all of this what you want in LDAP? a. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 243 bytes Desc: This is a digitally signed message part URL: From (spam-protected) Mon Jun 13 22:11:47 2011 From: (spam-protected) (Jernej Kos) Date: Mon, 13 Jun 2011 22:11:47 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> Message-ID: <4DF66F03.2030909@unimatrix-one.org> Hello! On 06/13/2011 01:30 PM, L. Aaron Kaplan wrote: > I understand it a little bit now. But I am not sure if I got it 100%. > This all sounds very very much like LDAP to me now. > Would it be possible to define all of this what you want in LDAP? Yes it is actually somewhat similar to LDAP because of its tree structure and extensible entry classes, but the current implementation is based on Django's ORM. So it would definitely be possible to define this in LDAP if one would want to do it that way. The major point (and the reason we need this) is that nodewatcher v3 schema needs to be easily extensible by modules. It should also enable automatic form generation and validation for this schema. Jernej -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 294 bytes Desc: OpenPGP digital signature URL: From (spam-protected) Mon Jun 13 22:17:19 2011 From: (spam-protected) (Mitar) Date: Mon, 13 Jun 2011 22:17:19 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <4DF66F03.2030909@unimatrix-one.org> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> <4DF66F03.2030909@unimatrix-one.org> Message-ID: <4DF6704F.4090104@tnode.com> Hi! > The major point (and the reason we need this) is that nodewatcher v3 > schema needs to be easily extensible by modules. It should also enable > automatic form generation and validation for this schema. Yes, the main point is to use Django's automatic mechanisms (model-view-controller) which can do many things by themselves. By that it will be very easy to write new/additinal/slightly changed features, while retaining the whole codebase. So it will not be necessary to write everything from scratch just to change one small part of the system, but you will just replace that part by something different or even just with a class extending the original code class but with slightly different functionality. So we are using object-oriented programing both in code and schema. Mitar From (spam-protected) Mon Jun 13 23:51:33 2011 From: (spam-protected) (L. Aaron Kaplan) Date: Mon, 13 Jun 2011 23:51:33 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <4DF66F03.2030909@unimatrix-one.org> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> <4DF66F03.2030909@unimatrix-one.org> Message-ID: <268E99E9-167D-4409-927B-61AD0784F607@lo-res.org> On Jun 13, 2011, at 10:11 PM, Jernej Kos wrote: > Hello! > > On 06/13/2011 01:30 PM, L. Aaron Kaplan wrote: >> I understand it a little bit now. But I am not sure if I got it 100%. >> This all sounds very very much like LDAP to me now. >> Would it be possible to define all of this what you want in LDAP? > > Yes it is actually somewhat similar to LDAP because of its tree > structure and extensible entry classes, but the current implementation > is based on Django's ORM. So it would definitely be possible to define > this in LDAP if one would want to do it that way. > > The major point (and the reason we need this) is that nodewatcher v3 > schema needs to be easily extensible by modules. It should also enable > automatic form generation and validation for this schema. > I see... hmm... interesting concept. Did you plan some versioning concepts for this? -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 243 bytes Desc: This is a digitally signed message part URL: From (spam-protected) Mon Jun 13 23:56:11 2011 From: (spam-protected) (Mitar) Date: Mon, 13 Jun 2011 23:56:11 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <268E99E9-167D-4409-927B-61AD0784F607@lo-res.org> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> <4DF66F03.2030909@unimatrix-one.org> <268E99E9-167D-4409-927B-61AD0784F607@lo-res.org> Message-ID: <4DF6877B.1070400@tnode.com> Hi! > Did you plan some versioning concepts for this? Not yet, but it is in a middle term goals. To enable reverting to old configuration. So that we can have a wiki-like open configuration or something. For now we will first make permission system extensible, so that you will be able to have different users having different permissions over nodes. Then later on we will play with both logging and versioning. It is a big enough rewrite already. And our codebase is now completely in flux. And we have to get to some working version soon. ;-) BTW, this will also support different tool-chains. This was also one of design requirements. So that once you configure the node it is configured in the device/platform-independent way, so that you can then generate configuration/firmware using OpenWrt, Ubiquiti SDK and others (for now we will support those two). Mitar From (spam-protected) Tue Jun 14 00:04:44 2011 From: (spam-protected) (L. Aaron Kaplan) Date: Tue, 14 Jun 2011 00:04:44 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <4DF6877B.1070400@tnode.com> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> <4DF66F03.2030909@unimatrix-one.org> <268E99E9-167D-4409-927B-61AD0784F607@lo-res.org> <4DF6877B.1070400@tnode.com> Message-ID: <95E4255B-B3E4-45F6-BCB3-B6944081998C@lo-res.org> Very nice. Ok, so the commonNodeDB concept still fits perfectly with your new structure since it is very flexible, if I understood you correctly. --- Mobil Am 13.06.2011 um 23:56 schrieb Mitar : > Hi! > >> Did you plan some versioning concepts for this? > > Not yet, but it is in a middle term goals. To enable reverting to old > configuration. So that we can have a wiki-like open configuration or > something. > > For now we will first make permission system extensible, so that you > will be able to have different users having different permissions over > nodes. > > Then later on we will play with both logging and versioning. It is a big > enough rewrite already. And our codebase is now completely in flux. And > we have to get to some working version soon. ;-) > > BTW, this will also support different tool-chains. This was also one of > design requirements. So that once you configure the node it is > configured in the device/platform-independent way, so that you can then > generate configuration/firmware using OpenWrt, Ubiquiti SDK and others > (for now we will support those two). > > > Mitar > From (spam-protected) Tue Jun 14 00:09:02 2011 From: (spam-protected) (Mitar) Date: Tue, 14 Jun 2011 00:09:02 +0200 Subject: [Nodedb-interop] Database shema of Nodewatcher In-Reply-To: <95E4255B-B3E4-45F6-BCB3-B6944081998C@lo-res.org> References: <201106111031.58483.clemens-john@gmx.de> <4DF33568.7070503@tnode.com> <4DF5C856.4000501@tnode.com> <4DF66F03.2030909@unimatrix-one.org> <268E99E9-167D-4409-927B-61AD0784F607@lo-res.org> <4DF6877B.1070400@tnode.com> <95E4255B-B3E4-45F6-BCB3-B6944081998C@lo-res.org> Message-ID: <4DF68A7E.3030407@tnode.com> Hi! > Ok, so the commonNodeDB concept still fits perfectly with your new > structure since it is very flexible, if I understood you correctly. So we could use common field names (and meaning, content, requirements ...). Only how/where those fields are is somewhat more complex. For tables we could define some mapping between tree structure and commonNodeDB tables. Like: core.interfaces.WifiInterfaceConfig with fields: wifi_radio, protocol, channel, bitrate, antenna Could map to some table in commonNodeDB. But it will be easier to talk about more concrete things. Mitar