Join efforts with Gluon

[CodeFetch]

Why doesn't Freifunk Berlin try to incorporate their packages into Gluon?
Having separate projects is much double-work. By joining efforts we could build better firmware.

[SvenRoederer]

I've only basic knowledge of Gluon, but the main problem I see is the different network-design.
Gluon is based on L2-switching (B.A.T.M.A.N.), Berlin is based in L3-Routing (sadly still Olsr v1).
But in the recent development-code we added the gluon-packages feed already.

[CodeFetch]

I've seen that you think about switching to another Layer 3 protocol. Gluon supports Babel which can be seen as a successor of OLSR. There is only missing a kernel module I'm developing for IPv4 support. At the moment Gluon's Babel implementation only supports IPv6. I don't know whether you can live with that restriction, but sooner or later it will be a full replacement of OLSR.

[pmelange]

I don't think that it makes a lot of sense to put the freifunk-berlin packages into gluon, but collaboration is still possible in the sense that we could work together to support new hardware and possibly also share patches to OpenWrt.

With regard to new hardware, one of the issues which freifunk-berlin is facing right now is driver support for the ath10k devices. We are migrating to 802.11s and the -ct drivers do not support 802.11s in wave1 devices.

[CodeFetch]

@pmelange Why do you think it does not make sense?

[pmelange]

Perhaps @booo @db4rne @sarumpaet can also give their impressions, but from what has been discussed in meetings and on chat the following points are important.

Some of these points may be incorrect, i'm just copying and pasting what I can quickly find. This includes feedback from other communities outside of Berlin

• gluon is like a "all-in-one" package with lot's of opaque c-stuff
• gluon is pretty centralised
• gluon has a lot of services that depend on a layer2 mesh and most of them are written in c = pain in the ass to debug on a router
• freifunk-berlin's wizard is quite hackable as it's posix sh and lua
• freifunk-berlin's openwifimap is also just a lua script
• freifunk-belrin is excellent for hacking/experiments/modifiying stuff
• the simplicity of the berlin firmware ffwizard.d directory, where you put a sh file and run some uci callback functions on config file, is pretty cool and super flexible

I'm sure I could dig around a bit more.

[SvenRoederer]

I remember a short moment in time, where we used you tunneldigger-package. But for some reason we started an own fork soon. @pmelange might remember the details.

[akira25]

@CodeFetch I don't understand your question completely. Do you want freifunk-berlin to port its original packages to gluon and to use a common package-base? Do you want freifunk-berlin to switch to gluon? Maybe these two points depend on each other...

For the first point: It sounds like a nice idea. I would agree on that for efficency reasons in developement. But as far as I understood pmlange right, gloun seems not flexible enough to grant berlin-specific adjustments.

For the second point: No.

I do not like gluon for some reasons:

• It is extremly centralised: one server doing wrong can set the whole network out of service (for example a dying DHCP-Server)
• for it is centralised, it needs an admin team. I heard several opinions on that and think, that everything gloun gives the users for carefreeness gets more workload for the admin team. I think most freifunkas in berlin would spend their free time more hapily with other funny things, than carrying that burden to administrate the whole network. Lets mind the concept of single point of failure. It might even rely on humans... ;-)
• gluon can only be configured when triggered by a hardware-botton. Please correct me, if i got it wrong. But that is an unsuitable requirement for the freifunk-berlin net. Core-routers must be configureable all time. Even remote.
• freifunk-berlin would loose (some) of its independency. I suppose there were good reasons to start to devolp an own firmware. Maybe for net-architectural resons, maybe for many others too. If freifunk berlin would switch completely to gluon (wich would be, by the way, a horrible pile of several months work), those might not be met anymore. What is fine in many other cities might be inapplicable in Berlin... In my opinion, an own firmware fulfill the own requirements best..

[SvenRoederer]

regarding a common packages-repo: there is https://github.com/freifunk/openwrt-packages. It's hosted under the flag of the global Freifunk-team.

[CodeFetch]

I see your points. You expect the development effort to increase, but I think in the long-run you wouldn't have any great effort if you don't want to participate in development anymore.
Actually I think if you'd use Gluon as your base and add the functionalities/packages you need the effort would decrease most as there is device support to handle etc.

• gluon is like a "all-in-one" package with lot's of opaque c-stuff

It is a framework and aims to be very customizable. You can build it with basic functionality only. The C-stuff is either something that just did not exist before or for a critical function that would take to much resources if it would have been written in an interpreter language.

• gluon is pretty centralised

We are working on changing that. With Babel everything will be completely decentral and more autonomous. We are also working on a similar solution with Batman, but as you said, you're not interested in layer 2.

• gluon has a lot of services that depend on a layer2 mesh and most of them are written in c = pain in the ass to debug on a router

We are changing these packages to comply with layer 3 mesh protocols atm.

• freifunk-berlin's wizard is quite hackable as it's posix sh and lua

Most of Gluon's scripts that are likely to be altered by beginners are written in Lua, too.
We consider ash messy and thus prefer Lua.

• freifunk-belrin is excellent for hacking/experiments/modifiying stuff

Gluon does not restrict you in any way. Packages that should be merged just need a relatively high coding quality.

• the simplicity of the berlin firmware ffwizard.d directory, where you put a sh file and run some uci callback functions on config file, is pretty cool and super flexible

We consider these things a hack as when you have a lot of those scripts, it is getting a bit messy, but we are working on providing a similarly modular functionality that is likewise easy to use.

• It is extremly centralised: one server doing wrong can set the whole network out of service (for example a dying DHCP-Server)

We are working on changing that. Every node will be able to provide Uplink/VPN or whatever with the upcoming Babel concept. We even have a decentralized DHCP daemon. (DDHCP) for that.

• for it is centralised, it needs an admin team. I heard several opinions on that and think, that everything gloun gives the users for carefreeness gets more workload for the admin team. I think most freifunkas in berlin would spend their free time more hapily with other funny things, than carrying that burden to administrate the whole network. Lets mind the concept of single point of failure. It might even rely on humans... ;-)

That applies to us, too. That's why we are working on the decentralization concept. A supernode should be nothing than a KVM running a Gluon image for supernodes in the end. Little knowledge needed for running the infrastructure then.

• gluon can only be configured when triggered by a hardware-botton. Please correct me, if i got it wrong. But that is an unsuitable requirement for the freifunk-berlin net. Core-routers must be configureable all time. Even remote.

You can configure it remotely using SSH. Do you have a web interface for that? We decided against this approach due to missing proper certificates/TLS. How did you solve the problem?

• freifunk-berlin would loose (some) of its independency. I suppose there were good reasons to start to devolp an own firmware. Maybe for net-architectural resons, maybe for many others too. If freifunk berlin would switch completely to gluon (wich would be, by the way, a horrible pile of several months work), those might not be met anymore. What is fine in many other cities might be inapplicable in Berlin... In my opinion, an own firmware fulfill the own requirements best..

The last thing Gluon wants is to become "the Freifunk firmware". What we want is to collaboratively develop a very customizable framework to make it easier to develop one's own firmware. As this is a lot of work it makes sense to me, to join efforts. If there wouldn't be a hundred linux distributions in the wild with different benefits, but people would work on a single distribution that tries to unify those benefits, Microsoft and Apple would be dead by the time. The best example for this is the linux kernel itself. It is a masterpiece of software and if we could craft a masterpiece of a mesh framework, this would be a great thing.

[booo]

@CodeFetch Could you provide a link to the babel stuff you mentioned?

[db4rne]

@CodeFetch let me ask why you don't join LibreMesh then? I Would also like to know if you tested babel in a big mesh already and how it performed? As far as I understand it, babel is proactive (triggers an update when a route changes), and that could bring problems in big mesh networks? So, if you already tested it I would be very interested in your results.

[CodeFetch]

Unfortunately most of the links (including the bigger test network's map) are offline:
https://wiki.ffm.freifunk.net/firmware:babel

On the Battlemesh we gave presentations about the plan, but unfortunately the video recordings were never published.

Actually I'd like Gluon to become the base of LibreMesh when we've done the needed changes.

Regarding the proactive triggers that lead to route changes: These are only being distributed to the nodes to which they matter. The scaling issues are less problematic compared to Batman. To be fair Batman V has improved much and with enough RAM and bandwidth even Batman V networks can scale quite good.
With the introduction of "domains" Gluon does not really have any scaling issues anymore. The new problem is that border routers (routers on the border of different domains) still need to learn to route traffic between the domains/act as a gateway between them. I hope that this will come after the merge of the hoodselector. At the moment domains mean a hard cut between networks and traffic between domains is being routed on layer 3 which means roaming might break and the devices depend on supernodes for client-to-client communication.

I'll try to briefly explain the Babel L3 concept:
l3roamd: Is a software that manages Babel's route distribution of clients in Babel networks. A special IPv6 management address is being calculated from the client's MAC address and routes to the node that serves the client. When a client roams to another node the new node sends a request to this special address and i.a. gets a response with the IP addresses the client uses. The old node releases the route of the special IP address while the new node sets it together with the client's IP addresses to point to itself. This is also a good example for why Babel does not need to distribute route changes through the whole network. Only the neighboring nodes of the nodes involved in the roaming process need to know these updates.
DDHCP: is a distributed DHCP daemon. Nodes acquire a range of leases and offer those to their clients. Before they run out of leases, they acquire a new range. Thus there does not need to be a central DHCP server.
mmfd: is a simple replacement for broadcast traffic which sends traffic to the other nodes as unicast. It is being used for important management traffic only in L3 networks like statistics or public S2N offerings.
S2N-VPN: Is a concept to let nodes serve as public supernodes by offering ports for VPN connections that can be established over the internet.
N2N-VPN: Is a concept to let nodes establish connections with each other if the user wishes to do so using manually defined "trusted" nodes.
PLAT: Is a NAT64 server which offers IPv6->IPv4 translation (acts as a gateway for IPv4 traffic in IPv6-only networks). You can e.g. use this to share your internet connection directly in the network (e.g. if the node is running in a cafe or if it is a KVM in a datacenter) or to offer exit of IPv4 traffic to a VPN provider.
NAT426: As our Babel networks are IPv6 only this takes care of "faking" an IPv4 gateway for the clients. It is a conceptual kernel module (in an experimental state due to my lack of time) that does NAT46 address translation so that multiple PLATs can be used transparently. It does connection tracking which allows connections that don't break when a node's route's metric to a specific PLAT becomes less attractive. New connections to different IPs use the better PLAT then while existing connections stay using their individual PLAT. This also allows seamless IPv4 roaming as l3roamd takes care of distributing the connection tracking information.
hoodselector/domains: Is a concept that in the end allows automatic clustering of mesh clouds which share a common IPv6 address space in the Layer3 case. There is still a unified concept missing for routing traffic between the domains, but it is only a matter of time and manpower.

As you can see the Babel concept is designed to allow a very high degree of decentralization. In the end the only thing that people need to achieve a consensus on is the software they use and some central servers or a public register to allow to enter the network. Having entered the network once, it is only a question of people willing to provide public peerings (s2n). But those are the same things we see in e.g. onion routing networks or DHT networks. At some point you just need an ability to easily enter the network. With the new concept it is not a single point of failure anymore as the network still runs even if the public entrance servers go offline. Of course people need to be encouraged to provide many S2Ns and do many N2N peerings, but those who really want to use Freifunk like its initial purpose suggests will do and those who don't won't...

You can't force people to build a zombie apocalypse network, but you can give them the tools to.

[SvenRoederer]

Thanks for the extensive overview. Some of the concepts you outline seem not the far from what we have, e.g. DDHCP (we have a config.berlin.freifunk.net a service where a node operator can request an IP-range that will be provided to the clients).
Others might be a good chance / idea to improve our net. Currently we are stuck in a nearly IPv4 only net and also looking for some replacement for OLSR. As every node can potentially share it's internet-connection, we have some ideas how to handle the multpile-IPv6-prefixes problem. Probably the PLAT and NAT426 are some options to discuss.
For roaming clients between some L3 neighbors we currently use BATMAN to form some clouds, but it's not our intention to provide seamless roaming over the whole city of Berlin.

P.S. I recent hacked a change that enables us to share some patches more painless (#714).

[SvenRoederer]

I'm looking forward to reuse the gluon-way of patching openwrt and feeds. For this I'm currently working on branch "https://github.com/freifunk-berlin/firmware/tree/master_gluon-patch"