King Ludd #17: Git and our distributed future

Tweet

Linus Torvalds, the creator of the Linux kernel, can be a divisive character thanks to his occasional, though generally warranted, abrasiveness. From the land of controversial opinions about Linus comes my own, which has nothing to do with his personality and all to do with his achievements: the world wouldn’t be that different without Linux (there are not too distant alternatives, such as the BSDs), but without his creation Git the future would look very different indeed. What Git is for is not really that important, nor what it does; what counts is how it does it.

Git is a tool used by software developers for collaboration and keeping track of their work history, like a giant undo/redo. It is fundamentally a distributed system that has ended up replacing a lot of centralized ones, so basically everyone has their own copy of the undo/redo history instead of there being one definitive version stored centrally, and there are mechanisms for merging two histories to create a new one. In many ways it is less efficient than a centralized approach (not least all the extra space needed for everyone to store that data), but it has also been the driving force that convinced most developers a decentralized approach to development was even possible, let alone workable or preferable. This means that you can continue to work offline, or even have an entire team working together on a private network with no internet connection and it making no difference at all.

The systems Git displaced look a lot like the products the big cloud service companies are currently trying to push to consumers. My contention here is these cloud services will be replaced with something Git like for normal people. For some this will be a question of owning their data and privacy concerns, but for most it will simply work better, and enable a detachment between the application provider and the data storage provider such that events like as Dropbox supporting Office won’t warrant a press release. In the process it will turn our understanding of the web upside down.

Imagine that your phone hosted a git repository of all the files you find interesting: your photos, your email, even maybe a facebook/twitter style feed you post, and that your laptop keeps a copy of the same repo. Whenever the two are nearby the new content from each could automatically hop to the other. Your photos could move to your systems for backup, and not be reliant on the cloud, while you get an even better user experience thanks to it working offline properly. (i.e. this process would still work if you were carrying your phone and laptop around together in the Arctic miles from a network). If you want it then you could host another copy of the repo on a cloud service of your choice (the protocol would be the same as to talk to any other device), but there is no reason that your document editor should care at all about your data storage provider.

The social feeds of your friends would show up on your device thanks to you subscribing to their repo where they publish their information. Your replies get made in your local copy of their repo instantly, and get pushed back to them when a network link between you becomes possible. You couldn’t do Snapchat this way, but then anyone that still believes that you can delete anything from a computer system needs their heads examined.

A more interesting example might be a distributed Wikipedia. In the future we could all carry round complete copies of it, and allow changes we make to propagate between each other based on who we trust to provide accurate information and not based on some central (possibly self appointed) authority. This would solve many problems with today’s Wikipedia, especially over trust within specific domains. It could also result in creation of various branded versions of the content where a certain level of fact checking has been achieved. This shifts websites to a model of local storage, with content quietly pushed to you in the background.

All this would fundamentally disturb the economics of most web businesses, but swings things radically round towards empowering the individual user over their service providers.

It was this thinking that led me to create Umbrella, which is an attempt to wrap JGit with some Bonjour service detection. “Create” is perhaps an optimistic choice of word. I thought of it, but got sidelined by an unrelated surprise wave of work before it got to implementing much at all. Now that has subsided, hopefully in the coming weeks I’ll get to proof-of-concept with it.

Ultimately, let’s be honest, a Starship Enterprise that has to connect to mother earth for everything is not what we dream about. Cloud computing, on the other hand, is precisely what the Borg are into, so if you want to be assimilated into a single minded soul destroying cybernetic organism then that’s totally the way to go.

Nigel Birkenshaw runs Atomirex.