Recent debates about whether people should be allowed to make their own changes to the bitcoin protocol have highlighted an important notion: perhaps developing Bitcoin Core, the reference version of the code, isn’t the only way for people to contribute.
A recent alteration to the bitcoin code that made its way into a Linux variant called Gentoo left some people fuming before the developer switched it off by default.
“These will never be merged into the bitcoin repository on Github, but people that want to use them can,” bitcoin lead developer Wladimir J van der Laan said.
But what is Github, why does van der Laan have the authority to choose what goes in it, and how does bitcoin get developed in the first place?
How bitcoin is developed
The reference implementation for bitcoin protocol is called the Bitcoin Core. This is the code that Satoshi originally handed down to a core group of developers before disappearing.
Those “disciples” now maintain that code, along with the help of a broader community of developers. The focus is on making the code more efficient, but doing it carefully, and conservatively, so that nothing gets broken.
Bitcoin Core is managed using a software version control system called Git. This enables people to keep track of which versions of their code they are working on, and what changes they have made.
Bitcoin developers running Git on their computers connect to a central service so that they can all work on versions the same project at once. This service, called Github, has many different projects maintained by different groups of people. Bitcoin is one of those projects and it has its own Github page.
The code for the project is held in a single place on Github, called a repository. The official, deployable version of the bitcoin repository is known as the upstream repository, but people who want to work on their own changes to the code can create their own versions of the repository, by copying it into an online ‘fork’.
Developers can modify their forks as much as they like. They can ask for their fork to be merged back into the master repository by issuing a ‘pull request’, which opens up their version of the repository to other project members, who can review it and comment on it.
“The idea is that other developers in the community will review the change,” explained van der Laan. “Then, the submitter fixes the issues brought up by others. It may also be needed to rally some people to test the change, especially if it is complicated, or if there is a subjective component (ie, for UI or RPC changes).”
If enough people like the changes made in a pull request, then it gets merged back into the master repository. But who actually gets to merge the pull?
It turns out that there is a bitcoin priesthood, of sorts, that stewards what finally makes it into the Bitcoin Core code. Van der Laan, chief scientist and former lead developer Gavin Andresen, Jeff Garzik, Gregory Maxwell, and Pieter Wuille are the team who make the final decision, and that isn’t something that’s decided by voting, as you might find in a democracy.
“Single Github repositories are not democratic,” van der Laan explained. “Its maintainers cooperate on development and decide what is merged and when, and what is not. Difficult technical issues are not solved by popular voting.”
BIPS and pull requests
Where possible, though, bitcoin development typically operates via popular consensus. There are two categories of change, broadly speaking.
The Bitcoin Core is maintained in an intentionally conservative way, and most changes are made in a “non-controversial and janitorial” way, van der Laan said. They deal with small, incremental changes, rather than large, revolutionary ones. A bitcoin patch might move some code around to make it more readable, or perhaps optimise some memory usage.
There is another class of changes to bitcoin that have far more ramifications, and those are ones that change the consensus rules. The consensus rules are the technical rules that all bitcoin clients must adhere to for the bitcoin network to operate properly.
“Those have to be scrutinized closely. They have to be discussed on the mailing list first, and there must be a BIP, and the pulls are generally controversial and stay open for a long time to discuss,” he said.
A BIP – short for Bitcoin Improvement Proposal – is a document suggesting a global change to some aspect of bitcoin. It can extend to things outside Bitcoin Core, including mobile wallets or key generation in hardware wallets. It can also govern processes around bitcoin, like changes to the decision making process.
Anyone can create a BIP, as long as they’re written in this format. The community talks about it, and if people like it, its status can be changed to “active” or “final”.
Changes along these lines are the change in BIP 62, which was a change dealing with the transaction malleability flaw in bitcoin.
What improves the chance of a proposed change being implemented in the protocol? It helps for the author of a BIP to have written an example of the code for people to test out and review, van der Laan added.
Review and approval
Bitcoin consultant and security auditor Sergio Lerner would like to see more formalisation for the code approval process.
“When you see a pull request that has been merged, it’s difficult to tell who approved it [and to] what extent the patch was reviewed,” he said. “You have to read a lot of comments and some ‘+1′ which you can interpret as ‘I agree to merge it’, but you can also interpret it as ‘I like it, but I haven’t really reviewed the code.’”
Lerner would like to see a multi-signature patch approval process, in which a certain proportion of developers formally approve the code by signing off the review. That would be a bigger version of the process currently used in some wallets, where multiple signatures have to be used for a bitcoin address to be used.
Other things Lerner would like to see include a log of bugs found and an analysis of why they were not caught on time, a per-patch, security-focused external code review, a formal description of documentation that should accompany a patch and a description of what reviewing a patch actually means.
“Does it mean a line by line source code review? Does it mean checking if the documentation of the change is enough?” Lerner asked. “Does it mean analysing the change against known attack vectors?”
The problem is that this all takes time and human resources, Lerner said:
“Obviously implementing all this requires more housekeeping, a higher budget, and more core developer resources (which currently are scarce). But a software that maintains an industry of $6bn requires it.”
Beyond Bitcoin Core
While Lerner outlines some requirements for code reviews, van der Laan echoes Gavin Andresen’s keynote speech at the Bitcoin 2014 conference, where he said that more could be done to streamline BIP approval.
“The BIP process could use some work. I would be happy if developers of other (full) node implementations were more active in commenting on proposals (or coming up with proposals),” he said.
Andresen also proposes moving BIP discussion and other cross-implementation concerns from the general bitcoin-development mailing list to a specific BIP mailing list.
Just as with software development on an open source project, the onus is always on users to make it happen.
“As it is inherently a global, distributed, disorganized process it’s no single organization’s job to manage the BIP process, so the onus would be here on people and organizations that care to band together and do something,” van der Laan suggested.
But shouldn’t the Bitcoin Foundation, bitcoin’s chief trade organisation, be looking after such things? No, he argues. Instead, things in the bitcoin world are expanding beyond that, and the development team welcomes different implementations of bitcoin.
Van der Laan said:
“Gavin’s talk at Bitcoin 2014 made it clear that his focus is on diversifying. He talked about different full node implementation, even said ‘more is better’. Even though maintaining Bitcoin Core is my job, I tend to agree with that.”
The onus should no longer be on the development of Bitcoin Core, van der Laan believes.
“In the initial years Bitcoin Core was maybe excessively important, and its developers had to keep the light on for the node infrastructure (and stay up at night to patch bugs as they appear). But, moving forward, for bitcoin to be the global distributed system it was supposed to be, we should move beyond that.”
So, there may be a benevolent priesthood for Bitcoin Core, in the sense that the final decision about what goes into the the code rests with a small group of people. But that doesn’t mean that this group wants things to be exclusive or elitist – far from it.
At least some of the core developers are actively encouraging others to expand the network with their own implementations, on the assumption that the majority of them will stick with the consensus rules. Those that don’t will fall out of sync, making it obvious who is in the minority and forcing them to fix it.
Evolving bitcoin in that direction could create room for the kinds of policy variances that some people have been asking for, while preserving the consensus rules: the parts that truly make bitcoin what it is. It would also ease the pressure on an overburdened set of people trying to support the technology underpinning a quickly growing business. And, done correctly, it might introduce some of the new processes that participants like Lerner are asking for.
The question is: how will bitcoin evolve such a variety of alternative implementations cleanly, efficiently and without any associated drama?
Diversify image via Shutterstock