Bitcoin wallet providers Darkwallet and Armory performed the best in an independent test that sought to identify the strongest wallet offerings for financial privacy.
Darkwallet version 0.8.0 and Armory version 0.93.1 both received top scores of 45 out of 100 from the Open Bitcoin Privacy Project (OBPP) Spring 2015 Wallet Privacy Rating Report, the first release from the open-source organisation that seeks to promote and improve privacy in the bitcoin ecosystem. Scores were based on performance in five categories that assessed the privacy the wallets offered from blockchain observers and network observers, among other factors.
Ranking last in the study was Coinbase‘s online wallet, which received a total score of eight out of 100. Coinbase notably received no points in three of the five categories, while elsewhere, Mycelium, the Bitcoin Wallet and Electrum rounded out the top five wallets.
OBPP contributing engineer Justus Ranvier said he hopes the report will be the first in a triannual series of releases aimed at providing a way for consumers to better evaluate marketing efforts around bitcoin wallets.
Ranvier told CoinDesk:
“For people to make informed privacy tradeoffs, they need informed information about what they are trading off. I’m not aware of any other group that has attempted to quantify privacy, everything before has been pretty vague.”
As evidence to how this approach is beneficial, Ranvier noted that, while Darkwallet and Armory scored similarly when evaluated, the former company more heavily advertised its privacy benefits to bitcoin users.
“Darkwallet put in lots of features, but the fact that Armory has a full node makes a huge difference,” he said.
Wallets were evaluated using a threat model, he explained, a process by which such statements are assessed through the use of measures similar to ones deployed by hackers.
Additional OBPP participants who conducted testing included Blockchain Security Engineer Kristov Atlas and OpenBazaar operations lead Sam Patterson.
First of many
As a first trial, Ranvier indicated that the OBPP team had a much larger list of wallets it was seeking to test, but that ultimately only 10 were chosen due to resource challenges. Final selections, he said, were based on contributor perceptions of the popularity of the offerings.
Going forward, Ranvier said he expects to revisit the findings every three months, at which point more wallets could be rated. The engineer went on to explain this initial release was meant to start the conversation surrounding privacy, and he is seeking to evolve the assessments with user feedback.
“We don’t expect our rating model is perfect but we think it can get better over time. It gives people something to work with when they’re trying to compare features,” Ranvier continued.
To assess these more specific marks, OBPP divided its five major categories into 14 sub-categories, each of which was assigned one or more of three classifications – usability, quality and feedback.
Ranvier said he envisions similar testing could be done to assess bitcoin wallet security, but there are no plans for OBPP to take up the task at this time.
Usability and quality
Two wallets built primarily with security and privacy as core focuses – Darkwallet and Armory – were praised as generally successful in their aspirations by the OBPP team.
Darkwallet scored the highest in the receiving address generation and backup categories, with a 16 of 21 category score. The wallet received the lowest marks in the privacy it offers from network observers, taking just three of 25 points.
By contrast, Armory scored its highest marks in the privacy its wallets offer from network observers, with its lowest marks awarded for the privacy it offers transaction recipients.
Darkwallet and Armory were the top two in overall usability and quality, with Armory taking the top spot in quality and Darkwallet scoring the best in the usability category. Ranvier described quality as a metric to determine how well bitcoin wallets were able to deliver privacy functions, while usability measured the effort users would have to exert to unlock benefits.
Airbitz tops feedback rankings
While the usability and quality rankings showed similarity in terms of the top offerings, OBPP found that Airbitz version 1.4.6 performed the best in terms of feedback, or how well the product warns users they could be compromising their privacy.
Airbitz was largely propelled to this top spot based on its strong performances in the address generation and backup, and change address generation and backup categories, but received no points for the privacy it awarded transaction recipients.
“AirBitz was one of the first mobile wallets to use an HD architecture, which permits it to easily protect user privacy by automatically generating new addresses for receipt of funds and change,” the report authors wrote.
Airbitz ranked fifth in terms of usability and eighth in quality. The Bitcoin Wallet also performed strongly in this category, ranking second.
Coinbase and Airbitz declined to comment on the report.
Privacy image via Shutterstock