Open Development: The Necessary Next Step for Open Source

tags: article / in process

This is the meant to be the beginning of a discussion on the topic of Open Development. It outlines how we think about it at Blockchain Commons, and why we think it's important. Please join us in those discussions on how we can make Open Development a reality.

The term "open source" gets a lot of attention. It usually refers to the open-source model, where developers collaboratively create software and then release their source code for free distribution, modification, and usage. The idea has entered the modern culture: definitions can be found in the Meriam-Webster Dictionary^[1] and the OED^[2].

Open-source software is very important to a variety of modern computer platforms. It's the heart of blog publishing in WordPress, the core of cryptocurrency in Bitcoin, and the foundation of web browsers in projects such as Firefox and Chromium. Unfortunately, open source is all too often seen as all that's needed. Though there are innovators who are thinking about open source as part of a larger ecosystem^[3], they're the exceptions. We need to more broadly realize that the open-source model needs to become a larger open development movement^[4] that can fill in the gaps and resolve the problems of traditional open-source software.

The Limits of Open Source

The biggest limitation of the traditional open-source model is that it's just a licensing regime: no less, no more. A license describes how software can be distributed and used but it doesn't require a project to be designed, developed, or supported in an open way. And those additional levels of consideration are crucially important: whereas the open-source model describes how you can use a product, a larger open-development movement is needed to reveal whether that product is actually useful or not.

Without open development, open source is full of limitations and disadvantages. These issues are not requirements of the open-source model, but they are obvious and expected consequences of open-source software as it currently exists, which focuses on specific licenses that are used for software created largely by volunteer teams of development.

Unfortunately, the open-source model doesn't even necessarily do its licensing well. The Free Software movement and then the open-source model that followed innovated free and open licenses, from copyleft to the GPL and the MIT License. However, open-source software organizations have been reluctant to update these classic licenses to the modern day. We need something not just newer, but better.

Beyond that, many disadvantages of open source stem from the fact than open-source projects tend to be volunteer-led. As a result, improvements found in open-source projects are typically driven by a developer "scratching … a personal itch" ^[5]. Though this certainly can create innovation, it's not necessarily directed toward production releases. Even though open-source advocate Eric S. Raymond encouraged involving users and customers, that often doesn't happen. As a result, open-source software has a reputation for being hard to use, with poor user interfaces and poor documentation^[6].

Finally, there's the issue of long-term support: what happens to open-source software after it's been released? As the initial developers move on and the years stack up? As new threats and the ever-changing internet require new changes? Too often, this continuing support isn't considered in the open-source model.

Limitations with licenses, development models, and support models all point toward the need for a larger open-development movement, as a level-up of the traditional open-source model.

Principles of Open Development

Open development is a larger infrastructure that encompasses open-source licenses but does so while laying out a complete methodology for these projects, from their initial conception through their long-term support. It is built on what we see as the intrinsic goal of the open-source model: for a community to create common resources that can be freely used for the good of all. However, open development looks at this goal as part of an ecosystem that goes well beyond the idea of just developing code and releasing it under an open license.

To be truly open, and to support development from inception through long-term support, we believe an open-development community must increasingly abide by a set of principles as it rises up through the spectrum of openess.

These principles include:

1. Accessibility. An open-development community must be open to all comers. There must be clearly stated rules for how to join the community, how to participate in the community, and how to remain a member of the community. Though there certainly may be (and should be) ways to remove problematic members from the community, the reasons and procedures for doing so should be clearly stated as well.

2. Collaboration. An open-development community must be collaborative. It must have ways that people can make themselves heard and can help to determine the community's direction through collective choice. This representation and collaboration should encompass developers, users, and distributors alike. The democratic and collaborative choice of an open-development community might be a true democracy, where everyone gets a vote, but more likely it's some other form of decision making that reflects the needs and limitations of an online community, such as rough consensus, a meritocracy, or representative democracy. Existing communities have already resolved this issue in well-considered ways, and should be considered models. ^[7][8]

3. Diversity. A truly open community must not only accept diversity of ideas, but also seek it out in order to improve its development. This should include finding people from many walks of life and with many backgrounds who think in many different ways and have many different skills: people of diverse cultures, classes, genders, races, and points of view. Studies have shown that deep-level diversity is beneficial to the core creativity of a project ^[9], and open-development projects should embrace that. Open development must further embrace "vertical" diversity: a community should include not just designers and developers but also potential consumers (users) and potential distributors (often, businesses). Through early interactions with people who might use and who might sell a product, an open-development community can conquer many of open source's current problems with usability.

4. Strategy. The collaboration of open development must be about a community, not just the individuals. That means that it needs to be able to thoughtfully plan, not just to respond to the whims of whoever is programming at the time. In order for the accessibility, diversity, and representation of an open-development community to actually matter, the community must always move forward in consideration with the needs, desires, and thoughts of its many members.

5. Transparency. Similarly, an open-development community must demonstrate that it is abiding by the desires of its members through full transparency, encompassing its membership, its goals, its governance, its processes, and its ongoing support. Obviously, code should be open, but so should membership statistics, meeting minutes, voting results, and more. The idea of "open" should be embraced throughout the community. This open transparency should not be confused with radical transparency, which has its own pitfalls ^[10]: transparency needs to be balanced with the privacy of developers, patrons, and clients, and it needs to be managed to enhance the goals of open development, not to deteriorate the work of its members.

6. Sustainability. Open development must be something that consumers and distributors can depend on over the entire lifespan of the software. That means that a concept of long-term commitment must be built in from the beginning, to provide for both upgrades and for fixes for the project. (The lack of this type of support is one of the biggest limitations of the traditional open-source model to date.)

7. Openness. The final principle of open development is simply openness. An open-development community must ultimately commit to interoperability within its ecosystem such that there is no lock-in, allowing users the opportunity for open choice among ecosystem elements and the ability to move their own information and assets to other platforms as they see fit.

Best Practices of Open Development

Open development isn't a binary choice, but rather a spectrum. No one, no matter how deeply committed they are to open development, is likely to immediately encompass the entire set of principles. However, software development can become increasingly open and increasingly dedicated to these principles as it incorporates additional best practices.

What follows is a description of seven levels for open development, each of which adds new best practices to the ones before.

Level 0: Open Source

A minimal level of open source represents the classic understanding of the term: code is placed under a license, made vaguely available to the public, and that might be it.

• Code is largely available for viewing.
• Code is largely available for use under some open source license.
• Some code may be restricted in viewing or usage based on hardware interfaces, software interfaces, or simply the privacy needs of the creator. There may be reasonable security, trade secrets, or licensing restrictions. These restrictions and their reasoning may not even be well defined.
• Non-open code must be UX or backends or servers.

Level 1: Inspectable

The lowest bar for open development requires that current code (not some old, dusty forgotten version) can be inspected.

• Code is versioned.
• Code is available via git in a public repo.
• Public code remains relatively current.
• Code in private repos is regularly published.

Level 2: Observable

For an open-open development project to be observable, more than the code is open: the whole development process is.

• Git Issues are used to record bugs and feature requests.
• Git Branches are used to process incorporation of new code.
• Git Releases are used to record official publications of code.
• Docs record parts of code that are not openly Inspectable and Observable, as well as documenting why.

(Though this level specifies "Git", equally open alternative systems would also meet the best practices.)

Level 3: Reproducible

Reproducibility considers those unpublished parts of a code base, and promises that code can still be compiled and tested, even without them.

• Stubs or fake services are used to fill in for code that is not open.
• Code is fully compilable and runnable using stubs.

Level 4: Testable

A minor advance on reproducability requires the creators of code to go the extra mile to help developers test their compiled code.

• Open test suites allow full testing of reproducible code.

Level 5: Cooperative

The next level of open development turns members of the public from observers into potential contributors.

• Road Maps are shared.
• Specs are shared.
• A person is dedicated to coordination of public contributions.
• Public can file issues and will receive responses in a reasonable amount of time.
• Public can contribute pulls, which will be considered.
• Public is clearly defined to include people not closely aligned with creators.

Level 6: Distributed

When an open-development project has reached the Distributed level, it has largely embraced the open-development principles.

• A product has been shipped.
• At least two companies or organizations are committed to the project.
• Open specifications prevent vendor lock-in.
• Open specifications prevent single points of failure.

Level 7: Standardized

Finally, standardized open-development can create tools available to everyone in the ecosystem.

• Specifications preventing vendor lock-in are under an IP Regime of an international standards organization.
• Specification defining project interoperability have been approved as international standards.

Integrating with Open Silicon

Even an open-software model that has been extended to Level 7 of Open Development doesn't fill in the whole picture. That's because any software is ultimately dependent on the hardware that's running it. Perhaps this isn't of much importance for the average software program, but it becomes crucially important when you have software that focuses on privacy, security, cryptography, or related categories. For those, you must have Open Silicon to ensure that the openly developed open software does what you expect it to.

The Principles of Open Development can be applied to Open Silicon with some challenges.

1. Accessibility. Obviously, open-software developers need to be invited into Open Silicon communities, so that they can ensure that their software can be properly represented in the silicon. But, there should also be way for external hardware developers to join as well.
2. Collaboration. Both open-software developers and external hardware developers should be able to make their own contributions to the silicon designs (or at the least to the goals of the silicon design).
3. Strategy. To create meaningful strategic plans, software developers and chip designers must work together on the requirements of silicon. Our Silicon Salons offer an example of what this strategic collaboration looks like for the cryptographic industry.
4. Diversity. Bringing in the open-software community should notably improve the diversity of silicon-design communities.
5. Transparency. Unfortunately, transparency will be one of the biggest challenges for Open Silicon. No matter how much silicon designers might want to be transparent, they're also going to be bound by IP restrictions that are unfortunately endemic to the interwoven field of chip design. A concerted effort to offer simulation of hardware that is linked to open-software development should be prioritized to help improve transparency despite these issues.
6. Sustainability. There may be less concerns about sustainability in Open Silicon, because the chips won't have the same issues that software tends to have with bitrot: where functioning apps stop working due to changes in lower level technologies. Nonetheless, it needs to still be an issue for consideration.
7. Openness. Interoperability among chips may be a sticking point due to the high cost of entry into the field, but it nonetheless is a crucial requirement to integrate with open-software models, since they need to be able to count on a reliable, deterministic experience across all hardware.

Some of the best practices may face even more challenges than these principles due not only to IP restrictions within the extant semiconductor field but also issues with reproducibility. Parallel best practices may need to be outlined for hardware-focused development. See the Musings on Open Silicon for more thoughts on these and other challenges.

Open Questions

Open Development's principles and best practices form a starting point for discussing open development. There's still a lot more to consider, including:

• Black Boxes. Black boxes that are part of an open system, but have been purposefully closed (which includes IP-restricted silicon), are obviously problematic. We need to better codify how they can be incorporated into open systems, and what we might need to verify about their inputs and outputs.
• Contracts. NDAs and other contracts are a common element of business life. Best practices for open development should ensure that they don't negatively impact the openness of the project except as they meet the same criteria for why elements of development might not be open in the first place.
• Ethics. Obviously, ethics also need to be a consideration in open development, especially because the openness of development could expose its developers to free riding and other threats. However, to date legal attempts to institute ethics, such as with licenses that require updates to be given back to the community, may have done more to make development uninteresting to potential partners.

Conclusion

Open development is much bigger than just open source. It requires committing to certain principles and adopting best practices that create a much larger ecosystem that ensures both the openness and the resilience of the technology being developed. In many case, that includes digging all the way down to the hardware.

This article provides a starting point for what's required to make that larger ecosystem function. It's intended to start conversations to help us better codify the goals of open development, its needs, and ultimately the best practices that will get us there.

Please just us to further discuss the topic of Open Development.

Appendix: The Heartbleed Use Case

The 2012-2014 Heartbleed bug in OpenSSL demonstrated why long-term commitment is a crucial element for open development.

When TLS first appeared in the late '90s, it was part of an enthusiastic, well-supported drive for improved internet security. Companies like Microsoft, Netscape, and Consensus Development were expending extensive commercial time and effort on the development of these standards, which in turn led to the creation of companies worth billions of dollars.

Fast forward fifteen years and the success of TLS was obvious. It secured almost all of the commerce on the internet, with about 60% of that occurring through a single open-source library: OpenSSL. Unfortunately, OpenSSL has never been properly supported. In fact, by spring 2014, OpenSSL was being developed by a single engineer working quarter time. Inevitably, this led to a major problem: Heartbleed, a massive security vulnerability that affected almost every commerce server on the internet.

This is a common problem in the open-source model. The source is created by enthusiastic developers, but eventually, they move on. Meanwhile, the corporations that are making extensive use of the open-source projects aren't supporting the community. This is one of the prime reasons that a larger open-development movement is required.

Notes

This is currently a mature draft, but we are seeking comments, especially on the following issues (some of which are expansions and explanations of topics in the "Open Questions" section).

1. How can we integrate these practices with hardware development?
   • Open sourcing hardware is very difficult because of the patents and NDAs that are rife in the space. Even if a hardware manufacturer wants to be totally open, compromises will be required because of other manufacturers that they're working with. How can we accomodate this without undercutting the core precepts we're laying out for open development? Can we even do so in a single document, or do we need to write separate open development principles for hardware?
2. How can we better integrate continued support?
   • Support is one of the linchpins of open development, but even so it's difficult to entirely codify its complexities. Among the open questions are:
   • When should open source be moved to a community support model if a creator isn't able to support it themselves?
   • When does it become appropriate to fork an unsupported project and begin support yourself?
   • What should the standards be for credit following either move?
3. How can ideas from Ostrom's Commons be integrated into the foundation of Open Development?
   • What can we introduce from Ostrom's Commons that we think is critical?
   • How can we do so without teaching the entirety of Ostrom's Commons?
   • What can we say about the various limited resources of the open development framework, such as programmer time, review time, monies to pay for reviews, etc?
4. What do we say about the difference between a cooperative and participatory ecosystem and a winners-takes-all ecosystem?
5. How do we allow for an integration of the possibilty for other agreements beyond open source and open development agreements?