# OCM nlnet questions

Hi Michiel,

Thanks for, as always, great questions! Here are our answers:

> Dear Michiel,

> you applied to the 2023-04 open call from NLnet. We have some questions regarding your project proposal Open Cloud Mesh.

> You requested 50000 euro. Can you provide some more detail on how you arrived at this amount? Could you provide a breakdown of the main tasks, and the associated effort? What rates did you use?

The requested amount of 50,000 euros for the Open Cloud Mesh project is intended to support the activities of 4 of the 6 team members, namely Michiel, Yashar, Mahdi, and Sandro, who will be working on the project for 250 each, at 50 euros per hour. Gianmaria and Giuseppe will work an additional 150 hours each in CERN time, this is CERN's in-kind contribution to this project. So we will have a total hours budget of 1300 hours. The budget breakdown and effort allocation for the main tasks are as follows:
1. Improving the specification and documentation: This task involves refining and expanding the existing specification text to a more comprehensive RFC-style document. It also includes enhancing the documentation around the protocol. The effort allocated for this task is estimated at 250 hours.
2. Protocol extensibility and compatibility: This task aims to make the Open Cloud Mesh protocol extensible without breaking backward compatibility and ensuring graceful degradation. It involves implementing better discovery and negotiation mechanisms and a better versioning policy and communication around backwards compatibility. The estimated effort for this task is 100 hours.
3. Support for additional use cases: The project aims to address specific use cases that have emerged from previous projects, such as bidirectional invites, federated groups, and multi-protocol shares. This task involves designing and implementing the necessary features to support these use cases. The estimated effort for this task is 200 hours.
4. Enhancing the test suite: The goal is to improve the existing test suite to enable it to run reliably in Continuous Integration (CI) for all vendors. We may stay with the current Puppeteer-based implementation, or perhaps align with the open source GUI tests that SUNET have developed. The estimated effort for this task is 150 hours.
5. Testing and bug fixing: The team will test different implementations of the protocol and provide pull requests and bug reports to address any incompatibilities found. This task requires thorough testing and collaboration with the implementing vendors. The estimated effort for this task is 250 hours.
6. Security self-review and vulnerability assessment: This task involves conducting a comprehensive security analysis of the Open Cloud Mesh project, identifying potential attack vectors, and considering necessary security measures. The estimated effort for this task is 100 hours.
7. Performance optimization: This task focuses on improving the performance of the Open Cloud Mesh protocol by analyzing and optimizing various components. The team will identify bottlenecks and implement optimizations to enhance the protocol's efficiency. The estimated effort for this task is 100 hours.
8. User interface design and improvement: This task involves enhancing the user interface of the Open Cloud Mesh project, making it more intuitive, user-friendly, and visually appealing. The team will gather user feedback and implement design improvements. The estimated effort for this task is 50 hours.
9. Community engagement and outreach: This task aims to engage with the community, gather feedback, and promote the Open Cloud Mesh project. The team will participate in relevant events, communicate with users, and provide support and guidance. The estimated effort for this task is 50 hours.
10. Project management and coordination: This task involves overall project management, including task coordination, scheduling, and communication among team members. The project manager will ensure smooth progress and effective collaboration. The estimated effort for this task is 50 hours.

In summary, the requested budget of 50,000 euros is based on 4 times 250 hours for the funded team members.

> Can you describe the concrete problem you are trying to solve in a concise way?

The concrete problem that this project aims to solve is the lack of substance and quality in the specification and test suite of the Open Cloud Mesh protocol. While the protocol is already being implemented and used by major Enterprise File Sync and Share (EFSS) vendors and National Research and Education Networks (NRENs), there are existing gaps and inconsistencies that need to be addressed. The project seeks to improve the specification text, develop a more comprehensive and RFC-style document, enhance the test suite for reliable Continuous Integration (CI) testing, and identify and clarify any incompatibilities between implementations. By doing so, the project aims to provide a robust and standardized framework for secure and efficient cloud-based file sharing and synchronization.

> Is there a standardisation venue this will be hosted at? Obviously, there is the liability of software patent protection - some obvious large US vendors missing - but in general it would be good to understand the level of built-in scrutiny that would be applied to the standardisation effort?

The proposed team consists of the current core contributors to the Open Cloud Mesh project who have worked on various production implementations of the protocol. Additionally, the project has the support of advisors from implementing vendors and high-profile users of OCM, including representatives from Nextcloud, ownCloud, Seafile, CERN, SUNET, Helmholtz, and others. This indicates that there is a collaborative and knowledgeable renowned community involved in the project that will contribute to the scrutiny and review of the standardization efforts.

In this project we mostly aim to fix the compatibility between existing implementations, and the quality of the test suite and specification. We will not significantly change how Open Cloud Mesh already works in production. Nonetheless we will discuss with the original author Björn Schiessle whether he is OK with submitting the resulting specification to the IETF for their RFC track, so that it will receive even more scrutiny and exposure than it has already received through Géant and the CS3 community until now.

> What is your technical assessment of the current state of the spec? To what extend was your own original internet draft stronger, and where did OCM improve upon it?

We think Open Cloud Mesh excels in its simplicity and its limited scope, and it has exceptionally good implementation experience, which has led for instance to the detailed handling of "reshare" (the situation where Alice shares a resource with Bob, and Bob re-shares this to Charlie, there is a well-tuned mechanism for squashing this two-hop share into one "reshare" directly from Alice to Charlie, and all the edge cases of that have been tried in production by many users over the years).

Assuming that by "your own original internet draft" you refer to https://datatracker.ietf.org/doc/draft-dejong-remotestorage, here the scope is different: remoteStorage is a protocol between personal data stores and applications, describing CRUD access and auth. Open Cloud Mesh is a protocol between two personal cloud servers, describing the notification of "Hello Bob, Alice wants to share X with you". So the two are actually complementary. You could in fact send an OCM share to someone, and in its details set the data access protocol for that share to "remotestorage" or to "solid" or to "ftp" etc, instead of to "webdav" which is currently the default.

> [question about link between OCM and ScienceMesh]

> Is there some form of E2EE, and if so how is key distribution handled? Could you reflect on ERIS?

ERIS would be a great fit for OCM, thanks for the pointer! You could use OCM to share an ERIS read capability, instead of using it to share the password to a webdav folder, which is the default way OCM is currently used in the wild. We are already defining additional protocols such as "data transfer" (i.e., rclone copy) and "web application" (i.e., a deeplink into a web application that can be viewed with a browser), we will define "ERIS read capability" as an additional one!

> What are you ideas on the potential to use content-based routing, distributed filesystems and DHT technologies?

Such systems can be used to host the content (where until now, webdav is usually used), and then OCM can be used to send the share that points to it.

At another level you could say that a DHT is an automated network of storage servers, whereas OCM helps to create a hand-curated network of storage servers. OCM allows the user to always be in charge of their own "sovereign" server, and in addition hand-pick which files or folders should be shared with other sovereign servers of other people.

> What kind of authentication mechanisms are supported? Are standards like SASL being considered?

When sending/receiving an OCM share, the users identify each other as "user@host". When accessing the content, webdav + http basic auth is generally used. The token that is sent as part of the share details is used as the http basic auth username, and the http basic auth password is left empty. This is a peculiar practice which was not really documented, and we recently found it by reverse-engineering existing OCM implementations in Nextcloud and ownCloud 10. While we will have to keep supporting this way of authenticating for backwards compatibility with existing deployments, SASL would be a great way forward for OCM, to decouple authentication from OCM's main task of standardising the server-to-server call that gestures one user's intent of sharing a resource with another user at another server. We'll definitely discuss if we can use SASL as a framework, for the sending and the receiving server to negotiate which authentication mechanism is available for accessing the shared resource.

> Will these tests be made reproducible? What are the implementations you would be looking at?

Yes! We will be testing Nextcloud, ownCloud 10 (the PHP-based one), ownCloud OCIS (the GO-based one), CERNBox (note that OCIS and CERNBox are two different configurations, each forked off the shared Reva code base), Seafile, and OCM-stub (which as the name says, is just a stub that helps us test an OCM implementation in isolation). It is our goal to get all of these 6 implementations of OCM to run the OCM test suite as part of their continuous integration, and we will also send them PRs to help them get all our tests to go green.

Running the tests in continuous integration instead of running them manually also ensures that machine-readable steps-to-reproduce are always available.

In more detail:
The OCM test suite, available at https://github.com/pondersource/ocm-test-suite, serves as a valuable mechanism for ensuring the reproducibility of tests and verifying the expected functionality of the OCM protocol across different vendors.

By utilizing the OCM test suite, vendors implementing the OCM protocol can run a standardized set of tests that assess the compatibility and compliance of their implementations. This test suite provides a collection of test cases, scenarios, and associated tools specifically designed to evaluate OCM functionality.

The test suite covers various aspects of the OCM protocol, including its core features, interoperability, performance, and security. It aims to validate that different OCM implementations behave consistently and correctly according to the defined specifications and requirements.

By using a shared and openly accessible test suite, vendors can ensure that their OCM implementations are tested against a common set of test cases. This promotes transparency, collaboration, and alignment among vendors in adhering to the OCM protocol's standards and specifications.

The use of a test suite like the one provided for OCM enhances reproducibility by enabling vendors and other stakeholders to run the same set of tests and obtain consistent results. It allows for verification and validation of OCM implementations across different vendors and environments.

> Looking forward to your answer and thank you very much for your timely reply. And have a great week!

Thank you!

The OCM team:
Michiel de Jong (independent / Ponder Source)
Sandro Mesterheide (independent)
Yashar PourMohammad (independent / Ponder Source)
Mahdi Baghbani (independent / Ponder Source)
Giuseppe LoPresti (CERN)
Gianmaria Del Monte (CERN)

With support from:
Björn Schiessle (Nextcloud, and original main author of OCM)
Holger Dryoff (ownCloud)
Jonathan Xu (Seafile)
Andreas Klotz (Helmholtz-Zentrum Berlin/HIFIS)
Guido Aben (SUNET)
Oliver Keeble (CERN)