# Milestones list

Topics (deliverables) for milestones might be:
### 1 test suite update 3.5k
The existing [OCM test suite](https://github.com/cs3org/ocm-test-suite) is outdated in a few ways:
it uses now-deprecated versions of the systems under test, it doesn't have some of the custom scripts we have been adding for development setups of similar projects, and it uses Puppeteer instead of the now-preferred Cypress as the headless browser. In this milestone, we will rebase the test suite on the latest [Ponder Source dev-stock scripts](https://github.com/pondersource/dev-stock) and add the Cypress test runner, to show we can test a basic check (the appearance of a login page) in GitHub actions.

### 2 fixes of the share-with flow tests 6.5k
We will deliver a rewrite of the share-with flow tests on top of Cypress, testing all combinations of the systems that support it, so Nextcloud -> Nextcloud, Nextcloud -> OC-10, OC-10 -> Nextcloud, and OC-10 -> OC-10. We will also add tests for OCM-to-group, in addition to the OCM-to-user test that already exists in the current Puppeteer-based test suite.

### 3 fixes of the public-link flow tests 4.5k
We will deliver a rewrite of the public-link flow tests on top of Cypress, testing both login-before and login-after, for all combinations of the systems that support it, so Nextcloud -> Nextcloud, Nextcloud -> OC-10, OC-10 -> Nextcloud, and OC-10 -> OC-10.

### 4 development of the invite flow tests 6.5k
We will develop automated tests of the invite flow (as used in ScienceMesh), testing all combinations of the systems that support it, namely Nextcloud+Reva, OC-10+Reva, CERNBox, and OCIS.

### 5 spec text development 8k
We will evolve the spec text to better describe notifications and any other feature that would benefit from a spec review. Furthermore, we will migrate from the current OpenAPI spec to a proper IETF Internet Draft.

### 6 design of new features 8k
We will design the specification for new features such as federated groups, MFA awareness, open-in-app flow, etc, and add these to the spec as optional discoverable capabilities.

### 7 tests for new features 4k
We will add tests to the test suite for systems that already support these new features, if any.

### 8 PRs and extensions (including OC-10 OCM app) to fix implementations 9k
In a previous project (executed by Ponder Source and funded by SURF), we already developed the [OCM app for OC-10](https://github.com/pondersource/oc-opencloudmesh) which fixes the lack of OCM-to-groups support in OC-10. We will update this app as necessary to pass the tests in all directions, and also provide bug reports and PRs to fix to Nextcloud, OCIS and CERNBox as and where we can.

3.5+6.5+4.5+6.5+8+8+4+9=50k

---
Money spent so far until 4 December -
Towards milestone 1:
* Sandro 1 day = 500
* Mahdi 2 months (Oct&Nov) = 3000

Towards milestone 8:
* Mahdi 2 months (Aug&Sep) = 3000 fixing the invite-first flow in Nextcloud+Reva and OC-10+Reva.
* Giuseppe 2 months (Aug&Sep) = 0 (in-kind contribution) fixing and completing the invite-first flow in Reva for Nextcloud 26+, OC-10 and CERNBox.

Money spent between 4 December and 8 February:
* Mahdi: 3 weeeks 4-22 Dec, then 4 weeks holidays + 11 days, until 8 Feb. starting on 10 February, Mahdi will work full-time again
* Sandro: how many?

---
# NGI grant application for Open Cloud Mesh

## Discussion
https://github.com/cs3org/OCM-API/issues/67
## Grant details
https://nlnet.nl/core/

## Thoughts:

## Web Form Answers
### General
Thematic Call: NGI Zero Core
Contact Information: Michiel de Jong
Project Name: Open Cloud Mesh
Website: https://github.com/cs3org/OCM-API

### Abstract: Can you explain the whole project and its expected outcome(s).
The Open Cloud Mesh protocol is implemented by some major Enterprise File Sync and Share (EFSS) vendors, and used in production by some major National Research and Education Networks (NRENs) and other organisations. But its specification and test suite are still lacking in substance and quality.

We came together at the recent CS3 conference in Barcelona, and now have momentum to improve the specification text, flesh it out to a more RFC-style text that addresses all aspects and considerations of the protocol, improve the test suite so that it can be run in Continuous Integration (CI) instead of requiring frequent manual intervention, and clarify any incompatibilities we find between implementations.

### Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?
The proposed team consists of:
Michiel de Jong (independent)
Sandro Mesterheide (independent)
Yashar PourMohammed (independent)

with the additional help from:
Giuseppe Lo Presti (CERN, unfunded)
Gianmaria Del Monte (CERN, unfunded)

And we count with the support of the following advisors from implementing vendors and from high-profile users of OCM:
Björn Schiessle (Nextcloud, and original main author of OCM)
Holger Dryoff (ownCloud)
Jonathan Xu (Seafile)
Andreas Klotz (Helmholtz-Zentrum Berlin/HIFIS)
Guido Aben (SUNET)
Oliver Keeble (CERN)

In the closing session of the CS3 conference we have announced the formation of this team and its list of advisors as the next step in the history of Open Cloud Mesh.

Sandro, Michiel, Giuseppe, Gianmaria and Andreas have all been core contributors to the Open Cloud Mesh project over the past years and have all worked with the details of various production implementations of the protocol.

Giuseppe is part of the CERN-IT team that created the open source implementation of Open Cloud Mesh that is now used in production at CERN.
Andreas leads a project for HIFIS to support federation-type shares on the Nextcloud stack. Sandro was contracted by HIFIS (through Nextcloud) to carry out the implementation of this new feature.
Michiel (through Ponder Source) has carried out a project SURF to add this same feature on the ownCloud stack and wrote the current OCM test suite.
Together, we came to a proposal of how federation-type shares can be added to the Open Cloud Mesh protocol, and to an agreement that the specification and test suite needs some more collaborative development work.

### Requested amount
50,000 euros

### Explain what the requested budget will be used for? Does the project have other funding sources, both past and present?
We request funding to support the activities of Michiel, Yashar and Sandro, at an hourly rate of 50 euros per hour.

CERN contributes the hours worked by Giuseppe and Gianmaria.
All advisors also work pro bono.

Since 2015, work on the Open Cloud Mesh protocol has taken place between vendors of EFSS systems, and its development was at first funded by these pioneering vendors (most notably ownCloud, Nextcloud, pydio and Seafile).

When the significance of this project became apparent to a number of NRENs, their umbrella organization Terena/Géant sponsored the project for some time (previously https://wiki.geant.org/display/OCM/Open+Cloud+Mesh).

Later, the development of the Open Cloud Mesh test suite was commissioned by CS3MESH4EOSC ("ScienceMesh") consortium and carried out by Ponder Source, but this funding is now also ending.

Independently, Helmholtz through intermediaries contracted Sandro Mesterheide to work on technology development that was also closely related to the Open Cloud Mesh.

Since the recent CS3 conference in Barcelona we have gathered a consensus between all main stakeholders, including and several high-profile scientific institutes including CERN and Helmholtz, to do some more work on the specification text, documentation, test suite, and extensibility of the protocol (see https://github.com/cs3org/OCM-API/issues/67).

So the reason we are requesting this NGI grant is that the CS3MESH4EOSC grant, which has supported Open Cloud Mesh for the past 3 years, is coming to an end, but there is amply momentum both on the vendor side and in the user community to continue work on Open Cloud Mesh.

The work we want to do includes:
* improving the text of the specification, and the documentation around it
* improve the existing test suite, to make it run reliably in CI for all vendors
* make the protocol extensible without breaking backwards compatibility and graceful degradation
* support use cases that came up during the ScienceMesh project, such as bidirectional invites and multi-protocol shares
* support requirements from the recent Helmholtz and SURF projects around federation-type shares
* test implementations and provide pull requests and bug reports where we find incompatibilities that can be fixed
* finalize the implementation of OCM-to-group for the PHP-based version of ownCloud, which was started as part of a project that Ponder Source executed for SURF.

### Compare your own project with existing or historical efforts.
There are not many server-to-server protocols for establishing and managing shared resources.

Centralized platforms like Google Docs have always had the option of sharing a document with another user on the same platform, but not outside of it.

The Fediverse allows sharing resources (mostly read-only) through ActivityPub but this functionality (and its address space) is not generally connected into EFSS systems.

In https://github.com/Decentralized-Sharing-Working-Group we explored this topic together with Cozy Cloud one year before
Open Cloud Mesh started getting more traction, and that effort was basically superseded by it,
see https://github.com/Decentralized-Sharing-Working-Group/internet-draft/issues/9#issue-163706977 .

The actual transfer of the resource, once the share is established, which by default this happens with a server-to-server WebDAV mount, is out of scope for Open Cloud Mesh, so Open Cloud Mesh does not replace WebDAV.

### What are significant technical challenges you expect to solve during the project, if any?)

Managing the trade off betweeen on the one hand keeping the specification compatible with existing deployments, while on the other hand enabling corrections and evolution of the protocol.

Dealing with the various graphical user interfaces of EFSS systems from a headless browsers as the test suite code tests how shares can be created, accessed, and removed.

Reverse-engineering how the various implementations have interpreted the currently underspecified parts of the specification, such as the relation between the sharedSecret for the WebDAV protocol, and the actual credentials sent in a WebDAV request.

Understanding all the edge cases and advanced features such as reshare requests, different whitelisting practices for both the sending and the receiving side, and step-by-step Gantt charts of endpoint discovery and server-to-server interaction.

### Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes? (E.g. which actors will you involve? Who should run or deploy your solution to make it a success?)
The Open Cloud Mesh protocol plays a central role in the CS3 community, which recently celebrated its annual get-together in Barcelona, with over 100 in-person participants https://www.cs3community.org/

It also plays a central role in the CS3MESH4EOSC ("ScienceMesh") project, which received 6 mln euros of EU funding over the last 3 years and which will hopefully find new funding in 2024.

This activity around the Open Cloud Mesh protocol would in itself be very helpful to keep the collaboration active between research organisations such as CERN, HIFIS, SURF, CESNET etcetera, and EFSS vendors that implement Open Cloud Mesh such as Nextcloud, ownCloud, and Seafile.

### Attachments
(none)