PDS Migrator - HackMD

<style> .reveal section img { background:none; border:none; box-shadow:none; } .reveal section .unimportant { opacity: 0.5; } .reveal section[data-section="demo"] { } .reveal section[data-section="demo"] iframe { position:absolute; top: 0; bottom: 0 right: 0; left: 0; } .reveal section blockquote { font-size: 0.85em; width: 100%; } </style>   # PDS Migrator Fighting Link Rot in Solid Data Note: Hi! I am Ben Peachey and I am here today to talk to you about our project: "PDS Migrator". ---   # PDS Interop stands for "**P**ersonal **D**ata **S**tores **Interop**erability" <img src="https://pdsinterop.org/logo/color.svg" height=200> - https://pdsinterop.org/ - https://github.com/pdsinterop Note: I am part of PDS Interop. We work on interopterability between personal data stores. ---   ![](https://solidproject.org/assets/img/solid-emblem.svg) > “Data is a precious thing and will last longer than the systems themselves.” > > ~ Tim Berners-Lee Note: One such data store is Solid. Solid is the latest project Sir Tim Berners-Lee is currently working on. His main idea is that you should have control over your data instead of Facebook or Google. It is part of the movement towards a more decentralized and semantically meaningful web. Solid consists of specifications for how Solid servers (personal data stores, or "Pods") and client applications should interact. Now, for Solid Pods to expose data in a meaningful way, Linked Data is used. Where a normal webpage might contains tens of links, Linked Data can contain hundreds of links, or more! In the past we have contributed to the Test-Suites for Solid server implementations, we've create an Solid server in PHP, and we have created a plugin that allows users to use their Nextcloud instance as a Solid Pod. All of this work is open-source, of course. During our work on Solid projects, the most used Solid Server at the time went out of commision, because its maintainer no longer had the energy and time required to keep things running. Users where given a two week notice to export their data, after that, everthing was gone. This caused a _lot_ of links to break, which was a major hassle, not only for its users, but also for our test-cases and development work of our, and other, teams. This problem, of data no longer being available, is called "Link Rot" ---   # This slide could not be found Note: If you've ever tried to visit a page again only to find it gone (giving you a 404 Not Found message), you have encounter this. Another form of link rot is when the URL is still available, but the data itself is gone or has changed. Link rot is bad enough for normal data, but for Linked Data, the problem becomes much, much worse, as the meaning of one document is lost because a linked document is rotten. ---   ## Fighting link rot 1. Allow easy moving of data between PODs 2. Leave a forwarding trail 3. Update PODs to send correct HTTP headers 4. Support Meta PODs to redirect to current URL 5. Support Archiving and Caching links Note: To fight link rot, the two most important things are: 1. Forwarding 2. Archiving In our solution for Solid Pods, we have to do more work for forwarding, as solutions for archiving already exist. This is reflected in both the specification as well as the implementations. To combat Link Rot in the Solid ecosystem, we need to: 1. Extend the existing specifications 2. Create a reference implementation to demonstrate the specifications 3. Build a test-suite to valide projects correctly implement the specifications ---   ```plantuml @startuml ditaa(scale=1.75) +---------------+ | Draft Spec | +---------------+ +---------------+ | Migrator App | +---------------+ -----=-----------------> We are here +---------------+ | Test Suite | +---------------+ +---------------+ | Solid Servers | +---------------+ +---------------+ | Demo App | +---------------+ @enduml ``` Note: For our specification draft, we have: - completed our research - defined the scope of the specification - created an RDF Onthology We have also created a first prototype of the PDS Migrator client application. This has been done in such a way that the UI and the logic have been created separately from each other, so both parts can be re-used by future projects. During development we have also been creating tutorials to explain how other teams can create client applications and work with Solid in general (and migration specifically). All of this work is open-source and can be found on our GitHub (at https://github.com/pdsinterop/solid-migrator) We need to finalize our specifications, once that is done, we can create test-suites and server implementations. Besides implementing these changes for the Solid servers we have created ourselves, we will also be opening merge-requests with implementations for open-source solid server created by other teams. After that we will create example client applications and finalize the tutorials we have created. ---   . . . . . . . . . . . <iframe src="https://pdsinterop.org/simply-solid-app-tutorials/10.%20UI%20Design/" height="100%" width="100%"></iframe> Note: For the specifications there isn't much to show but text, so today we will be demonstrating our Solid Pod Migrator prototype. So... This is the prototype of our Solid Migrator. To start using the migrator, the user needs to connect to their Solid Pod. I'll be using my pod on the new home of the popular community pod. Once the user is logged in, they need to select a pod they want to use as a source. Once the file and folder listing is loaded, they can select what they want to migrate. Next they need to select a target that they want to migrate to. Once that is done, they can start the migration. To be clear, all of this happens purely in the browser. There are no server side components needed to migrate, other than the pods themselves. ---   ## Business Case <img style="height: 10em;" src="https://i.postimg.cc/RFny0KyX/441411041bi01.jpg"> Government / Healthcare / Education Note: Our business case isn't the PDS Migrator directly. In fact, we don't really expect to make any money from the migrator itself. Our business case is coupled to the success of Solid, as we are working deep within the Solid ecosystem. Part of this success is the stability and reliability of Solid implementation. Our project contributes by adding specifications and implementations that help data in Solid become more portable and improve GDPR complicance. The market we think Solid is a good match for, besides regular consumer use-cases, is - Government - Healthcare - Education As a company we have a lot of experience in these markets. We have several prospects that are interested in using Solid as a solution for their users. One such prospect is related to the Dutch Ministerie of Education. Another is with a commercial partner who works with local governments. To give you an idea: Currently, the British National Healthcare service (the NHS), is doing a pilot with Solid pods. So is the government of Belgium. If a European citizen were to relocate from England to Flanders, currently, they wouldn't be able to migrate their Solid data. Based on previous business-cases of our company, and those of our partners, we are betting on more governments becoming interested in using Solid technology. ---   # The End _**[Your questions go here]**_ - https://pdsinterop.org/solid-migrator/ - https://hackmd.io/@muze/pds-migrator - ben@muze.nl Note: Thats my presentation, we will now be open to answering questions. ###### tags: `solid` `dapsi` `pds-interop` `pds-migrator`