<center><img src="https://i.imgur.com/sP7cyyX.png" alt="drawing" width="200"/></center> # NeLS and Storebioinfo storage exercises <html> <div style="background-color: #f2f3f4"> In this hands-on exercise, you will log in to the **NeLS Portal** and practice how to maintain data in NeLS projects and StoreBioinfo projects, two different storage layers in the NeLS architecture. You can access the **NeLS portal** at [https://nels.bioinfo.no/nels-web/#/]. </div> </html> ## Access your NeLS projects in the NeLS portal <html> <div style="background-color: #f2f3f4"> The key mechanism to collaborate on data in **NeLS**, is through shared data in a NeLS project, where project members will have different **roles**, giving different **access rights**. To get access to a NeLS project, you would normally either need to be - added to an exisiting project as a member by a PI or DM of that project - added to a new project upon creation by the Elixir Norway helpdesk (<contact@bioinfo.no>) In this training you will have access to some training projects for the exercises. </div> </html> I. In a web browser go to the [NeLS portal](https://nels.bioinfo.no/nels-web/#/). <br/> II. If you are not already logged in, you need to select Feide or NeLS idp, add your username and password when promted. <br/> III. Click the `Projects` tab in the top menu, and review the list of NeLS projects you are a member of. - Which roles have you been assigned in the different projects? - What can you do in these roles? <div> <center> <img src="https://i.imgur.com/qAF58jx.png" alt="drawing" width="600"/></center> </div> <br/> <br/> IV. Inspect - How many fastq files are there in total in the `ELIXIR_online_course_2022 project`? - Navigate around in the project to get familarized with the folder structure. <br/> V. Upload/transfer - At the top level of your assigned training project `NeLS_training_project_X`, create a new folder called `your-name-and-todays-date`. - (If you get re-directed back to `Personal`, navigate back to your assigned training project under the `Projects` tab) - Upload a small sample file from your computer into your new folder (find a fancy image/logo on the web and save it to a file on your computer if you can't find a suitable test file). - Copy a file from the `ELIXIR_online_course_2022` project, and Paste it into your new folder in your training project. <br/> VI. Share with course mate - Agree with a course mate (or the breakout room instructor) that you add eachother as "Normal User" to each others NeLS projects (tip: check the <u>Members</u> link in the Project overview page). - If possible, team up with a person that is not already member of the same NeLS training project as yourself. - This is also a good time to see who else has access and roles in the same project as you! - Enter the new project you have gotten access to, and find the folder with data that your course mate prepared in step V., and inspect the content there. <!-- - Look into other folders at the top level of your assigned training project 'NeLS_training_project_X', to see what data the other course participants have uploaded and shared with the members of this project. - This is also a good time to see who else has access and roles in the same project as you! (Tip: check the <u>Members</u> link in the Project overview page). - NB: Normally users with PI or DM roles in a project, can add additional members with a given role to that project. This is the primary mechanism to share data with others in NeLS --> VII: Workshop sync: Please enter **"Done with NeLS part"** in the breakout room chat :) ## Access your StoreBioinfo projects in the NeLS portal <html> <div style="background-color: #f2f3f4"> The **StoreBioinfo (SBI)** layer has large capacity, intended for collecting authorative orginals of all data of a project, and would usually allow fewer project members write/modify access. A SBI project can hold one or several **datasets**. A dataset holds data of a certain type, i.e. Illumina sequencing data., and will have different type of top level sub datatype folders depending on the type of data. These will correspond to a high-level structure of: - Raw data - Processed/derived data - Analysis/interpretation level data If you have two types of omics data in your project, you would typically organize the raw data files and the derived data/analysis resulting from each omics technology into a corresponding dataset reflecting the origin of the data. If you have two different datasets obtained at different times, but using the same high-throughput technology, you would create two different datasets to reflect how files naturally belong together in your project. </div> </html> I. If not already logged in to [NeLS portal](https://nels.bioinfo.no/nels-web/#/), please do. II. Click the "StoreBioinfo" tab in the menu. You will then see a panel view, with your StoreBioinfo projects to the right. Review the list of SBI projects you are a member of. - Which role have you been assigned in the project you have access to? - What can you do in this role? <center><img src="https://i.imgur.com/0upIT8e.png" alt="drawing" width="600"/></center> <html> <div style="background-color: #f2f3f4"<br/>> <br/>Next you will try out to transfer some data from SBI to NeLS. This you may need to do from time to time, if the data you need for analysis or similar is only available in SBI (where all data of the project are (should be) stored) and not in the more limited space available in the NeLS storage layer (where hot/active copies of data should live). </div> </html> III. Find a fasta or fastq file (contains sequence data) in the `SBI_training_project` inside the `SBI_training_dataset` and transfer it to your assigned `NeLS_training_project_X` (somewhere under the folder with your name). <html> <div style="background-color: #f2f3f4"> An important task of a Data Manager is to transfer any new valuable data from the NeLS Storage layer to StoreBioinfo. In the next series of steps you will first prepare some data in your NeLS traninig project, create a suitable place to store it inside the `SBI_training_project`, and transfer all the data in one transfer operation. </div> </html> IV. Prepare a folder structure witin your assigned NeLS training project: simply a new folder with some fastq files inside it, possibly organized into more subfolder etc. V. Create a new Data Set in the `SBI_training_project` of datatype Illumina_seq_data, and name it `Your-Name-Todays-Date` VI. Then transfer your prepared data (select the top folder) into the correct subdatatype of your new Data Set (tip: reads). Note: When you click the `Data Operations` button, a new job will appear of your `Data Operations` list to the right. It may take some time to get started and completed depending on other simultanous transfers ongoing. You can safely navigate around in NeLS, and monitor your job under `Data Operations` while doing other tasks. <html> <div style="background-color: #f2f3f4"> Then collaborate a bit on your data :) </div> </html> VI. Ask a course mate to inspect your Data Set (and possibly retrieve a file back to his/her Personal area). Return the favor to your course mate:) VII: Workshop sync: Please enter **"Done with SBI part"** in the breakout room chat :)