<center><img src="https://i.imgur.com/sP7cyyX.png" alt="drawing" width="200"/></center> # Logging in to usegalaxy.no, data transfer and deleting data - Nov 2022 <html> <div style="background-color: #f2f3f4"> In this hands-on exercise, you will log in to **usegalaxy.no**, get to know the web interface and transfer data to and from **usegalaxy.no**. You will also perform some simple manipulations of your data and delete data. You can access the **usegalaxy.no** at [https://usegalaxy.no](https://usegalaxy.no) </div> </html> <br/> :::info :information_source: **Note:** Names of files, folders, commands, and other functionality elements will be highlighted through the exercises in the following way: `file_name_example.txt`. User specific (and thus variable) character strings as for example user_name will be written between `<` and `>` as `<user_name>` to indicate the meaning. ::: ## Login to usegalaxy.no and get to know the user interface <html> <div style="background-color: #f2f3f4"> Galaxy is an open, web-based platform for accessible, reproducible, and transparent computational biomedical research. **usegalaxy.no** is a national Galaxy server for life science data hosted and supported by ELIXIR Norway. **usegalaxy.no** is accessible to all Norwegian users and their collaborators - just like NeLS. Non-FEIDE users (e.g. collaborators) can get access using a **NeLS ID** which can obtained by contacting the **ELIXIR Norway helpdesk**: <contact@bioinfo.no> </div> </html> <br/> I. In a web browser, go to [usegalaxy.no](https://usegalaxy.no). If not already logged in, you will be asked to login by pressing the `Feide or NeLS ID sign in` button and adding your login credentials. <br/> <center><img src="https://i.imgur.com/DSPImhS.png" alt="drawing" width="800"/></center> <br/> II. For first time users, and users with no data in usegalaxy.no the front page should look like below. <br/> <center><img src="https://i.imgur.com/z3WyULD.png" alt="drawing" width="800"/></center> <br/> III. If you have used usegalaxy.no before, you should start a "fresh" history by pressing the `+` sign in the **History panel** (upper right of the screen). <br/> <center><img src="https://i.imgur.com/TcB40pY.png" alt="drawing" width="200"/></center> <br/> IV. Rename your history by pressing on the current name (`Unnamed history`). Give the history any name you prefer, and press `enter`. <br/> <center><img src="https://i.imgur.com/NX1uQ5b.png" alt="drawing" width="200"/></center> <br/> V. Galaxy allows you to add custom tags and annotations to you histories (and datasets) that can make it easier to search/sort your histories, and describe the content. Add a tag [1] and an annotation [2] to your history. <br/> <center><img src="https://i.imgur.com/4VZY7EA.png" alt="drawing" width="800"/></center> <br/> ## Import data from NeLS <html> <div style="background-color: #f2f3f4"> <br/> **usegalaxy.no** is directly connected to your data storage in **NeLS**. This excludes the need for local copy of your data. You can simply transfer data directly between **usegalaxy.no** and **NeLS**. </div> </html> <br/> I. In the **Tools** panel under the `Get Data` section (2), press `Upload File from your computer` (3). You can see in the menu that there also are many other possibilities for uploading data. Alternatively, just press the `Upload Data` button (1). This opens the **upload files dialog** box. <br/> <center><img src="https://i.imgur.com/JNOuKvo.png" alt="drawing" width="800"/></center> <br/> <center><img src="https://i.imgur.com/Kfk9vw4.png" alt="drawing" width="800"/></center> <br/> II. Click on `Choose remote files` (at the bottom). Here you can upload data from many sources, including the NCBI or Ensembl ftp servers or dropbox. You also find `NeLS Storage` and `NeLS Storage Projects` here, which are links to your Personal and Projects NeLS folder, respectively. Click on `NeLS Storage Projects`, and browse to the subfolder `intro_usegalaxy` (The path is: `/Projects/ELIXIR_online_course_2022`). <br/> <center><img src="https://i.imgur.com/nsUyLmH.png" alt="drawing" width="800"/></center> <br/> III. When you are in `intro_usegalaxy`, tick the boxes next to the files named `test_NeLS_import.txt`and `test_NeLS_DNAimport.txt`, and press `Ok`. <br/> <center><img src="https://i.imgur.com/usJPHaP.png" alt="drawing" width="800"/></center> <br/> IV. You will be redirected back to the **upload files dialog** box. You can change the file `Type` (e.g. txt, fasta, bed, fastq, fastqsanger, and many more) for each file separately (1) or all at the same time, if they are all the same type (2). You can also leave the default 'Auto-detect' and let Galaxy attempt to determine the file type. Do this now. You can easily change file type later, in Galaxy, if it is wrong. If the data is associated with any reference genome, for example, reads that are mapped to or will be mapped to a genome, you can specify 'Genome'. For now, leave this unspecified.<br/> Press `Start` (arrow below) to start transferring the files from NeLS to your current usegalaxy.no history. Then press the `Close` button. Data transfer will finish in a few seconds. For large files, it will take much longer. <br/> <center><img src="https://i.imgur.com/FQKfEjo.png" alt="drawing" width="800"/></center> <br/> V. When importing data from **NeLS**, you cannot define the datatype (format). Preview the imported datasets by clicking on the file names [1]. Notice that both files are imported as `txt` [2], but that `test_NeLS_DNAimport.txt` is actually a `fasta file` [3] <br/> V. You now have the two files/datasets as the first two items in your history (red circle below). Press the "eye" button (red arrow) to see the content of one dataset. Then do the same for the other dataset. The `x`/`Delete` button will delete the dataset from your history. Do not do this now, but if you, by mistake, delete a dataset, you can download again from NeLS as described above. <br/> <center><img src="https://i.imgur.com/5vmyH6G.png" alt="drawing" width="800"/></center> <br/> VI. Press the `Edit attributes` button (the little "pencil") for the `test_NeLS_import.txt` dataset (red arrow below). Here you can "Edit attributes", that is, add information about the dataset, add annotation. If the dataset is connected to any reference genome, for example, contains reads that should be mapped to a genome, you could have specified this when the data was imported, as described above. If that was not done, you could have specified reference genome here with the `Database/Build` drop-down menu. As we are just looking at a test file here, we do not need to specify a genome. <br/> <center><img src="https://i.imgur.com/MHQRRVd.png " alt="drawing" width="800"/></center> <br/> VII. In the `Datatypes` card (red arrow below), you can see that Galaxy "auto-detected" that this is an ordinary text file (txt) which is correct. If this was wrong, you could have changed it to the correct data type with the `New Type` drop-down menu. Start typing in the search field to get the list limited to data types that matches your search string, for example `fastq` as seen below. Since the data type is correct, you do not have to change it. Check that the data type is correct for the `test_NeLS_DNAimport.txt` dataset. It is a fasta file, also correct. <br/> <center><img src="https://i.imgur.com/BROALDe.png" alt="drawing" width="800"/></center> <br/> ## Export data from usegalaxy <html> <div style="background-color: #f2f3f4"> Datasets and histories can be exported from **usegalaxy.no** either to your local system or to the **NeLS storage**. Since **usegalaxy.no** is not meant for storage, we encourage you to move your data when your analysis is done. First, you will run a simple tool that turn the fasta file into another dataset, so that we can export a different file than we imported. You will learn more about tools later in the course. </div> </html> <br/> I. In the **Tools** panel under the `FASTA/FASTQ` category [1] locate the tool `FASTA-to-Tabular` [2]. Select `test_NeLS_DNAimport.txt` as input [3] and press `Execute` to run the tool. <br/> <center><img src="https://i.imgur.com/lBb3A2D.png" alt="drawing" width="800"/></center> <br/> As you might have noticed, each dataset/job in the history is shown in a color that reflects its status. Green means finished/ok. <br/> <center><img src="https://i.imgur.com/ZM7eVUr.png " alt="drawing" width="800"/></center> <br/> II. Galaxy will run the tool and produce a new dataset. This will be listed at the top of your history and contain a tab-separated file with the fasta header in the first column and the sequence in the second column. <br/> III. Click on the new dataset to expand the information that comes with it. Preview the file content and datatype. Press on `View details` [1]. Can you tell which **version of the tool** (i.e., "Galaxy tool ID") that was used? <br/> IV. Dataset names produced by Galaxy are informative in that they describe the tool and the input data, but sometimes it can be useful to give it a different name. Rename the new dataset to `test_NeLS_export` by pressing `Edit attributes` [2]. Select `Attributes` [3], rename the dataset [4] and press `Save` [5]. <br/> <center><img src="https://i.imgur.com/KbLZ6WY.png" alt="drawing" width="800"/></center> <br/> V. Download this file to your local system by pressing `Download` (the floppy disk icon) and save the file to your local disk. <br/> VI. Export the `test_NeLS_export` dataset file to **NeLS**. In the **Tools** panel under the `Send Data` section (1), press `Export datasets` (2). On the resulting page, make sure `test_NeLS_export` is chosen/highlighted. <br/> <center><img src="https://i.imgur.com/o1H3vE5.png" alt="drawing" width="800"/></center> <br/> Press the `Select` button and then `NeLS Storage`. Browse to where you want to put the file, and press `Select this folder`. <br/> <center><img src="https://i.imgur.com/WPwuYVd.png" alt="drawing" width="300"/></center> <br/> <br/> <center><img src="https://i.imgur.com/F8T6d00.png" alt="drawing" width="400"/></center> <br/> Back on the **Export datasets** page, press `Execute`. An export job starts and will finish in a few seconds. You get a new "Export Logs" item in your history. Check **NeLS**, as you learnt before, and make sure the file is where you just put it. <br/> <center><img src="https://i.imgur.com/Z1A9QLc.png" alt="drawing" width="800"/></center> <br/> VII. Now export the complete history as a single file to **NeLS** by pressing `History options` (1) (upper right corner, the little cogwheel) and `Export History` under the `NeLS storage` section (2). <br/> <center><img src="https://i.imgur.com/tgWWjz3.png" alt="drawing" width="200"/> <img src="https://i.imgur.com/OpzL9rb.png" alt="drawing" width="200"/></center> <br/> You will be redirected to the **NeLS portal** where you need to choose the destination folder. Since you do not have permissions to write to the shared project folder in **NeLS**, you need to export it into your `Personal` folder. Browse to your `Personal` folder (or a subfolder here) and press `Use current folder` (3). You should be sent back to **usegalaxy.no** where you can view the transfer progress (you might need to scroll down a bit in the **usegalaxy.no** main window). When the transfer is complete, the progression bar will turn green (4). If you want to double check, you can open a new web browser and check that the file is present in the folder you chose in (3). <br/> <center><img src="https://i.imgur.com/1B6KbCD.png" alt="drawing" width="350"/> <img src="https://i.imgur.com/8mtxbuU.png" alt="drawing" width="350"/></center> <br/> ## Sharing history in usegalaxy <html> <div style="background-color: #f2f3f4"> Datasets and histories can be shared with other users of **usegalaxy.no**. You can choose to share histories (or only datasets) with all users, a single user or via an URL. Users you have shared the history with, can import it and continue working on their copy of your history. They will not have access to your personal history and cannot edit your version of the history. You can unshare the history at any time, but users that have imported the history will still have their copy of it. </div> </html> <br/> I. Press the `History option` and select `Share or Publish` (1). In the new window that appear, you have to switch on `Make History accessible` (2) to make it accessible via a link. <br/> <center><img src="https://i.imgur.com/trSANza.png" alt="drawing" width="800"/></center> <br/> II. You should get an URL that you can share with other **usegalaxy.no** users (3). You also have the option to make the history accessible to all **usegalaxy.no** users with `Make History publicly available...` (4). (5) allows you to share the history with individual **usegalaxy.no** users via their **usegalaxy.no** users emails. <br/> <center><img src="https://i.imgur.com/7MdHX3E.png" alt="drawing" width="400"/></center> <br/> ## Importing shared history in usegalaxy.no <html> <div style="background-color: #f2f3f4"> Histories that have been shared by other users in **usegalaxy.no** can be imported from the `Shared Data`. </div> </html> <br/> I. Press the `Shared Data` and select `Histories` [1] <br/> II. From the list of shared histories, press on the history named `ELIXIR workshop history` [2]. <br/> III. In the new window that apperar, a short description of the history and the datasets are presented. Press the `+` sign [3] <br/> IV. You can now choose to import the history [4]. You can also rename the history before importing it <br/> V. The imported history will now be the active history [5] <br/> <center><img src="https://i.imgur.com/isYhyrg.png" alt="drawing" width="800"/></center> <br/> ## Importing history from NeLS in usegalaxy.no <html> <div style="background-color: #f2f3f4"> Histories can also be imported from a project in **NeLS** or your local machine - given that you have histories stored here. </div> </html> <br/> I. Importing history from NeLS is done from the `History option`. Press `Import History` [1] under `NeLS Storage`. <br/> II. You will be redirected to the **NeLS storage**. Browse to the history you exported to you personal folder earlier in this exercise, select the file [2] (tick the checkbox) and press `Send to Galaxy` [3]. <br/> III. You will be redirected back to **usegalaxy.no** and you can monitor the progress of the import [4]. The progress bar will turn green when the import is complete [5]. <br/> IV. The name of the imported history will be `imported from archive: ....`. The history can be accessed by selecting `Histories` [6] under the **User**" menu. You can rename the history if you like. <br/> <center><img src="https://i.imgur.com/2SBoHsy.png" alt="drawing" width="800"/></center> <br/> ## Change between histories <html> <div style="background-color: #f2f3f4"> Normally you will accumulate histories in Galaxy. In this part of the exercise, you will move from the active history into another history, which will become the active history. </div> </html> <br/> I. You can access the overview of your histories two ways; by selecting `Histories` [1] under the **"User"** menu or by pressing `View all histories` [2] in the "**History**" panel. <br/> II. Depending on which method you use, the list of your histories will be displayed in slightly different manner. <br/> III. You can change from the current history to any histories in the list by either selecting the history and click on `Switch` [3] or by clicking on `Switch to` [4] above the history you want to change to. <br/> IV. The history you chose will become the active (current) history. To get back to the "frontpage" of galaxy, click the little "house" button at the top, the `Tools and Current History` button (A). <br/> <center><img src="https://i.imgur.com/WNEgCmN.png" alt="drawing" width="800"/></center> <br/> <center><img src="https://i.imgur.com/PNU41Sy.png" alt="drawing" width="800"/></center> <br/> ## Delete datasets and complete histories <html> <div style="background-color: #f2f3f4"> Galaxy offers multiple ways to delete datasets and histories, and you will try two of them in this exercise. There are two major types of deletion: 1. Delete datasets or histories - This option temporary delete data (basically just hide the data) 2. Permanently delete datasets or histories - This option permanently delete data from the disk NB! Even if you select to permanently deletes data, the available space in your galaxy disk quota does not change before the “Trash bin” has been emptied. This will happen automatically every week. </div> </html> <br/> I. Make sure your current history is the history you imported from **NeLS** previously (if you didn't rename it, it should be called something that starts with `imported from archive: ....`). <br/> II. Select the dataset(s) you want to delete (temporary) by pressing `X` next to the dataset in your "**History**" panel [1]. The dataset should disappear from your history, but you still can access it. For example, it here says "2 shown, 1 deleted" [2]. If you click on the word "deleted", information about it will be shown with an orange box [3]. <br/> III. You have only temporary deleted the data (equal to moving it to the trash bin on your computer), and it can be undeleted (moved back from trash bin), or permanently deleted by pressing `Permanently remove it from disk` [4]. <br/> IV. Press `OK` to confirm the permanent deletion [4]. <br/> V. You can still see traces of the dataset in your history [5], but the data in the file is deleted from the disk and you have freed up disk space. <br/> <center><img src="https://i.imgur.com/WsL65bF.png" alt="drawing" width="800"/></center> <br/> VI. Similarly, you can temporary and permanently delete histories. Access the overview of your histories in either of two ways described earlier; by selecting `Histories` [1] under the "**User**" menu or by pressing `View all histories` [2] in the "**History**" panel. <br/> VII. From the drop-down menu of the history you want to delete (still the imported history from NeLS named `imported from archive: ....`), press `Delete Permanently` [3]. If you chose the other history view, from the drop-down menu of the history you want to delete press `Purge` [4]. Both options will ask you to confirm the deletion, and both options will erase the data from disk. :::info :information_source: We strongly encourage that you copy your histories to NeLS and permanently delete data from usegalaxy.no when you are done with your data analysis. This will free up your personal disk space in usegalaxy.no ::: <br/> <center><img src="https://i.imgur.com/6LrbAtv.png" alt="drawing" width="800"/></center> <br/> :::success :+1: That was the end of this practical. Please remember to type "done" in the chat. :::