<center><img src="https://i.imgur.com/sP7cyyX.png" alt="drawing" width="200"/></center> # Compose and run a simple workflow in usegalaxy.no <html> <div style="background-color: #f2f3f4"> Tools in **Galaxy** can be run interactively, one by one, or combined into multi-step workflows that can be executed as a single analysis. If compatible, the output from one tool act as input for the next tool. In this hands-on exercise, you will import data from NeLS and create your first workflow. You can access the Norwegian Galaxy at [usegalaxy.no](https://usegalaxy.no/). </div> </html> <br/> :::info :information_source: **Note:** Names of files, folders, commands, and other functionality elements will be highlighted through the exercises in the following way: `file_name_example.txt`. User specific (and thus variable) character strings as for example user_name will be written between `<` and `>` as `<user_name>` to indicate the meaning. ::: ## Import exercise data <html> <div style="background-color: #f2f3f4"> The data for this exercise is stored in the shared project folder in NeLS. Let's go grab them! </div> </html> <br/> I. Create a new history and rename it to `My first workflow` <br/> II. The tools in the **Tools** menu is organized into categories. Expand the `Get Data` category and select `Get files from NeLS storage` <br/> ![](https://i.imgur.com/2jXBf8o.png =200x) <br/> II. This will redirect you to the NeLS portal (you might need to log in). Under `Projects`, browse to the directory `ELIXIR_online_course_2021/demo_workflow/` <br/> III. Select the two files in the `demo_workflow` folder and press `Send to Galaxy` <br/> ![](https://i.imgur.com/WAKhOVz.png =200x) ![](https://i.imgur.com/bnhVBTj.png =210x) ![](https://i.imgur.com/Scr2rPC.png =200x) ![](https://i.imgur.com/bFPdoCD.png =285x) <br/> IV. This will redirect you back to usegalaxy.no. Make sure that the data you imported are present in your current history. There should be two text files named `file1.txt` and `file2.txt` <br/> ![](https://i.imgur.com/p17XVQG.png =200x) <br/> ## Create the workflow <html> <div style="background-color: #f2f3f4"> This workflow uses the basic principles you have learned in the lectures, where you create a string of tools with output from the previous tool as input for the next one. It is only meant for exercise purposes, as it will only produce meaningful results with the text files you imported from NeLS The workflow will consist of five tools (in bold below) that can perform various manipulation of text files. 1. Import two datasets (tabulated text files) using **Input datasets** 2. Sort the data in each file numerically using **Sort data** 3. Cut the specified column from each file using **Cut columns from a table** (not **Cut columns from a table (cut)**) 4. Merge two files side by side using **Paste** 5. Select specified rows (lines) using **Select first** The workflow you are about to create will end up looking like this: <br/> </div> </html> <center><img src="https://i.imgur.com/2d4kTCb.png" alt="drawing" width="800"/></center> <br/> :::info :information_source: Using the the two tab-separated text files you imported from NeLS as input, the workflow should produce a text file as output containing a salute to a masked and swarded fellow, also known as the spanish fox ::: <br/> I. Create a new workflow: Navigate to the top menu and select `Workflow` section [1]. Create a workflow by clicking on `+ Create` button [2] in the right top of the workflow panel and name the new workflow `my first workflow` [3]. Save [4] the new (empty) workflow <br/> <center><img src="https://i.imgur.com/OKohrbm.png" alt="drawing" width="500"/></center> <br/> II. An empty **Workflow editor** will appear. You can add tools to your workflow by browsing the **Tools** menu [1] and selecting the tool [2] you want to include. The selected to will appear in the **Workflow editor** [3]. :::info :information_source: **Tip:** Using the search function in the **Tools** menu will make it easier to find tools. Start typing the name of the tool in the search field [4] and tools in the **Tools** menu will be filtered accordingly [5] ::: <br/> <center><img src="https://i.imgur.com/MLzWZmA.png" alt="drawing" width="800"/></center> <br/> III. Now you can start adding tools and connecting them as outlined below. Each individual tool has its separate parameter settings that you need to adjust. <br/> IV. Start by adding "**Input file**" twice! Add the label "Input file 1" for the first input and "Input file 2" for the second input <br/> V. Add the tool "**Sort**" and adjust the parameters so that the tool will: * label "Sort file 1" * sort on column 4 * alphabetical * ascending order <br/> VI. Add the tool "**Sort**" again and adjust the parameters so that the tool will: * label "Sort file 2" * column 2 * alphabetical * descending order VII. Add the tool "**Cut**" and adjust the parameters so that the tool will: * label "Cut file 1" * cut column c4 <br/> VII. Add the tool "**Cut**" again and adjust the parameters so that the tool will: * label "Cut file 2" * cut column c2 <br/> VIII. Add the tool "**Paste**". * label "Paste file 1 and file 2" <br/> IX. Add the tool "**Select first**" and adjust the parameters so that the tool will: * label "Select first row" * select the first row (number of lines = 1) <br/> X: Connect the tools so that the output from "**Input file 1**" is the input for "**Sort file 1**" and so on. File 1 and 2 will converge at "**Paste file 1 and file 2**" XI. By default each tool write output files to you history [1]. You can change this [2] so that only the output from the last tool is written to your history [3] <br/> <center><img src="https://i.imgur.com/4RDjkuP.png" alt="drawing" width="800"/></center> <br/> :::warning :warning: Remember to save the workflow before closing the **Workflow editor** ::: XII. Make sure you are still in the history with the two text files you imported from NeLS and try to run the workflow <br/> <center><img src="https://i.imgur.com/BsEwXFm.png" alt="drawing" width="800"/></center> <br/> XIII. You will not see any input files because the workflow only accept the data type `tabular` XIV. Change data type of the text files in your history by pressing the `Edit attributes` icon [1]. Select `Datatypes` in the new window that appears and start typing `tabular` [3] in the text field [2]. A drop-down menu will appear and you can select the data type from here [4]. Press `Change datatype` [5] to complete, and repeat for the second file. <br/> <center><img src="https://i.imgur.com/OSZ3XGs.png" alt="drawing" width="800"/></center> <br/> XV. Try to run the workflow again and have a look at the output from the final tool. <br/> # Modify an existing workflow in Galaxy <html> <div style="background-color: #f2f3f4"> In the first part of this exercise you will import an existing workflow, in the second part you will modify it. More specifically, you should replace a tool in the workflow with other tools. In the final part you will run the workflow on proper data. The existing workflow perform taxonomic analysis of random shotgun metagenomic data, and is shared in with you in usealaxy.no. </div> </html> ### Import an existing workflow I. Pressing `Shared` -> `Workflow` [1] will display a list of all workflows that has been shared with all users in usegalaxy.no <br/> II. Press the drop-down menu of the workflow named **"Modify this workflow"** and select `Import` <br/> III. You will be redirected to a new page. Select `Start using this workflow` <br/> <center><img src="https://i.imgur.com/Qo7jOtc.png" alt="drawing" width="800"/></center> <br/> IV. This will take you to the list of all your workflows (imported or selfmade) <br/> V. Select the newly imported workflow. Note that is is given the pre-index "**imported**" [1] in front of the name. You can remove/change this if you like <br/> VI. The workflow can be run by pressing the `Run workflow` button [2] - Don't do this yet! <br/> VII. Instead, press on the name of the workflow and select `Edit`. <br/> <center><img src="https://i.imgur.com/Jgu6jGy.png" alt="drawing" width="800"/></center> <br/> VIII. You should now see the structure of the workflow in the workflow editor with a general description of the workflow next to it [1] and [2] <br/> IX. Select any of the tools in the workflow by clicking on it [3]. A detailed description of the tool will be displayed [4] where you can make pre-setting of the parameters for the tool <br/> X. In this example we pre-set the output format of the image that the tool "**Generate Heatmap**" producing to **PNG** [5] <br/> <center><img src="https://i.imgur.com/n1Z157b.png" alt="drawing" width="800"/></center> #### The existing workflow include the following tools: 1. Input Paired-End dataset 2. Concatenate R1 and R2 tail to head using **Concatenate datasets** 3. Taxonomic profiling of sample using **MetaPhlAn2** 4. Generate heatmaps using hierarchical clustering using "**Generate Heatmap**" ### Modify the imported workflow <html> <div style="background-color: #f2f3f4"> This outputs the taxonomic profile of the sample and a heatmap. Visualization using hierarchical clustering better suitable when comparing multiple samples. Since we are only performing analysis on a single sample, we should change the visualization part of the workflow to a more suitable output. </div> </html> <br/> XI. Go to the overview of your workflows [1] and make a copy of the imported workflow [2] <br/> <center><img src="https://i.imgur.com/WOxzvS4.png" alt="drawing" width="500"/></center> <br/> XII. A copy of the workflow should appear in the overview of your workflows <br/> XIII. Select this workflow [1] and rename it to e.g. "**Modified workflow**" [2]. Delete the "**Generate Heatmap**" tool [3] <br/> <center><img src="https://i.imgur.com/Mbv8YcW.png" alt="drawing" width="800"/></center> <br/> XIV. Add the tool **Format MetaPhlAn2** and **Krona Pie Chart** [1]. The first tool convert the **MetaPhlAn2** output so that the data can be imported into the next tool. :::info :information_source: You can use the **search** function in the **Tool Menu** to find the tools ::: <br/> XV. In the workflow editor, select the **Krona Pie Chart** and pre-set the parameter `What is the type of your input data` to `Tabular` [2] <br/> XVI. Connect the tool such that the output from **MetaPhlAn2** is the input for **Format MetaPhlAn2**, and the output from **Format MetaPhlAn2** is the input to **Krona Pie Chart** [3] <br/> XVII. Save the workflow [4] <br/> <center><img src="https://i.imgur.com/aymfLsM.png" alt="drawing" width="800"/></center> #### The modified workflow should include the following tools: 1. Input Paired-End dataset 2. Concatenate R1 and R2 tail to head using **Concatenate datasets** 3. Taxonomic profiling of sample using **MetaPhlAn2** 4. File format conversion from **MetaPhlAn2** to **Krona** using **Format MetaPhlAn2** 5. Visualisation of taxonomic profile using **Krona Pie Chart** ### Running the modified workflow XVIII. Create a new history and import data here from this shared folder in NeLS `Projects/ELIXIR_online_course_2021/fastq`. It should be two fastq files there. :::info :information_source: You have both created a new history and imported data from NeLS previously in this course. Have a look at these exercises or ask one of the instructors if you need help ::: <br/> XIX. From the overview of your workflows [1], press the `Run workflow` button [2]. <br/> XX. Select the datasets you copied from NeLS as input to the workflow [3]. It is possible to see all the tools in the workflow [4] and change the pre-set parameters for each tool by expanding any of the tools [5] and make changes. These changes will only be applied for this run. <br/> <center><img src="https://i.imgur.com/6TIsNCd.png" alt="drawing" width="800"/></center> <br/> XXI. Press `Run Workflow` and cross your fingers. If all goes well, you should generate a Krona chart that allow you to explore the taxonomic composition of the sample. <br/> :::success :+1: That was the end of this practical. Please remember to type "done" in the chat. :::