Metadatathon 5th Nov / 16th Dec 2021 / Apr 6th 2022
---
# Welcome!
- Thank you for contributing!
- SPAAM:
- Community of researchers working on ancient metagenomics
- Regular discussion workshops
- 👉 Collaborative projects for the field 👈
---
**ADVERTISMENT**
---
**ADVERTISMENT**
Want to go BIGGER with metadata: standardised metadata sheet for **ALL of ancient DNA**?
</img><img width="40%" src="https://i.imgur.com/uDxFo4v.png"></img><img width="20%" src="https://i.imgur.com/NGwL0rk.png"><img width="40%" src="https://i.imgur.com/Xy2Wxd9.png"></img>
Join MInAS-MG working group! Slack: [#metadata-standards](https://spaam-community.slack.com/channels/metadata-standards)
**Next meeting**: TBA 2022
---
# AncientMetagenomeDir
> AncientMetagenomeDir is a community curated resource of **lists of all published shotgun-sequenced ancient metagenome[s]** or microbial genome-level enriched samples. It is primarily meant to act as a reference guide to **help point researchers toward any relevent public data for comparative analysis**. It is hoped it will help researchers track growth and development of the field of ancient metagenomics over time.
https://github.com/SPAAM-community/AncientMetagenomeDir
---
# Code of Conduct
Reminder: [https://spaam-community.github.io/#/code_of_conduct](https://spaam-community.github.io/#/code_of_conduct)
---
# Issue
- Current sample level metadata doesn't hold _all_ useful metadata
- What about presence of damage? Depth of sequencing?
---
# Aims and Objectives
- Aim: aggregate and standardise **library** level metadata and make **data accessible**
- Objectives:
- Training for use of Git(Hub)
- Aggregate library level metadata for all raw sequencing data
- Build a tool to allow efficient downloading of data based on metadata
---
# Schedule
| Date | Time | Event |
|--------|-----------|------------------------------------------------------------|
| Apr. 5 | 08:30 CEST | **Group Check-in (Asia-Pacific): Welcome and training** |
| Apr. 5 | 09:30 CEST | Metadata processing and programming! |
| Apr. 5 | <...> | Continue... |
| Apr. 6 | 09:00 CEST | **Check-in/out (EMEA/Asia-Pacific): Welcome and training** |
| Apr. 6 | 10:00 CEST | Continue... |
| Apr. 6 | 11:00 CEST | Continue... |
| Apr. 6 | 12:00 CEST | **Lunch break (EMEA)** |
| Apr. 6 | 13:00 CEST | Metadata processing and programming! |
| Apr. 6 | 14:00 CEST | **Check-in (Americas): Welcome and training** |
| Apr. 6 | 15:00 CEST | Metadata processing and programming! |
| Apr. 6 | 16:00 CEST | Continue... |
| Apr. 6 | 17:00 CEST | **Check-out (EMEA)** |
| Apr. 6 | 18:00 CEST | Metadata processing and programming! |
| Apr. 6 | 19:00 CEST | Continue... |
| Apr. 6 | 20:00 CEST | Continue... |
| Apr. 6 | 21:00 CEST | **Check-out (Americas)** |
---
# General procedure
_Check whiteboard!_
1. Check the project board!
2. Assign yourself to an issue, and move the card to 'In progress'
3. Make a new branch **from `dev`**, and adding to the library metadata
a. Note that `RAW.tsv` is already in the library folder for for you,
4. Once it's ready, move your issue to 'Requires review'
---
# General procedure
6. Review another PR, and put the issue you are reviewing 'Under Review'
7. Once you get a review on your PR, make changes as necessary
8. Once your PR is approved and check your issue is moved to Done
9. Check the PR you reviewed has requested re-review
10. Go back to number 1!
---
# **New** Dedicated reviewing team!
<img width="20%" src="https://pbs.twimg.com/profile_images/1079300898000683009/nr8tnhPM_400x400.jpg"> <img width="20%" src="https://pbs.twimg.com/profile_images/829634139527774209/pFE0NFhv_400x400.jpg"> <img width="20%" src="https://images.zapnito.com/users/268977/avatar/medium_IMG_1272-1.jpg"> <img width="20%" src="https://pbs.twimg.com/profile_images/1175500247390461952/6sa4Tus2_400x400.jpg">
<p style="font-size:16pt">Alex (@alexhbner) | Aida (@aidaanva) | Irina (@ivelsko) | James (@jfy133)</p>
**@thedir-metadatathon-reviewers**
---
# Otherwise
- Ask questions to the group
- GitHub, about papers, research etc.
- Work in groups or individually
- Take breaks
- Feel free to socialise etc.
---
# Tutorial
---
<img src="https://i.imgur.com/JOBlRRo.png" width=75%>
---
Check the project board!
---
---
Pick a publication, assign yourself, and move card to `In progress`
---
Back on the `Code` table
<img src="https://i.imgur.com/JOBlRRo.png" width=75%>
---
Switch branch to `dev`
---
Make new branch called `<Publication>-libs`
---
---
---
Open `RAW`
<img src="https://i.imgur.com/4wUYjmy.png" width=75%>
---
Press Download (can re-use across PRs!)
---
Also open 'main' table, press `raw`, copy and paste the entire thing into your preferred spreadsheet tool.
---
<img src="https://i.imgur.com/uV32t2l.png" width=50%>
---
My working setup: Left `_RAW`, right 'main' table
---
To start: just copy and paste!
---
Important: Not all columns have same column name, and some don't have corresponding `RAW` column!
Some require manual checks in the publication
---
Mappings:
| Raw | Cleaned |
|-----|---------|
| project_name | project_name |
|| publication_year |
| publication_doi | publication_doi |
| sample_name | sample_name[*] |
| archive | archive |
| study_accession | archive_project |
| secondary_sample_accession | archive_sample_accession |
| center_name | sequencing_center[@] |
| library_name | library_name |
| | strand_type [#] |
| | library_polymerase |
| | library_treatment|
| | library_concentration |
| instrument_model | instrument_model |
| library_layout | library_layout |
| nominal_length[/] | sequencing_cycles |
| library_strategy | library_strategy |
| read_count | read_count |
| run_accession | archive_run_accession |
| fastq_ftp | download_links |
| fastq_md5 | download_md5s |
| fastq_bytes | download_sizes |
<div style="width=20%">
<blockquote style="font-size:10pt">
[*] check what to do if inconsistency....
[@] sub == unknown
[#] check both methods and SI, search with browser
[/] ENA only! For SRA check publication
</blockquote>
</div>
---
Tips:
- sometimes multiple project accession for each publication check
- Somtimes multiple libraries per sample, per each library (i.e. run) on one line
- This means the comma separated sample-table list will be spearated to one per line
---
Once filled in, do consistency check! Are all IDs matching other/correct order etc across all columns
---
Once complete, **on your branch** edit the 'main' table file, and paste your lines to the bottom, and save!
<p style="font-size:0.7em">
Commit messsage: <code>Add <publication ID> libs</code>
</p>
---
Back on GitHub
---
Make a Pull Request (also Pull Request = new). **Important**: make PR into `dev`
---
Add title, put `x` for checkmarks, add the issue by `This is to close:` by typing `#<keyword>` to find the one saying 'add metadata'
<img src="https://i.imgur.com/jO7oc1Q.png" width="66%">
---
Once you press 'Open PR', check bottom of opened PR for auto checks
<img src="https://i.imgur.com/VzEXHcB.png" width="66%">
---
All good, if not, follow instructions on table 😁 (i.e., check README for guidelines)
<img src="https://i.imgur.com/54CSULx.png" width="50%">
---
Example of errors: check README for guidelines, consider updating `assets/enum/<column_name>.json` **on `master`**
---
If you have to update the `enum` for a particular column, once merged into master run-checks on your PR by going to bottom, press details then 're-run'!
---
Once checks pass, label as `Ready for Review`
---
On progress board, move to `Requires review`
---
Review another PR in the `Requires review` column!
Tutorial on [AncientMetagenomeDir wiki](https://github.com/SPAAM-community/AncientMetagenomeDir/wiki/Reviewing-a-Pull-Request:-Step-by-Step)
---
Back on your branch: if a reviewer requests changes modify your branch again (edit button), make a new commit, and request re-review.
---
Once ✅, onto the next available publication on the Project board 😎!
---
Record all the problems we encounter on the repository wiki!
https://github.com/SPAAM-community/AncientMetagenomeDir/wiki/Issues-Encountered-when-Metadata-Scraping
This will be important for future plans 😈
<style>
@import url("https://fonts.googleapis.com/css?family=Varela+Round:300,400,600");
.reveal section img { background:none; border:none; box-shadow:none; }
.reveal {
font-family: 'Varela Round', sans-serif;
}
body {
background-image: url(https://raw.githubusercontent.com/SPAAM-community/AncientMetagenomeDir/master/assets/images/spaam-AncientMetagenomeDir_logo_colourmode.svg);
background-size: 7.5%;
background-repeat: no-repeat;
background-position: 3% 96%;
font-size: 1.1em;
}
.reveal h1 {
font-size: 1.5em;
font-family: 'Varela Round', sans-serif;
font-weight: bold;
}
.reveal blockquote {
font-size: 0.75em;
width: 95%;
}
.reveal .progress {
color: #732a82;
}
.reveal .controls button {
color: #732a82;
}
.reveal thead {
font-size: 0.3em;
}
.reveal tbody {
font-size: 0.3em;
}
.reveal code {
background-color: #eceaea;
border-radius: 5px;
padding: 2px;
color: #acadb1;
}
quote { width: 95% }
</style>
{"metaMigratedAt":"2023-06-16T13:07:55.513Z","metaMigratedFrom":"YAML","title":"AncientMetagenomeDir Metadatathons 2021","breaks":true,"description":"View the slide with \"Slide Mode\".","slideOptions":"{\"theme\":\"simple\"}","contributors":"[{\"id\":\"6357fb8d-b0f8-4b7d-9fbe-6d813ad67700\",\"add\":14589,\"del\":3634}]"}