--- tags: BMMB554-23 --- [![](https://imgs.xkcd.com/comics/file_extensions.png)](https://xkcd.com/1301) # Python 5 - Processing files ## Quiz ### Prep 1. Go to https://colab.research.google.com 2. Log in with your PSU account and create a new notebook 3. Complete the quiz 4. Share you notebook with `aun1@psu.edu`. Make sure you set me as an editor: ![](https://i.imgur.com/StyngZm.png) ### Questions #### Question 1 (50 pts) Using a `for` loop write code that will calculate the sum of the numbers 1 through 100. :::info Remember that you can use `range(101)` function to create a sequence of numbers from 0 to 100 ::: #### Question 2 (50 pts) Given the following dictionary: ```python= table = { 'ATA':'I', 'ATC':'I', 'ATT':'I', 'ATG':'M', 'ACA':'T', 'ACC':'T', 'ACG':'T', 'ACT':'T', 'AAC':'N', 'AAT':'N', 'AAA':'K', 'AAG':'K', 'AGC':'S', 'AGT':'S', 'AGA':'R', 'AGG':'R', 'CTA':'L', 'CTC':'L', 'CTG':'L', 'CTT':'L', 'CCA':'P', 'CCC':'P', 'CCG':'P', 'CCT':'P', 'CAC':'H', 'CAT':'H', 'CAA':'Q', 'CAG':'Q', 'CGA':'R', 'CGC':'R', 'CGG':'R', 'CGT':'R', 'GTA':'V', 'GTC':'V', 'GTG':'V', 'GTT':'V', 'GCA':'A', 'GCC':'A', 'GCG':'A', 'GCT':'A', 'GAC':'D', 'GAT':'D', 'GAA':'E', 'GAG':'E', 'GGA':'G', 'GGC':'G', 'GGG':'G', 'GGT':'G', 'TCA':'S', 'TCC':'S', 'TCG':'S', 'TCT':'S', 'TTC':'F', 'TTT':'F', 'TTA':'L', 'TTG':'L', 'TAC':'Y', 'TAT':'Y', 'TAA':'_', 'TAG':'_', 'TGC':'C', 'TGT':'C', 'TGA':'_', 'TGG':'W', } ``` write a snippet of code that translates the following DNA sequence: ```ATGCGT``` :::info Remember that you can split a string into codons by iterating over it like this: ```python= seq = 'ATGCGT' for i in range(0, len(seq), 3): codon = seq[i:i+3] .... ``` ::: #### Bonus question (+30 pts) Write a function called `is_sorted` that takes a list as a parameter and returns `True` if the list is sorted in ascending order and `False` otherwise. For example: ```python= is_sorted([1, 2, 2]) True is_sorted(['b', 'a']) False ``` ## Reading and writing files in Python First let's download a file we will be using to your notebooks: ```= !wget https://raw.githubusercontent.com/nekrut/BMMB554/master/2023/data/l9/mt_cds.fa ``` In Python, you can handle files using the built-in `open` function. The `open` function creates a file object, which you can use to read, write, or modify the file. Here's an example of how to open a file for reading: ```python= f = open("mt_cds.fa", "r") ``` In this example, the open function takes two arguments: the name of the file, and the mode in which you want to open the file. The `r` mode indicates that you want to open the file for reading. After you've opened the file, you can read its contents using the read method: ```python= contents = f.read() print(contents) ``` You can also read the file line by line using the readline method: ```python= line = f.readline() print(line) ``` When you're done reading the file, you should close it using the close method: ```python= f.close() ``` You can also use the `with` statement to automatically close the file when you're done: ```py= with open("mt_cds.fa", "r") as f: contents = f.read() print(contents) ``` You can also write to files using `write` method (note the `"w"` mode): ```py= f = open("sample.txt", "w") f.write("This is a new line.") f.close() ``` If you open an existing file in write mode, its contents will be overwritten. If you want to append to an existing file instead, you can use the `"a"` mode: ```py= f = open("sample.txt", "a") f.write("This is another line.") f.close() ``` When you're writing to a file, it's important to make sure you close the file when you're done. If you don't close the file, any changes you make may not be saved. In addition to reading and writing text files, you can also use Python to handle binary files, such as images or audio files. Let's download an image: ``` !wget https://imgs.xkcd.com/comics/file_extensions.png ``` Here's an example of how to read an image file: ```py= with open("file_extensions.png", "rb") as f: contents = f.read() ``` Note that when working with binary files, you must use the `"rb"` mode for reading and the `"wb"` mode for writing. There are many more features and methods related to file handling in Python, but the basics covered here should be enough to get you started. ## [Fasta](https://en.wikipedia.org/wiki/FASTA_format) Fasta is a file format that is commonly used to store biological sequences, such as DNA or protein sequences. In Python, you can read a Fasta file by opening the file, reading the lines one by one, and processing the data as needed. Here's an example of how you might read a Fasta file in Python: ```py= sequences = {} with open("mt_cds.fa", "r") as file: header = "" sequence = "" for line in file: line=line.rstrip() if line.startswith('>'): if header != "": sequences[header] = sequence sequence = "" header = line[1:] else: sequence += line if header != "": sequences[header] = sequence ``` The code above uses a `with` statement to open the file and read the lines one by one. If a line starts with a `">"`, it is assumed to be a header, and the current sequence is stored in the dictionary using the current header as the key. If the line does not start with a `">"`, it is assumed to be part of the current sequence. ## [FASTQ](https://en.wikipedia.org/wiki/FASTQ_format) Let's download a sample fastq file: ```py= !wget https://raw.githubusercontent.com/nekrut/BMMB554/master/2023/data/l9/reads.fq ``` Fastq is a file format that is commonly used to store high-throughput sequencing data. It consists of a series of records, each of which includes a header, a sequence, a quality score header, and a quality score string. In Python, you can read a Fastq file by opening the file, reading the lines four at a time, and processing the data as needed. Here's an example of how you might read a Fastq file in Python: ```py= def read_fastq(file_path): records = [] with open(file_path, "r") as f: while True: header = f.readline().strip() if header == "": break sequence = f.readline().strip() quality_header = f.readline().strip() quality = f.readline().strip() records.append((header, sequence, quality)) return records ``` In this example, the `read_fastq` function takes a file path as an argument, and returns a list of records, where each record is a tuple of four strings: the header, the sequence, the quality score header, and the quality score string. The function uses a `while` loop to read the lines four at a time until the end of the file is reached. You can use this function to read a Fastq file like this: ```py= records = read_fastq("reads.fq") for header, sequence, quality_header, quality in records: print(header) print(sequence) print(quality_header) print(quality) ``` This will print the headers, sequences, quality score headers, and quality scores in the Fastq file. You can modify the read_fastq function to process the data in any way you need. ## [SAM](https://en.wikipedia.org/wiki/SAM_(file_format)) Let's download an example SAM file: ```= !wget https://raw.githubusercontent.com/nekrut/BMMB554/master/2023/data/l9/sam_example.sam ``` SAM (Sequence Alignment/Map) is a file format that is used to store the results of DNA sequencing alignments. In Python, you can read a SAM file by opening the file, reading the lines one by one, and processing the data as needed. Here's an example of how you might read a SAM file in Python: ```py= def read_sam(file_path): records = [] with open(file_path, "r") as f: for line in f: if line.startswith("@"): continue fields = line.strip().split("\t") records.append(fields) return records ``` In this example, the `read_sam` function takes a file path as an argument, and returns a list of records, where each record is a list of fields. The function uses a `with` statement to open the file and read the lines one by one. If a line starts with an `"@"`, it is assumed to be a header and is ignored. If the line does not start with an `"@"`, it is assumed to be a record, and the fields are extracted by splitting the line on tabs. You can use this function to read a SAM file like this: ```py= records = read_sam("sam_example.sam") for fields in records: print(fields) ``` This will print the fields in the SAM file. You can modify the read_sam function to process the data in any way you need. For example, you might want to extract specific fields, such as the reference name, the start position, and the cigar string.
Last changed by  
 
Anton Nekrutenko
251

Read more

WF Invocation UI BoF

Log in into https://usegalaxy.org

6/27/2024
Lecture 24: Sequence alignment

In the previous lecture we have seen the principle behind dynamic programming. This approach is extremely useful for comparing biological sequences, which is coincidentally one of the main points of this course. This lecture explain how this is done. In writing this text I heavily relied on wonderful course taught by Ben Langmead at Johns Hopkins. The cover image shows pairwise alignments for human, mouse, and dog KIF3 locus from Dubchak et al. 2000. How different are two sequences? Suppose you have two sequences of the same length: A C A T G C C T A A C T G C C T A C How different are they? In other words, how many bases should be change to turn one sequence onto the other: A C A T G C C T A

4/18/2024
Galaxy Outreach Style Guide

Colors

3/22/2024
Lecture 4: Intermission + History of Sequencing

The problem The difficulty with sequencing nucleic acids is nicely summarized by Hutchinson:2007: The chemical properties of different DNA molecules were so similar that it appeared difficult to separate them. The chain length of naturally occurring DNA molecules was much greater than for proteins and made complete sequencing seems unapproachable. The 20 amino acid residues found in proteins have widely varying properties that had proven useful in the separation of peptides. The existence of only four bases in DNA therefore seemed to make sequencing a more difficult problem for DNA than for protein. No base-specific DNAases were known. Protein sequencing had depended upon proteases that cleave adjacent to certain amino acids. It is therefore not surprising that protein-sequencing was developed before DNA sequencing by Sanger and Tuppy:1951.

1/22/2024
Read more from Anton Nekrutenko
Published on HackMD