---
tags: ResBaz2021
---
# Prefect (an Airflow alternative) for data pipelines in Python
## Thursday, May 20th, 2021 1\:00-3\:00

[Back to Resbaz HackMD Directory](https://hackmd.io/@ResBaz21/directory)

Conference Room 1: https://arizona.zoom.us/j/82636592469

Prerequisites: Intro to Python I and II, or equivalent knowledge of Python

We will cover a brief overview of the data pipeline options available in Python from Airflow to Luigi, then create together a pipeline in Prefect.

Prefect is an improvement to Airflow workflows that puts data first in workflow automation. We use Prefect at Roche Tissue Diagnostics to automate the transfer and analysis of images. Prefect provides us a full orchestration, monitoring, and pipeline system for our data

*** __*Note: I am in no way affiliated with Prefect, I just like and use their system and thought others might find it useful*__ ***

Franklin Ventura has been working at Roche Tissue Diagnostics (Ventana Medical Systems) since 2011. Over the years he has developed software for various research needs using Python for many applications from controlling and automating hardware systems and microscopes to data processing, orchestration, and visualization. His latest projects include cobot development and digital pathology application full stack development. Franklin earned his BS in Biosystems Engineering from University of Arizona and M.Eng. in Software Engineering from Arizona State University.

## Getting Started
- Python 3.6+ installed
- Have a virtual environment with prefect installed see: https://docs.prefect.io/core/getting_started/installation.html
- pip install prefect
- (OPTIONAL) have docker installed
- (OPTIONAL) you can make an account at https://cloud.prefect.io/
- git clone https://github.com/VenturaFranklin/resbazTucson2021-Prefect.git
- Follow steps in the repository
---

## Introductions
Name, Affiliation, Title, Email, Social Media
- Your Name, University of Arizona, Your title, [e-mail], your social media
- Franklin Ventura, Roche Tissue Diagnostics (Ventana Medical), Senior Software Engineer, [e-mail], venturafranklin

## Questions and Answers
In this section, you can post your questions and feel free to answer if you have it. Questions will be answered during or after the workshop.

1. Ask your question.
- Here is an answer
2. Have you used a data pipeline system before? (eg. Airflow, luigi, prefect, etc)
- Here is an answer

---

:::info
**Session Feedback :mega:**

Use the link below to provide your feedback on the session:
[**Session Feedback Form**](https://forms.gle/TrnJpr9qRBEKdnVVA)
:::