# Data Warehousing on AWS

Welcome to the Data Warehousing on AWS class. This will be used as supporting material above and beyond Redshift that you can take with you.

### Kinesis

Title | Link
--- | ---
Kinesis Best Practices | https://www.youtube.com/watch?v=jKPlGznbfZ0
Enhanced Fanout | https://aws.amazon.com/blogs/aws/kds-enhanced-fanout/
Streaming ETL|https://aws.amazon.com/de/blogs/big-data/unified-serverless-streaming-etl-architecture-with-amazon-kinesis-data-analytics/
Storm Integration|https://github.com/amazon-archives/kinesis-storm-spout
Flink and Kinesis|https://aws.amazon.com/blogs/big-data/streaming-etl-with-apache-flink-and-amazon-kinesis-data-analytics/

### Lambda

Title | Link
--- | ---
Lambda Destinations|https://aws.amazon.com/blogs/compute/introducing-aws-lambda-destinations/
2|https://www.refinery.io/post/how-to-chain-serverless-functions-call-invoke-a-lambda-from-another-lambda
3|https://www.youtube.com/watch?v=Jkx6kVbDpL4
4|https://www.thoughtworks.com/insights/blog/mitigating-serverless-lock-fears

### Step Functions

Title | Link
--- | ---
1|https://aws.amazon.com/de/blogs/aws/new-using-step-functions-to-orchestrate-amazon-emr-workloads/
2|https://aws.amazon.com/about-aws/whats-new/2020/09/aws-step-functions-adds-support-for-aws-x-ray/

### MSK Managed Kafka

Title | Link
--- | ---
1|https://www.youtube.com/watch?v=HtU9pb18g5Q

### EMR

Title | Link
--- | ---
Deep Dive and Best Practices|https://www.youtube.com/watch?v=dU40df0Suoo
Whats new 2020|https://pages.awscloud.com/Deep-Dive-into-Whats-New-in-Amazon-EMR_2020_0230-ABD_OD.html
EMR 6.0.0|https://www.youtube.com/watch?v=M_EOXbJhD3g
2|https://aws.amazon.com/de/blogs/big-data/build-a-self-service-environment-for-each-line-of-business-using-amazon-emr-and-aws-service-catalog/
3|https://aws.amazon.com/de/blogs/big-data/apply-record-level-changes-from-relational-databases-to-amazon-s3-data-lake-using-apache-hudi-on-amazon-emr-and-aws-database-migration-service/
Flink|https://aws.amazon.com/de/blogs/big-data/use-apache-flink-on-amazon-emr/
File Formats|https://www.youtube.com/watch?v=aIcxFIyL6xo
Spark Optimization|https://www.youtube.com/watch?v=daXEp4HmS-E
Spark on EMR|https://www.youtube.com/watch?v=aIwJlfEAlHQ
EMR vs Glue|https://aws.amazon.com/de/blogs/big-data/how-drop-used-the-amazon-emr-runtime-for-apache-spark-to-halve-costs-and-get-results-5-4-times-faster/

### Hive

Title | Link
--- | ---
ACID Transactions|https://aws.amazon.com/de/blogs/big-data/amazon-emr-supports-apache-hive-acid-transactions/

### Resiliency

Title | Link
--- | ---
1|https://aws.amazon.com/de/blogs/big-data/optimizing-amazon-emr-for-resilience-and-cost-with-capacity-optimized-spot-instances/
2|https://www.youtube.com/watch?v=Fup5vHEvU50
3|https://github.com/bbc/chaos-lambda

### DynamoDB

Title | Link
--- | ---
1|https://www.youtube.com/watch?v=HaEPXoXVf2k
2|https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf

### Glue

Title | Link
--- | ---
Data Brew|https://aws.amazon.com/glue/features/databrew/
Data Lakes with Glue|https://www.youtube.com/watch?v=JsNR8uBVSiA
1|https://www.youtube.com/watch?v=S_xeHvP7uMo
2|https://aws.amazon.com/de/blogs/big-data/aws-serverless-data-analytics-pipeline-reference-architecture/
3|https://aws.amazon.com/de/blogs/big-data/crafting-serverless-streaming-etl-jobs-with-aws-glue/

### Athena

Title | Link
--- | ---
1|https://www.youtube.com/watch?v=tzoXRRCVmIQ
2|https://www.youtube.com/watch?v=JIviltfpul0

### Spark

Title | Link
--- | ---
Spark Jobs on EKS|https://www.youtube.com/watch?v=Om8RRGbZ6zA
Spark on EKS Best Practices | https://www.youtube.com/watch?v=3EbTr79wLkU

### BigData Architecture

Title | Link
--- | ---
Spark on EKS |https://www.youtube.com/watch?v=lHM96P5kP2k
Use case 1|https://www.youtube.com/watch?v=XpFNznmRoQ0
Use case 2|https://www.youtube.com/watch?v=wbh51O3QrE4
Data Lake|https://www.youtube.com/watch?v=7i1tj59pvYw
Hearst Corp|https://www.youtube.com/watch?v=6cwbbqi36k8

### Nitro System

Title | Link
--- | ---
AWS Hypervisor Security|https://www.youtube.com/watch?v=0qcUOKupt7Y

### Re:Invent 2020

Title | Link
--- | ---
Re:Invent | https://reinvent.awsevents.com