# AZURE DP203: Day 3

AZURE DP203 課程介紹與上課心得: Day 3 (Full eng. version)
# Labs Overviews:
## Module 6: Transform data with Azure Data Factory or Azure Synapse Pipelines
* Data integration with Azure Data Factory or Azure Synapse Piplines
* Code-free transformation at scale with Azure Data Factory or Azure Synapse
* Pipline

## Module 7:Integrate data from notebooks with Azure Data Factory or Azure Synapse Pipelines
* Orchestrate data movement and transformation in Azure Data Factory or Azure Synapse Piplines

## Module 8: End-to-end security with Azure Synapse Analytics
* Secure a data warehouse in Azure Synapse Analytics
* Configure and manage secretes in Azure Key Vault
* Implement compliance controls for sensitive data

## Extra Lab: Movies Analytics using Azure Data Factory
* https://github.com/djpmsft/adf-ready-demo/blob/master/ADF_Lab_Instructions.md

# Course Overviews:
* Linked services in Azure Data Factory and Azure Synapse Analytics: https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services?tabs=data-factory
* Ingest and Transform Data with Azure Synapse Analytics: https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/ingest-and-transform-data-with-azure-synapse-analytics-with-ease/ba-p/1975563
* Mapping data flows in Azure Data Factory: https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-overview
* Integration data from Azure Data Factory or Azure Synapse Piplines:
* 1. Use Storage Account
* 2. Use Azure Synapse Pipline
* 3. Create data workflow pipline
* 4. Add a Notebook Activity to the pipline
* 5. Set parameters & dependency condition
* Secure a data warehouse in Azure Synapse Analytics:
* Network security
* Identity and access management
* Managin sensitive data
* Encryption capabilities but into Azure
* Network Security: Securing your network from attcks and unauthorized access is an important part of any architecture
* Internet protection
* Firewall
* DDos Protection
* Networking security groups
### What is NSG (Network Security Group)?
* "NSG" is the one that connect to the subnet and the nic.
* On the other hand, subnet need to use nic to connect Azure Serice. The the nic is known as "Private Endpoint".
* The connection between the Virtual Network is call "Peering"
* We can use "Express Route or VPN" to connect the Virtual Network and local Network.
* The connection between Azure Service and Virtual Macine(VM) is known as "Service Connection"

#### Virtual Network
    L
####      subnet
                L
####                nic
                        L
####                        VM

### Identity and access:
* Authentication vs. Authorization:

* Azure Active Directory features:
* Single Sign-On
* Apps & Device Management
* Identity Service
* What is Intune (MAM & MDM)? : https://learn.microsoft.com/zh-tw/mem/intune/apps/app-management
* Difference between MDM and MAM: https://learn.microsoft.com/en-us/microsoft-365/business/ui/mam-and-mdm?view=o365-worldwide
* Azure Key Vault protects:
* Quickstart: Set and retrieve a secret from Azure Key Vault using the Azure portal: https://learn.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal
* More details (Manage secrets in your server apps with Azure Key Vault): https://learn.microsoft.com/en-us/training/modules/manage-secrets-with-azure-key-vault/
* Data engineers are typically concerned with accessing the data contained in Key Vault to apply to linked service.
* 1. Secretes
* 2. Keys
* 3. Certificates