###### tags: `Demo` `FHIR` `MTC 2022`
# FHIR Server on AKS
[ToC]
## Deploy FHIR Server
### Prerequisites
[Read document](https://github.com/microsoft/fhir-server/tree/main/samples/kubernetes)
[HL7 website](https://www.hl7.org/fhir)
### AKS resource creating
1. Config env variable
```azurecli-interactive
REGION_NAME=southeastasia
RESOURCE_GROUP=rg-fhirserver
SUBNET_NAME=subnet-fhirserver
VNET_NAME=vnet-fhirserver
AKS_CLUSTER_NAME=aks-fhirserver
```
2. Deploy resorce group
```azurecli-interactive
az group create \
--name $RESOURCE_GROUP \
--location $REGION_NAME
```
3. Deploy virtual network
```azurecli-interactive
az network vnet create \
--resource-group $RESOURCE_GROUP \
--location $REGION_NAME \
--name $VNET_NAME \
--address-prefixes 10.0.0.0/8 \
--subnet-name $SUBNET_NAME \
--subnet-prefixes 10.240.0.0/16
```
4. Capture subnet id
```azurecli-interactive
SUBNET_ID=$(az network vnet subnet show \
--resource-group $RESOURCE_GROUP \
--vnet-name $VNET_NAME \
--name $SUBNET_NAME \
--query id -o tsv)
```
5. Deploy AKS
```azurecli-interactive
az aks create \
--resource-group $RESOURCE_GROUP \
--name $AKS_CLUSTER_NAME \
--node-vm-size Standard_D2s_v3 \
--location $REGION_NAME \
--vm-set-type VirtualMachineScaleSets \
--load-balancer-sku standard \
--enable-cluster-autoscaler \
--min-count 3 \
--max-count 5 \
--generate-ssh-keys \
--kubernetes-version 1.21.7 \
--network-plugin azure \
--vnet-subnet-id $SUBNET_ID \
--service-cidr 10.2.0.0/24 \
--dns-service-ip 10.2.0.10 \
--docker-bridge-address 172.17.0.1/16
```
:exclamation: Notice the kubernetes runtime version
### ASO (Azure Service Operator)
0. Downloads credentials and configures the Kubernetes CLI to use them
（如果沒加這行會出現ASO controller manager CrashLoopBackOff的問題！）
```sh
az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER_NAME
```
1. Install cert-manager
```sh
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.12.0/cert-manager.yaml
```
```
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.0.4/cert-manager.crds.yaml
```
2. Addin the Helm repo for Azure Service Operator (Helm 3)
```sh
helm repo add aso https://raw.githubusercontent.com/Azure/azure-service-operator/main/charts
```
3. Get the tenant ID and subscription ID
```sh
az account show
```
```yaml
AZURE_TENANT_ID=<your-tenant-id-goes-here>
AZURE_SUBSCRIPTION_ID=<your-subscription-id-goes-here>
```
4. Create an Azure Service Principal
:warning:change <azure-service-operator> to your own name
```sh
az ad sp create-for-rbac -n "<azure-service-operator>" --role contributor \
--scopes /subscriptions/$AZURE_SUBSCRIPTION_ID
```
This should give you output like the following:
```sh
"appId": "xxxxxxxxxx",
"displayName": "azure-service-operator",
"name": "http://azure-service-operator",
"password": "xxxxxxxxxxx",
"tenant": "xxxxxxxxxxxxx"
```
Once you have created a service principal, set the following variables to your app ID and password values:
```sh
AZURE_CLIENT_ID=<your-client-id> # This is the appID from the service principal we created.
AZURE_CLIENT_SECRET=<your-client-secret> # This is the password from the service principal we created.
```
5. Install the Azure Service Operator on your cluster using Helm
```sh
helm upgrade --install aso aso/azure-service-operator \
--create-namespace \
--namespace=azureoperator-system \
--set azureSubscriptionID=$AZURE_SUBSCRIPTION_ID \
--set azureTenantID=$AZURE_TENANT_ID \
--set azureClientID=$AZURE_CLIENT_ID \
--set azureClientSecret=$AZURE_CLIENT_SECRET
```
6. Check if your ASO is running
```
kubectl get pods -n azureoperator-system
```
You should get the result as below:
If the status of ASO is **Error** or **CrashLoopBackOff**, check the error message:
```
kubectl logs <your-pod-name> manager -n azureoperator-system
```
#### Reference
- [ASO for K8S tutorial](https://github.com/Azure/azure-service-operator/blob/main/docs/v1/README.md#quickstart)
- [Use ASO to deploy Azure resource](https://www.youtube.com/watch?v=gJyuRb6O6KU)
### Install FHIR Server to AKS
1. Install Pod Identities (needed for export)
```sh
helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts
helm install aad-pod-identity aad-pod-identity/aad-pod-identity --set createNamespace=true \
--set nmi.allowNetworkPluginKubenet=true
```
2. Edit line 91, 96, 101 in [fhir-server/samples/kubernetes/helm/fhir-server/templates/deployment.yaml](https://github.com/microsoft/fhir-server/blob/main/samples/kubernetes/helm/fhir-server/templates/deployment.yaml)
```yaml
azuresqlserver-{{ include "fhir-server.fullname" . }}
```
3. Deploy FHIR service that have a public IP address
:warning:change <your-fhir-release> to your own name
```bash
helm install <your-fhir-release> helm/fhir-server/ \
--set database.resourceGroup=<your-resource-group> \
--set database.location="southeastasia" \
--set service.type=LoadBalancer
```
4. Once resource the deployment process finished, you can see the resource on Azure Portal
> 可跳至 section [Use FHIR Server](#Use-FHIR-Server)
5. Get the public IP address
```bash
kubectl get svc my-fhir-release-fhir-server
```
6. Testing
```bash
curl -s http://<my-fhir-release-fhir-server-public-IP>/Patient | jq .
```
### Useful Kubectl Command
- Check ASO deployment status
```bash
kubectl get azuresqlserver
```
- Get sceret data and decode
```bash
kubectl get secret azuresqlserver-my-fhir-release-fhir-server -o jsonpath='{.data}'
echo 'bXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVy' | base64 --decode
echo 'bXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVyLmRhdGFiYXNlLndpbmRvd3MubmV0' | base64 --decode
echo 'V1JDN3E3TWFAbXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVy' | base64 --decode
echo 'RE5aZ2U1KkBmKjBqUjY3dQ==' | base64 --decode
echo 'YW1aZUJ1UFo=' | base64 --decode
```
- Create Secret
```bash
kubectl create secret generic my-fhir-release-fhir-server \
--from-literal=azureSqlServerName=my-fhir-release-fhir-server \
--from-literal=fullyQualifiedServerName=my-fhir-release-fhir-server.database.windows.net \
--from-literal=fullyQualifiedUsername=amZeBuPZ@my-fhir-release-fhir-server \
--from-literal=username=amZeBuPZ \
--from-literal=password=Zd6%wXqwyKH5jj$k
```
- Deploy CosomosDB (Not validate yet)
```bash
helm install my-fhir-release helm/fhir-server/ \
--set database.dataStore="CosmosDb" \
--set database.resourceGroup="rg-aso" \
--set database.location="southeastasia" \
--set service.type=LoadBalancer
```
- 檢查default namespace底下的pods
``` sh
kubectl get pods -n default
```
- 檢查aso namespace底下的pods
``` sh
kubectl get pods -n azureoperator-system
```
- 卸除已存在的aad-pod-identity
```sh
helm uninstall aad-pod-identity
```
- 卸除已存在的fhir release
```sh
helm uninstall <your-fhir-release>
```
## Use FHIR Server
- Demo URL: http://20.212.129.87/Patient
### 新增資料-Using POST to upload patient data via postman
Reference: See section **Create or update your FHIR resource** of [Use postman](https://docs.microsoft.com/en-us/azure/healthcare-apis/use-postman)
1. Login to [Postman](https://www.postman.com/), select or create a new workspace
2. Get your EXTERNAL IP by using the command
```bash
kubectl get svc <your-fhir-release>
```
3. Create a new request, change the method to **POST**, and enter the value
```
http://<your-fhir-EXTERNAL_IP>/Patient
```
4. Select **Authorization**, change the type to **Bearer Token**
enter ```{{bearerToken}}``` for the **Token**
5. Go to **Body**, select **raw** and change the body text format to **JSON**
6. Paste the value in the box, and send the request
```json
{
"resourceType": "Patient",
"active": true,
"name": [
{
"use": "official",
"family": "Kirk",
"given": [
"James",
"Tiberious"
]
},
{
"use": "usual",
"given": [
"Jim"
]
}
],
"gender": "male",
"birthDate": "1960-12-25"
}
```
### 查看更新的資料-Check the updated data
1. Create a new request, change the method to **GET**, and enter the value
``` sh
http://<your-fhir-EXTERNAL-IP>/Patient
```
2. Select **Authorization**, change the type to **Bearer Token**
enter ```{{bearerToken}}``` for the **Token**
3. Send the request, and you should see the updated value below as the figure shows
### 查找特定一筆資料
可透過指定ID做指定資料，例如:
```
http://<your-fhir-EXTERNAL-IP>/Patient/<PATIENT-ID>
```
### 刪除資料-TODO
### FHIR patient templete
```json
{
"resourceType" : "Patient",
// from Resource: id, meta, implicitRules, and language
// from DomainResource: text, contained, extension, and modifierExtension
"identifier" : [{ Identifier }], // An identifier for this patient
"active" : <boolean>, // Whether this patient's record is in active use
"name" : [{ HumanName }], // A name associated with the patient
"telecom" : [{ ContactPoint }], // A contact detail for the individual
"gender" : "<code>", // male | female | other | unknown
"birthDate" : "<date>", // The date of birth for the individual
// deceased[x]: Indicates if the individual is deceased or not. One of these 2:
"deceasedBoolean" : <boolean>,
"deceasedDateTime" : "<dateTime>",
"address" : [{ Address }], // An address for the individual
"maritalStatus" : { CodeableConcept }, // Marital (civil) status of a patient
// multipleBirth[x]: Whether patient is part of a multiple birth. One of these 2:
"multipleBirthBoolean" : <boolean>,
"multipleBirthInteger" : <integer>,
"photo" : [{ Attachment }], // Image of the patient
"contact" : [{ // A contact party (e.g. guardian, partner, friend) for the patient
"relationship" : [{ CodeableConcept }], // The kind of relationship
"name" : { HumanName }, // A name associated with the contact person
"telecom" : [{ ContactPoint }], // A contact detail for the person
"address" : { Address }, // Address for the contact person
"gender" : "<code>", // male | female | other | unknown
"organization" : { Reference(Organization) }, // C? Organization that is associated with the contact
"period" : { Period } // The period during which this contact person or organization is valid to be contacted relating to this patient
}],
"communication" : [{ // A language which may be used to communicate with the patient about his or her health
"language" : { CodeableConcept }, // R! The language which can be used to communicate with the patient about his or her health
"preferred" : <boolean> // Language preference indicator
}],
"generalPractitioner" : [{ Reference(Organization|Practitioner|
PractitionerRole) }], // Patient's nominated primary care provider
"managingOrganization" : { Reference(Organization) }, // Organization that is the custodian of the patient record
"link" : [{ // Link to another patient resource that concerns the same actual person
"other" : { Reference(Patient|RelatedPerson) }, // R! The other patient or related person resource that the link refers to
"type" : "<code>" // R! replaced-by | replaces | refer | seealso
}]
}
```
## Fhir SQL Database
- prerequire(擇一使用即可 範例使用 SSMS):
[SQL Server Management Studio (SSMS)](https://docs.microsoft.com/zh-tw/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver15)
[Azure Data Studio](https://docs.microsoft.com/zh-tw/sql/azure-data-studio/download-azure-data-studio?view=sql-server-ver15)
- SSMS:
由左上角[檔案]->[連線物件總管]
將伺服器名稱、登入帳號密碼輸入
```
ServerName: harris0210-fhir-release-fhir-server.database.windows.net
Uid: gBckJmki
PWD: MTCP@ssw0rd
```
為了Demo需要，有預先將SQL Server的FireWall IP 全開
可進到SQL Server做Firewall設定
成功連線畫面
可對資料表(Table)右鍵選擇[選取前1000個資料列] 做快速查詢
## Fhir Server Dashboard
[fhir-server-dashboard](https://github.com/smart-on-fhir/fhir-server-dashboard)
### UPDATE
- Hueir說要完成後面掛ingress的部分才能存取到server IP。掛ingress上去之後可以掛domain上去
- Installation
Run these commands in your terminal:
```bash
cd my/directory/somewhere
git clone https://github.com/smart-on-fhir/fhir-server-dashboard.git
cd fhir-server-dashboard
npm i
```
- Usage
Specify the target server in server/config.js:
find the server name on azure portal sql database
```js
SERVER: 'my-fhir-server-url.com'
```
To aggregate the data on a FHIR server, run this command in your terminal from the project directory:
```bash
npm run aggregate
```
After these two steps, simply open client/index.html in the browser of your choice.
- error
after run the command ```npm run aggregate```, the internal server error will show up
## SQL server connect
- Tools
SQL Server Management Studio (SSMS) - [Azure Data Studio](https://docs.microsoft.com/en-us/sql/azure-data-studio/download-azure-data-studio?view=sql-server-ver15)
## Web app tutorial
[Document](https://docs.microsoft.com/en-us/azure/healthcare-apis/azure-api-for-fhir/tutorial-web-app-fhir-server)
- https://fhirapifordemo.azurehealthcareapis.com/metadata
- Call FHIR API
- CORS error
- [reason](https://stackoverflow.com/questions/20035101/why-does-my-javascript-code-receive-a-no-access-control-allow-origin-header-i)
- CORS defines the restrictions relative to the origin (URL domain) of the page which initiates the request. But in Postman the requests doesn't originate from a page with an URL so CORS does not apply.
## Configuring Nginx
1. Create namespace
```.sh
kubectl create namespace ingress-controller
```
2. Install nignx ingress controll
```.sh
helm upgrade --install nginx-ingress stable/nginx-ingress \
--namespace ingress-controller \
--set controller.replicaCount=2 \
--set controller.nodeSelector."kubernetes\.io/os"=linux \
--set defaultBackend.nodeSelector."kubernetes\.io/os"=linux
```
3. Get ingress external IP
```
kubectl get svc -n ingress-controller
```
4. Note the EXTERNAL-IP of nginx-ingress-controller
5. create an ingress-values.yaml file
```.yaml
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/cors-allow-methods: "*"
nginx.ingress.kubernetes.io/cors-allow-headers: "*"
nginx.ingress.kubernetes.io/cors-max-age: "1440"
hosts:
- host: fhirfront.<ingress-external-ip>.nip.io
paths:
- /
```
6. Update FHIR-SERVER setting
```
helm upgrade <your-fhir-release> helm/fhir-server/ \
-f ingress-values.yaml \
--set database.location=$REGION_NAME \
--set database.resourceGroup=$RESOURCE_GROUP
```
## FHIR Web App
- [Document](https://docs.microsoft.com/en-us/azure/healthcare-apis/azure-api-for-fhir/tutorial-web-app-public-app-reg)
- 安裝[CORS extension](https://chrome.google.com/webstore/detail/moesif-origin-cors-change/digfbfaphojjndkpccljibejjbppifbc)
- 新增`index.html`檔案，login with msmtctwaiwan github account，貼上[這裡](https://github.com/msmtctaiwan/FHIR-Demo-Web)的code
- Open in your browser
## Create Azure SQL Managed Instance
## [FHIR Demo Web Github code](https://github.com/msmtctaiwan/FHIR-Demo-Web)
## [原始文件Web](https://github.com/microsoft/fhir-server/tree/main/samples/apps/SmartLauncher/wwwroot)
Sign in with Wallet
Connect another wallet