---
# System prepended metadata

title: FHIR Server on AKS
tags: [FHIR, Demo, MTC 2022]

---

###### tags: `Demo`  `FHIR` `MTC 2022`
# FHIR Server on AKS
[ToC]
## Deploy FHIR Server
### Prerequisites
[Read document](https://github.com/microsoft/fhir-server/tree/main/samples/kubernetes)
[HL7 website](https://www.hl7.org/fhir)
### AKS resource creating
1. Config env variable
    ```azurecli-interactive
    REGION_NAME=southeastasia
    RESOURCE_GROUP=rg-fhirserver
    SUBNET_NAME=subnet-fhirserver
    VNET_NAME=vnet-fhirserver
    AKS_CLUSTER_NAME=aks-fhirserver
    ```
2. Deploy resorce group
    ```azurecli-interactive
    az group create \
        --name $RESOURCE_GROUP \
        --location $REGION_NAME
    ```
3. Deploy virtual network
    ```azurecli-interactive
    az network vnet create \
        --resource-group $RESOURCE_GROUP \
        --location $REGION_NAME \
        --name $VNET_NAME \
        --address-prefixes 10.0.0.0/8 \
        --subnet-name $SUBNET_NAME \
        --subnet-prefixes 10.240.0.0/16
    ```
4. Capture subnet id
    ```azurecli-interactive
    SUBNET_ID=$(az network vnet subnet show \
        --resource-group $RESOURCE_GROUP \
        --vnet-name $VNET_NAME \
        --name $SUBNET_NAME \
        --query id -o tsv)
    ```
5. Deploy AKS
    ```azurecli-interactive
        az aks create \
        --resource-group $RESOURCE_GROUP \
        --name $AKS_CLUSTER_NAME \
        --node-vm-size Standard_D2s_v3 \
        --location $REGION_NAME \
        --vm-set-type VirtualMachineScaleSets \
        --load-balancer-sku standard \
        --enable-cluster-autoscaler \
        --min-count 3 \
        --max-count 5 \
        --generate-ssh-keys \
        --kubernetes-version 1.21.7 \
        --network-plugin azure \
        --vnet-subnet-id $SUBNET_ID \
        --service-cidr 10.2.0.0/24 \
        --dns-service-ip 10.2.0.10 \
        --docker-bridge-address 172.17.0.1/16
    ```
    :exclamation: Notice the kubernetes runtime version
### ASO (Azure Service Operator)
0. Downloads credentials and configures the Kubernetes CLI to use them
   （如果沒加這行會出現ASO controller manager CrashLoopBackOff的問題！） 

    ```sh
    az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER_NAME
    ```
    

1. Install cert-manager
    ```sh
    kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.12.0/cert-manager.yaml
    ```
    ```
    kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.0.4/cert-manager.crds.yaml
    ```
2.  Addin the Helm repo for Azure Service Operator (Helm 3)

    ```sh
    helm repo add aso https://raw.githubusercontent.com/Azure/azure-service-operator/main/charts
    ```
3. Get the tenant ID and subscription ID
    ```sh
    az account show
    ```

    ```yaml
    AZURE_TENANT_ID=<your-tenant-id-goes-here>
    AZURE_SUBSCRIPTION_ID=<your-subscription-id-goes-here>
    ```
4. Create an Azure Service Principal

    :warning:change <azure-service-operator> to your own name
    ```sh
    az ad sp create-for-rbac -n "<azure-service-operator>" --role contributor \
        --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID
    ```
    

   This should give you output like the following:
    ```sh
    "appId": "xxxxxxxxxx",
    "displayName": "azure-service-operator",
    "name": "http://azure-service-operator",
    "password": "xxxxxxxxxxx",
    "tenant": "xxxxxxxxxxxxx"
    ```

   Once you have created a service principal, set the following variables to your app ID and password values:
    ```sh 
    AZURE_CLIENT_ID=<your-client-id> # This is the appID from the service principal we created.
    AZURE_CLIENT_SECRET=<your-client-secret> # This is the password from the service principal we created.
    ```

5. Install the Azure Service Operator on your cluster using Helm

    ```sh
    helm upgrade --install aso aso/azure-service-operator \
            --create-namespace \
            --namespace=azureoperator-system \
            --set azureSubscriptionID=$AZURE_SUBSCRIPTION_ID \
            --set azureTenantID=$AZURE_TENANT_ID \
            --set azureClientID=$AZURE_CLIENT_ID \
            --set azureClientSecret=$AZURE_CLIENT_SECRET
    ```
6. Check if your ASO is running
    ```
    kubectl get pods -n azureoperator-system
    ```
    You should get the result as below:
    ![](https://i.imgur.com/3KcTI2B.jpg)
    
    
    If the status of ASO is **Error** or **CrashLoopBackOff**, check the error message:
    ```
    kubectl logs <your-pod-name> manager -n azureoperator-system
    ```
    
    
#### Reference
- [ASO for K8S tutorial](https://github.com/Azure/azure-service-operator/blob/main/docs/v1/README.md#quickstart)
- [Use ASO to deploy Azure resource](https://www.youtube.com/watch?v=gJyuRb6O6KU)

### Install FHIR Server to AKS
1. Install Pod Identities (needed for export)
    ```sh
    helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts
    helm install aad-pod-identity aad-pod-identity/aad-pod-identity --set createNamespace=true \
    --set nmi.allowNetworkPluginKubenet=true
    ```
2. Edit line 91, 96, 101 in [fhir-server/samples/kubernetes/helm/fhir-server/templates/deployment.yaml](https://github.com/microsoft/fhir-server/blob/main/samples/kubernetes/helm/fhir-server/templates/deployment.yaml)
    ```yaml
    azuresqlserver-{{ include "fhir-server.fullname" . }}
    ```
    ![](https://i.imgur.com/z8yGys3.png)

3. Deploy FHIR service that have a public IP address

    :warning:change <your-fhir-release> to your own name
    ```bash
    helm install <your-fhir-release> helm/fhir-server/ \
      --set database.resourceGroup=<your-resource-group> \
      --set database.location="southeastasia" \
      --set service.type=LoadBalancer
    ```

4. Once resource the deployment process finished, you can see the resource on Azure Portal
    ![](https://i.imgur.com/tzlv52H.png)
    >  可跳至 section [Use FHIR Server](#Use-FHIR-Server)
5. Get the public IP address
    ```bash
    kubectl get svc my-fhir-release-fhir-server
    ```
6. Testing
    ```bash
    curl -s http://<my-fhir-release-fhir-server-public-IP>/Patient | jq .
    ```
    ![](https://i.imgur.com/gMrFRPo.png)

    
### Useful Kubectl Command
- Check ASO deployment status
    ```bash
        kubectl get azuresqlserver
    ```
    ![](https://i.imgur.com/5FcOFzu.png)
- Get sceret data and decode
    ```bash
    kubectl get secret azuresqlserver-my-fhir-release-fhir-server -o jsonpath='{.data}'

    echo 'bXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVy' | base64 --decode
    echo 'bXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVyLmRhdGFiYXNlLndpbmRvd3MubmV0' | base64 --decode
    echo 'V1JDN3E3TWFAbXktZmhpci1yZWxlYXNlLWZoaXItc2VydmVy' | base64 --decode
    echo 'RE5aZ2U1KkBmKjBqUjY3dQ==' | base64 --decode
    echo 'YW1aZUJ1UFo=' | base64 --decode
    ```

- Create Secret
    ```bash
    kubectl create secret generic my-fhir-release-fhir-server \
    --from-literal=azureSqlServerName=my-fhir-release-fhir-server \
    --from-literal=fullyQualifiedServerName=my-fhir-release-fhir-server.database.windows.net \
    --from-literal=fullyQualifiedUsername=amZeBuPZ@my-fhir-release-fhir-server \
    --from-literal=username=amZeBuPZ \
    --from-literal=password=Zd6%wXqwyKH5jj$k
    ```
- Deploy CosomosDB (Not validate yet)
    ```bash
    helm install my-fhir-release helm/fhir-server/ \
      --set database.dataStore="CosmosDb" \
      --set database.resourceGroup="rg-aso" \
      --set database.location="southeastasia" \
      --set service.type=LoadBalancer
    ```


- 檢查default namespace底下的pods
    ``` sh
    kubectl get pods -n default
    ```
    
- 檢查aso namespace底下的pods
    ``` sh
    kubectl get pods -n azureoperator-system
    ```
    
- 卸除已存在的aad-pod-identity
    ```sh
    helm uninstall aad-pod-identity
    ```
    

- 卸除已存在的fhir release
    ```sh
    helm uninstall <your-fhir-release>
    ```    
    
## Use FHIR Server
    
    - Demo URL:  http://20.212.129.87/Patient
    
### 新增資料-Using POST to upload patient data via postman
Reference: See section **Create or update your FHIR resource** of [Use postman](https://docs.microsoft.com/en-us/azure/healthcare-apis/use-postman)

1. Login to [Postman](https://www.postman.com/), select or create a new workspace 
2. Get your EXTERNAL IP by using the command 
    ```bash
    kubectl get svc <your-fhir-release>
    ```
3. Create a new request, change the method to **POST**, and enter the value
            ``` 
            http://<your-fhir-EXTERNAL_IP>/Patient
            ```
4. Select **Authorization**, change the type to **Bearer Token**
    enter  ```{{bearerToken}}``` for the **Token**
     ![](https://i.imgur.com/rF5IxZH.png)
5. Go to **Body**, select **raw** and change the body text format to **JSON**
    ![](https://i.imgur.com/GYfMjKY.png)
6. Paste the value in the box, and send the request
```json
{
    "resourceType": "Patient",
    "active": true,
    "name": [
        {
            "use": "official",
            "family": "Kirk",
            "given": [
                "James",
                "Tiberious"
            ]
        },
        {
            "use": "usual",
            "given": [
                "Jim"
            ]
        }
    ],
    "gender": "male",
    "birthDate": "1960-12-25"
}
```
### 查看更新的資料-Check the updated data
1. Create a new request, change the method to **GET**, and enter the value
``` sh
        http://<your-fhir-EXTERNAL-IP>/Patient
```
2.  Select **Authorization**, change the type to **Bearer Token**
    enter  ```{{bearerToken}}``` for the **Token**
    ![](https://i.imgur.com/Kt4ChsW.png)
3. Send the request, and you should see the updated value below as the figure shows
    ![](https://i.imgur.com/nH8G3yU.png)

### 查找特定一筆資料
可透過指定ID做指定資料，例如:
```     
    http://<your-fhir-EXTERNAL-IP>/Patient/<PATIENT-ID>
```

### 刪除資料-TODO

### FHIR patient templete
```json
    {
        "resourceType" : "Patient",
      // from Resource: id, meta, implicitRules, and language
      // from DomainResource: text, contained, extension, and modifierExtension
      "identifier" : [{ Identifier }], // An identifier for this patient
      "active" : <boolean>, // Whether this patient's record is in active use
      "name" : [{ HumanName }], // A name associated with the patient
      "telecom" : [{ ContactPoint }], // A contact detail for the individual
      "gender" : "<code>", // male | female | other | unknown
      "birthDate" : "<date>", // The date of birth for the individual
      // deceased[x]: Indicates if the individual is deceased or not. One of these 2:
      "deceasedBoolean" : <boolean>,
      "deceasedDateTime" : "<dateTime>",
      "address" : [{ Address }], // An address for the individual
      "maritalStatus" : { CodeableConcept }, // Marital (civil) status of a patient
      // multipleBirth[x]: Whether patient is part of a multiple birth. One of these 2:
      "multipleBirthBoolean" : <boolean>,
      "multipleBirthInteger" : <integer>,
      "photo" : [{ Attachment }], // Image of the patient
      "contact" : [{ // A contact party (e.g. guardian, partner, friend) for the patient
        "relationship" : [{ CodeableConcept }], // The kind of relationship
        "name" : { HumanName }, // A name associated with the contact person
        "telecom" : [{ ContactPoint }], // A contact detail for the person
        "address" : { Address }, // Address for the contact person
        "gender" : "<code>", // male | female | other | unknown
        "organization" : { Reference(Organization) }, // C? Organization that is associated with the contact
        "period" : { Period } // The period during which this contact person or organization is valid to be contacted relating to this patient
      }],
      "communication" : [{ // A language which may be used to communicate with the patient about his or her health
        "language" : { CodeableConcept }, // R!  The language which can be used to communicate with the patient about his or her health
        "preferred" : <boolean> // Language preference indicator
      }],
      "generalPractitioner" : [{ Reference(Organization|Practitioner|
       PractitionerRole) }], // Patient's nominated primary care provider
      "managingOrganization" : { Reference(Organization) }, // Organization that is the custodian of the patient record
      "link" : [{ // Link to another patient resource that concerns the same actual person
        "other" : { Reference(Patient|RelatedPerson) }, // R!  The other patient or related person resource that the link refers to
        "type" : "<code>" // R!  replaced-by | replaces | refer | seealso
      }]
        }
```
## Fhir SQL Database

- prerequire(擇一使用即可 範例使用 SSMS):
    [SQL Server Management Studio (SSMS)](https://docs.microsoft.com/zh-tw/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver15)
    [Azure Data Studio](https://docs.microsoft.com/zh-tw/sql/azure-data-studio/download-azure-data-studio?view=sql-server-ver15)
    - SSMS:
    由左上角[檔案]->[連線物件總管]
    ![](https://i.imgur.com/4cRzOvI.png)
    
    將伺服器名稱、登入帳號密碼輸入

    ```
    ServerName:    harris0210-fhir-release-fhir-server.database.windows.net
    Uid:    gBckJmki
    PWD:    MTCP@ssw0rd
    ```
    
    ![](https://i.imgur.com/tCQ18ds.png)
    
    為了Demo需要，有預先將SQL Server的FireWall IP 全開
    可進到SQL Server做Firewall設定
    ![](https://i.imgur.com/fAeRkdh.png)
    

    成功連線畫面
    ![](https://i.imgur.com/j393dXg.png)
    
    可對資料表(Table)右鍵選擇[選取前1000個資料列] 做快速查詢
    ![](https://i.imgur.com/03lDuuq.png)



    
## Fhir Server Dashboard
[fhir-server-dashboard](https://github.com/smart-on-fhir/fhir-server-dashboard)
    
### UPDATE
- Hueir說要完成後面掛ingress的部分才能存取到server IP。掛ingress上去之後可以掛domain上去


- Installation
Run these commands in your terminal:
```bash    
cd my/directory/somewhere
git clone https://github.com/smart-on-fhir/fhir-server-dashboard.git
cd fhir-server-dashboard
npm i
```
- Usage
    Specify the target server in server/config.js:
    find the server name on azure portal sql database
    ![](https://i.imgur.com/dj3rooJ.png)

    ```js
    SERVER: 'my-fhir-server-url.com'
    ```
    To aggregate the data on a FHIR server, run this command in your terminal from the project directory:
    ```bash
    npm run aggregate
    ```
After these two steps, simply open client/index.html in the browser of your choice.
    

- error
    after run the command ```npm run aggregate```, the internal server error will show up
       

## SQL server connect
    
- Tools
    SQL Server Management Studio (SSMS) - [Azure Data Studio](https://docs.microsoft.com/en-us/sql/azure-data-studio/download-azure-data-studio?view=sql-server-ver15)
    
## Web app tutorial
[Document](https://docs.microsoft.com/en-us/azure/healthcare-apis/azure-api-for-fhir/tutorial-web-app-fhir-server)
    
- https://fhirapifordemo.azurehealthcareapis.com/metadata
    
- Call FHIR API
    - CORS error
![](https://i.imgur.com/lDREY64.png)
        - [reason](https://stackoverflow.com/questions/20035101/why-does-my-javascript-code-receive-a-no-access-control-allow-origin-header-i)
    ![](https://i.imgur.com/mV0U24d.png)
        - CORS defines the restrictions relative to the origin (URL domain) of the page which initiates the request. But in Postman the requests doesn't originate from a page with an URL so CORS does not apply.
    
## Configuring Nginx
1. Create namespace
    ```.sh
    kubectl create namespace ingress-controller
    ```
2. Install nignx ingress controll
    ```.sh
    helm upgrade --install nginx-ingress stable/nginx-ingress \
      --namespace ingress-controller \
      --set controller.replicaCount=2 \
      --set controller.nodeSelector."kubernetes\.io/os"=linux \
      --set defaultBackend.nodeSelector."kubernetes\.io/os"=linux
    ```
3. Get ingress external IP
    ```
    kubectl get svc -n ingress-controller
    ```
4. Note the EXTERNAL-IP of nginx-ingress-controller   
    ![](https://i.imgur.com/dDIy1hZ.png)

5.  create an ingress-values.yaml file
    ```.yaml
    ingress:
        enabled: true
        annotations:
          kubernetes.io/ingress.class: nginx
          nginx.ingress.kubernetes.io/enable-cors: "true"
          nginx.ingress.kubernetes.io/cors-allow-origin: "*"
          nginx.ingress.kubernetes.io/cors-allow-methods: "*"
          nginx.ingress.kubernetes.io/cors-allow-headers: "*"
          nginx.ingress.kubernetes.io/cors-max-age: "1440"
        hosts:
          - host: fhirfront.<ingress-external-ip>.nip.io
            paths:
            - /
    
    ```
6. Update FHIR-SERVER setting
    ```
    helm upgrade <your-fhir-release> helm/fhir-server/ \
      -f ingress-values.yaml \
      --set database.location=$REGION_NAME \
      --set database.resourceGroup=$RESOURCE_GROUP
    ```
![](https://i.imgur.com/2RayjBy.png)

    
## FHIR Web App
- [Document](https://docs.microsoft.com/en-us/azure/healthcare-apis/azure-api-for-fhir/tutorial-web-app-public-app-reg) 
- 安裝[CORS extension](https://chrome.google.com/webstore/detail/moesif-origin-cors-change/digfbfaphojjndkpccljibejjbppifbc) 
- 新增`index.html`檔案，login with msmtctwaiwan github account，貼上[這裡](https://github.com/msmtctaiwan/FHIR-Demo-Web)的code
- Open in your browser
                                                                    
## Create Azure SQL Managed Instance
![](https://i.imgur.com/2Ag6XPv.png)

## [FHIR Demo Web Github code](https://github.com/msmtctaiwan/FHIR-Demo-Web)

## [原始文件Web](https://github.com/microsoft/fhir-server/tree/main/samples/apps/SmartLauncher/wwwroot)

