FHIR Radiology-Structured Report

--- title: FHIR Radiology-Structured Report tags: fhir-project-ig --- <font color="red">FHIR</font> Radiology-Structured Report (SR) === This project plans to improve radiologists’ performance in practice by designing a FHIR structured reporting template for radiology report. The structured report can be used to describe image annotation, image finding, and diagnostic report. This project proposes a FHIR solution for recording annotations, findings, and diagnostic report. According to the specifications of FHIR standard resources (Observation and DiagnosticReport), the annotations, findings, and diagnostic report would be XML or JSON formatted and could be stored in FHIR server. The FHIR-based annotations could be referenced by FHIR-based image findings and diagnostic reports. Using HL7 FHIR to integrate structured reporting reports will bring much more advantage for radiologist in performing their tasks. It is convenient for sharing the reports among radiologists and clinician, and also it can be helpful integrating with AI systems in the future. ## Table of Contents [TOC] ## 1. Background  ### 1.1 What is Structured Report (SR)? ![](https://i.imgur.com/ndgdsub.png) *Figure 1. The example of plain text report and structured report* Currently there are 2 methods for radiologist to write radiology report: 1. Provides a word processing application（e.g. Ms.word) for radiologists to input the report content, which will be generated into a plain text report (figure 1 left side) 2. Provides a point-and-click interface for radiologists to select image finding and its characteristic, which will be generated into a structured report (figure 1 right side) The well-designed structured report form can save the time for radiologists to write a report. It also can generate a structured document which is useful for further analysis and interpretation. However, it is necessary to design input interface for each image examination type and image finding. Besides, the development itself is time-consuming. If using plain text, the development is easy, since the input interface can be used to input various type of report contents. But, it would be difficult for the integration in the future and also for further analysis ### 1.2 Why using standard Structure Report (SR)? Image structured report is the fundamental for precision healthcare, smart healthcare, and internationalized healthcare (automatic translation of report result). Both radiologists' and pathologists' data are important fundamental for clinical diagnosis. In radiology field, radiologists diagnose whether the result is benign or malignant based on the image finding and their own experience. In this case, image finding structured report plays en essential role in making correct diagnoses, treatment decisions, and for future analysis. The correctness of the diagnostic result depends on the clear references information written in the description of the image finding. The other advantage of SR is the ability to link clinical documents with the referenced images for simultaneous retrieval and display.  ### 1.3 International Standardized Structure Report (SR) Example #### 1.3.1 DICOM SR As DICOM aims to develop standards for documents that incorporate references to images and associated data, DICOM SR was added to the DICOM standard at the beginning of year 2000. Structured data, analytic results and clinical observations made in the imaging environment were standardized with SR, extending DICOM beyond just images. The conventional DICOM SR is a structured report described in DICOM format. The ability to “collect” references to different types of object into a single persistent document is possible in DICOM SR. For more detail information please reference to [DICOM Content Mapping Resource in part 16 of the DICOM standard](https://dicom.nema.org/medical/dicom/current/output/chtml/part16/ps3.16.html) #### 1.3.2 FHIR SR Below are some SR example defined by HL7 FHIR: - [Breast Report]( https://build.fhir.org/ig/HL7/fhir-breast-radiology-ig/StructureDefinition-MGFinding.html) -> [Mammography findings]( https://build.fhir.org/ig/HL7/fhir-breast-radiology-ig/StructureDefinition-MGFinding.html) -> [Mammo.mass](https://build.fhir.org/ig/HL7/fhir-breast-radiology-ig/StructureDefinition-AbnormalityMass.html) - [FHIR Skin Wound Assessment Implementation Guide STU1](https://cimi.hl7.org/submissions/september_2018/skinwoundig/fullcimi/site/index.html)、[confluence: Lower+Extremity+Skin+Wound]( https://confluence.hl7.org/display/FHIR/Lower+Extremity+Skin+Wound+Assessment+FHIR+IG#LowerExtremitySkinWoundAssessmentFHIRIG-Scopeofcoverage) **Challenges of developing FHIR SR** - Various of applications: images which generated from different body part, different modality, different tissues, different pathological images of specimens, each of them will have different structured report format - Cross-disciplinary collaboration: Medical informatics department professionals need to collaborate with medical professionals department professionals in developing the structured report - The scope of medical imaging structured report is quite wide, international DICOM and FHIR have define some type of medical imaging structured report, but still there are a lot of image exam item which currently have no standard reporting specification. Pathological imaging and genetic testing also require structured and standardized reports. We can take a reference of current specifications to expand the application in other professional fields #### 1.3.3 Code and terminolgy - [LOINC Code Radiology](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6016707/)  ## 2. Project's Goal Our main goal is to adopt FHIR solution in designing medical image structured report. FHIR structured report is in XML or JSON formatted, it can be used to describe image annotation, image finding, and diagnostic report. Many of the defined elements in a resource are references to other resources. Reporting system mainly divided into 2 parts: 1. Report Creator: refers to the process when radiologist create the report. The FHIR workflow in report creator will look like this: For abnormal diagnostic result: ```flow st1=>operation: FHIR Imaging Study st2=>operation: DICOM Key Image st3=>operation: FHIR Observation Annotation st4=>operation: FHIR Observation Finding st5=>operation: FHIR DiagnosticReport st6=>operation: FHIR Document st1->st2->st3->st4->st5->st6 ``` For normal diagnostic result: ```flow st1=>operation: FHIR Imaging Study st2=>operation: FHIR DiagnosticReport st3=>operation: FHIR Document st1->st2->st3 ``` 2. Report Reviewer: After the report is created, then the clinician will review the report. Therefore, the workflow in report reviwer will be reversible from the workflow in report creator. ![](https://i.imgur.com/aIfvgcI.png =300x600) ## 3. Methods **FHIR Solution for Medical Image Annotation, Finding, and Report** As we explained earlier, FHIR structured report can be used to describe annotation, finding, and report. Here we show a FHIR solution for recording annotation, finding, and report: <table> <tr> <td>DICOM Image:</td> <td>FHIR ImagingStudy</td> </tr> <tr> <td>Annotation:</td> <td>FHIR Observation</td> </tr> <tr> <td>Finding:</td> <td>FHIR Observation</td> </tr> <tr> <td>Report:</td> <td>FHIR DiagnosticReport</td> </tr> </table> In this part, we are going to use breast image exam as the tutorial example. ### Prerequisite 1. [Create FHIR ImagingStudy to store DICOM image information](https://hackmd.io/@victoriatjia/fhir-resources-imagingStudy) 2. [Define image finding form & diagnostic report format](https://hackmd.io/@victoriatjia/fhir-module-radiology) ### Step 1. Annotation **1.1 Store Image Annotation into SVG format** Annotation is stored on SVG based on **coordinate in the original image**. Please take a look of the scenario below: ![](https://i.imgur.com/9d91lKC.jpg) Below are the real example using mammography image: Scenario 1: Radiologist display image in viewer (Fit-to-canvas Image with ratio= 0.18) ![](https://i.imgur.com/HTz7g3a.png =300x200) Scenario 2: Radiologist mark annotation ![](https://i.imgur.com/sazU0q1.png =300x200) Scenario 3: Radiologist adjust window center & window width ![](https://i.imgur.com/1c3iHij.png =300x200) Scenario 4: Radiologist zoom in ![](https://i.imgur.com/X4uPrU6.png =300x200) Scenario 5: Radiologist do image panning ![](https://i.imgur.com/5fYLFNb.jpg =300x200) Scenario 6: Radiologist save the image Scenario 7: Clinician show image to patient ![](https://i.imgur.com/1c3iHij.png =300x200) Even after radiologist perform 5 different scenario (scenario 1-5) in processing medical image, the circle annotation position still remain the same in the scenario 7. When clinician show the image to the patient, then the image will shown in a fit-to-canvas size with annotation drawn on it.  **1.2 Encode SVG into Base64 format** <table> <tr> <td><img src="https://i.imgur.com/7eoSy7P.png" width="10000" height="300"/> </td> <td><b>SVG annotation example:</b><br> < svg width="2560" height="3328"><br>< ellipse cx="1492" cy="1325" rx="371" ry="504" stroke="#ff0000" stroke-width="4" fill="none"/><br>< /svg> <br><br> <b>Base 64 conversion result:</b> PHN2ZyB3aWR0aD0iMjU2MCIgaGVpZ2h0PSIzMzI4Ij4<br>8ZWxsaXBzZSBjeD0iMTQ5MiIgY3k9IjEzMjUiIHJ4PSIzN<br>zEiIHJ5PSI1MDQiIHN0cm9rZT0iI2ZmMDAwMCIgc3Ry<br>b2tlLXdpZHRoPSI0IiBmaWxsPSJub25lIi8+PC9zdmc+ </td> </tr> </table> *Figure 3. SVG annotation for medical image* As demonstrated in figure 3, a Scalable Vector Graphics (SVG) circle is presented on a mammography image. The medical images were stored in DICOMWeb server. And the images can be accessed through DICOM WADO protocol. A web-based image viewer can use JavaScript to access the image that stored in DICOMWeb server. We can also make annotation on the image by our viewer. However, the annotation would not be stored into DICOMWeb server. This project suggests using W3C SVG to store medical image annotations. SVG is a XML-based vector image format for two-dimensional graphics. SVG specifications have clear defined geometric graphics, such as line, ellipse, etc. We can create a simple SVG webpage that links to WADO image and contains geometric graphic annotation as demonstrated in following example: ``` gherkins= <html><body><svg width="2560" height="3328"> <image x="0" y="0" width="2560" height="3328" xlink:href= "https://orthanc.dicom.tw/wado/?requestType=WADO&studyUID=1.2.1124.113532.134.0.1.3.20030423.143824.800378&%20seriesUID=1.2.1840.113681.2198909122.3931.3228559234.99.1&objectUID=1.2.1840.113681.2198909122.3931.3228559234.102.1&contentType=image/jpeg"/> <ellipse cx="1492" cy="1325" rx="371" ry="504" stroke="#ff0000" stroke-width="4" fill="none"/> </svg></body></html> ``` *Example 1. SVG web page with DICOMWeb image and a rectangle graphic* Example 1 is a WADO image and would be jpg formatted as the default WADO response result format specified in DICOM standard. And ==<ellipse cx="1492" cy="1325" rx="371" ry="504" stroke="#ff0000" stroke-width="4" fill="none"/== represent a rectangle that has 4 pixel width red border. SVG standard has richful graphic specifications supported by most browsers. Consequently, we can open example 1 by Chrome or Firefox that would have the same result as that presented in figure 3. However, the SVG annotation and WADO image might also be presented in an application based viewer. The self-developed viewer might not have the browser that support all SVG specifications defined in W3C. We must confine a subset of W3C SVG specifications that have simple and enough geometric graphics and styles that can be easily supported by viewers. **1.3 Bundle the SVG Annotation into FHIR Observation and upload into FHIR Server** We suggest that the SVG annotation would be contained in FHIR Observation and be stored in FHIR server. The FHIR Observation.valueString may be used for storing annotation information. However, FHIR Observation should be XML or JSON formatted. And W3C SVG graphics should be XML formatted. If we put SVG data directly into XML or JSON FHIR Observation.valueString, it would cause error when we upload the FHIR Observation to FHIR server. To solve the problem, we suggest that the SVG data should be base64 encoded as demonstrated in figure 3. Consequently, we could put the SVG data into FHIR Observation and upload the result to FHIR Server. The annotation FHIR Observation is demonstrated in Example 2. ```gherkin= { "resourceType": "Observation", "id": "misac.annotation01", "derivedFrom": [ { "reference": "ImagingStudy/misac.dicom.image01" } ], "component": [ { "code": { "coding": [ { "system": "https://www.dicom.org.tw/SVG", "code": "SVG.Annotation", "display": "SVG Annotation" } ] }, "valueString": "PHN2ZyB3aWR0aD0iMjU2MCIgaGVpZ2h0PSIzMzI4Ij48ZWxsaXBzZSBjeD0iMTQ5MiIgY3k9IjEzMjUiIHJ4PSIzNzEiIHJ5PSI1MDQiIHN0cm9rZT0iI2ZmMDAwMCIgc3Ryb2tlLXdpZHRoPSI0IiBmaWxsPSJub25lIi8+PC9zdmc+" }, { "code": { "coding": [ { "system": "https://www.dicom.org.tw/DCM_File", "code": "DCM File", "display": "DCM File" } ] }, "valueString": "https://orthanc.dicom.tw/wado/?requestType=WADO&studyUID=1.2.1124.113532.134.0.1.3.20030423.143824.800378& seriesUID=1.2.1840.113681.2198909122.3931.3228559234.99.1&objectUID=1.2.1840.113681.2198909122.3931.3228559234.102.1&contentType=application/dicom" } ] } ``` *Example 2. SVG base64 annotation contained in FHIR Observation.component.valueString (line 14)* The FHIR Observation can be access at following: https://hapi.fhir.org/baseR4/Observation/misac.annotation01 Beside of SVG Annotation, the dicom image url is also stored in the FHIR Observation. There are 2 types of dicom image url which store in 2 different fields in FHIR Observation: 1. A study of image: The purpose of storing this data is to record its actual source of DICOM image information. A study of image means there will be multiple series and also multiple instance, so instead of storing each instance UID of multiple images, we use the Observation.derivedFrom (see example 2 line 4) field reference to its ImagingStudy 2. An instance of image: This data refers to a single image where the annotation is marked on. Since it only has 1 image then we can record the full url of the image into Observation.component.valueString (see example 2 line 23) ### Step 2. Finding This tutorial will use breast ultrasound case as an example to describe more thoroughly about image finding. In the case of breast ultrasound, image finding have 4 types, which are mass, calcifications, focal asymmetry, and architectural distortion. Each type of lesion has its own code and represent a finding. And each lesion is represented by an Observation Resource. Below is the example of FHIR Observation that contains Mass type finding. ```gherkin= { "resourceType": "Observation", "id": "misac.finding01", "identifier": [ { "system": "http://127.0.0.1:5500/MedicalImagingExamWorkflow/findingReport/MammoMass.html?annotationID=769&rowNum=1&findingType=Mammo%20Mass&imagingStudyID=757&patientStudyID=858b1cca-246800b9-ccf2765e-96783fe1-109d3b18", "value": "mass" } ], "status": "final", "code": { "coding": [ { "system": "http://hl7.org/fhir/STU3/valueset-observation-codes.html", "code": "RID39055" } ] }, "subject": { "reference": "Patient/misac.patient01" }, "derivedFrom": [ { "reference": "Observation/misac.annotation01" } ], "component": [ { "code": { "coding": [ { "system": "http://hl7.org/fhir/valueset-bodysite-laterality.html", "code": "MB", "display": "Bilateral" } ] } }, { "code": { "coding": [ { "fhir_comments": [ "Location" ], "system": "http://203.64.84.218/mammoDicomWebviewer/newCodeSystem/LocationCS.html", "code": "breast.left.uoq", "display": "Upper outer quadrant of left breast" } ] } }, { "code": { "coding": [ { "fhir_comments": [ " One view Only " ], "system": "http://203.64.84.218/mammoDicomWebviewer/newCodeSystem/HemisphereCS.html", "code": "UH", "display": "Upper_hemisphere" } ] } }, { "code": { "coding": [ { "system": "http://203.64.84.218/mammoDicomWebviewer/newCodeSystem/SizeCS.html", "code": "3-4 cm", "display": "3-4 cm" } ] } }, { "code": { "coding": [ { "system": "http://hl7.org/fhir/us/breast-radiology/2019Sep/ValueSet-breastrad-ShapeVS.html", "code": "IrregularShape", "display": "Neither round nor oval" } ] } }, { "code": { "coding": [ { "system": "http://hl7.org/fhir/us/breast-radiology/2019Sep/ValueSet-breastrad-MarginVS.html", "code": "ObscuredMargin", "display": "Obscured Margin" } ] } }, { "code": { "coding": [ { "system": "http://hl7.org/fhir/us/breast-radiology/2019Sep/ValueSet-breastrad-AbnormalityDensityVS.html", "code": "FatContaining", "display": "Includes all masses containing fat, such as oil cyst, lipoma, or galactocele as well as mixeddensity lesions such as hamartoma." } ] } } ] } ``` *Example 3. The annotation (FHIR Observation) can be referred by image finding (another FHIR observation) stored in Observation.derivedFrom* The FHIR Observation can be access at following URL:https://hapi.fhir.org/baseR4/Observation/misac.finding01 This is an example of mass type finding. The value of Observation.derivedFrom (see example 3 line 3) object reference to the FHIR Observation resource which contains the finding’s annotation. Meanwhile the value of the “component” object contains finding’s description such as the location, margin, size, density, and the other. ### Step 3. Diagnostic Report After all findings have been written into its own type of lesion, radiologists need to complete the diagnostic report. It contains one report and multiple findings. The diagnosticReport which consists of the entire inspection data after a certain medical visit will be uploaded to the FHIR server in diagnosticReport resource type. ``` gherkin= { "resourceType": "DiagnosticReport", "result": [ { "reference": "Observation/misac.finding01" } ], "imagingStudy": [ { "reference": "ImagingStudy/misac.dicom.image01" } ], "conclusionCode": [ { "coding": [ { "system": "Category0", "code": "Category0", "display": "Category 0:\n\t\t\t\t\t\t\t\t\tNeed Additional Imaging Evaluation" }, { "system": "ThickeningSkin_0", "code": "ThickeningSkin_0", "display": "Thickening of the skin of right breast" } ] } ] } ``` The FHIR diagnosticReport can be accessed from following URL: https://hapi.fhir.org/baseR4/DiagnosticReport/misac.report.radiology01 ### Step 4. Bundle (Update soon) ### 4. Result #### 4.1 Report Creator (from Radiologist point of view) 1. Mark annotation ![](https://i.imgur.com/NaYVhHM.png) 2. Input image finding form ![](https://i.imgur.com/Ly3keTO.png) 3. Input diagnostic report form ![](https://i.imgur.com/KaWmr6C.png =450x240) Demo website: https://victoriatjia.github.io/FHIR_ImageExam/report-creator.html #### 4.2 Report Reviewer (from Clinicians point of view) ![](https://i.imgur.com/65EqTIV.gif) Demo website: https://victoriatjia.github.io/FHIR_ImageExam/report-reviewer.html You can use this "misac.patient01" patient ID on the demo website ## References - [DICOM SR](https://www.youtube.com/watch?v=1ShD631sZfg)