--- tags: English --- # SideeX Test Report Specification This specification describes the test report format of SideeX. A report file contains the test results (may be of multiple test suites) of one-shot playback. The file is a general HTML file with file extension name ```.html``` or a JSON file with ```.json``` > - [HTML Format](#HTML-Format) > - [JSON Format](#JSON-Format) ## HTML Format - Starting with a ```head```, and a ```body``` - An example ```html <html> <head> ... </head> <body> ... </body> </html> ``` ### The head part - contains some meta data with `<meta>`, css style with `<style>`, title of html with `<title>` - meta tags - the charset is "UTF-8" - title tag - the playback start time - style tag - modal style: for viewing snapshot / video in full screen - log style: success / fail / idle - table style - th, td style - script code - The code for showing report charts in browser. - An example ```html <meta content="text/html; charset=UTF-8"> <title>20201208 14-50-30</title> <style> table { border-collapse: collapse; width: 100%; } tr, td { padding: 2px; padding-left: 4px; } .success { background-color: #e5ffe5; } ... </style> <script> // drawing chart... </script> ``` ### The body part - contains 4 parts: the report summary, the status and snapshot for each command, the time charts, and the log. - summary - The summary is presented as a ```<table>``` with ```id="summary"``` - The ```<tbody>``` contains the infromation about SideeX version, running browser, platform, language, start time, end time, numbers of passed suites and passed cases - An example ```html <table id="summary"> <thead> <tr><td>Summary</td></tr> </thead> <tbody> <tr> <td>SideeX Version</td> <td>:</td> <td id="sideex-version">3.4.2</td> </tr> <tr> <td>Browser</td> <td>:</td> <td id="browser">Chrome 86.0.4240.198</td> </tr> <tr> <td>Platform</td> <td>:</td> <td id="platfrom">Windows 10 64-bit</td> </tr> ... </tbody> </table> ``` - commands - The status and snapshots of all commands are presented as a ```<table>``` with ```id="commands"```. - The `<tbody>` contains a sequence of ```<tr>``` elements, each `<tr>` element includes: - `test suite` - `test case` - `local variable` - `global variable` - `command` - A ```<tr>``` test suite is followed by one or multiple ```<tr>``` test cases. A ```<tr>``` test case is followed by one or multiple ```<tr>``` commands, local variables or global variables. Each ```<tr>``` or ```<td>``` is annotated with a class attribute. - An example ```html <table id="commands"> <thead> <tr><td>Commands</td></tr> </thead> <tbody> <tr class="suite-title"><td colspan="4">Untitled Test Suite</td></tr> <tr class="case-title"><td colspan="4">Untitled Test Case</td></tr> <tr class="local-variable"> <td class="variable-name">Name</td> <td class="variable-value">Value</td> </tr> <tr class="global-variable"> <td class="variable-name">Name</td> <td class="variable-value">Value</td> </tr> <tr class="command success"> <td class="command-name">Command</td> <td class="command-target">Target</td> <td class="command-value">Value</td> <td class="command-snapshot"><img src="data:image/jpeg;base64,..."</td> </tr> <tr class="command fail"> ... </tbody> </table> ``` - charts - Showing time charts using [Google visualization line chart](https://developers.google.com/chart/interactive/docs/gallery/linechart) API ```html <div class="time-line" id="time-line-chart">Connect to network to display the chart.</div> <div class="time-line" id="time-line-chart-time">Connect to network to display the chart.</div> <div class="time-line" id="time-line-chart-suites">Connect to network to display the chart.</div> ``` - logs - All logs are presented as a ```<table>``` with ```id="logs"``` - Each `<tr>` has line number, log type and log message - An example ```html <table id="logs"> <thead> <tr><td>Logs</td></tr> </thead> <tbody> <tr class="log info"> <td class="log-line">1</td> <td class="log-type">INFO</td> <td class="log-content"><span>Playing test suite Untitled Test Suite</span></td> </tr> <tr class="log error"> ... </tbody> </table> ``` - script code - The code for showing snapshots and videos while viewing the report in browser. ### Example file ```html <html> <head> <meta content="text/html; charset=UTF-8"> <title>20201208 14-50-30</title> <style> </style> </head> <body> <div> <img src="data:image/png;base64,..."> <h1>Test Report</h1> </div> <table id="summary"> <colgroup> <col span="1" width="150px;"> <col span="1"> <col span="1"> </colgroup> <thead> <tr> <td class="table-title" colspan="3">Summary</td> </tr> </thead> <tbody> <tr> <td>SideeX Version</td> <td>:</td> <td id="sideex-version">3.4.2</td> </tr> <tr> <td>Browser</td> <td>:</td> <td id="browser">Chrome 86.0.4240.198</td> </tr> <tr> <td>Platform</td> <td>:</td> <td id="platform">Windows 10 64-bit</td> </tr> <tr> <tr> <td>Language</td> <td>:</td> <td id="language">(default)</td> </tr> <td>Start Time</td> <td>:</td> <td id="start-time">20201208 14:47:23</td> </tr> <tr> <td>End Time</td> <td>:</td> <td id="end-time">20201208 14:47:24</td> </tr> <tr> <td>Passed Suite(s)</td> <td>:</td> <td id="suite-runs">1 / 1</td> </tr> <tr> <td>Passed Case(s)</td> <td>:</td> <td id="case-runs">1 / 1</td> </tr> </tbody> </table> <table id="commands"> <colgroup> <col span="1" style="width: 12%;"> <col span="1" style="width: 28%;"> <col span="1" style="width: 30%;"> <col span="1" style="width: 30%;"> </colgroup> <thead> <tr class="suite-title"> <td colSpan="4">Untitled Test Suite</td> </tr> <tr class="case-title"> <td colSpan="4">Untitled Test Case</td> </tr> <tr class="case-localVars"> <td colSpan="4">Local Variables</td> </tr> <tr> <td colSpan="2">Name</td> <td colSpan="2">Value</td> </tr> <tr class="local-variable"> <td colSpan="2" class="variable-name">title</td> <td colSpan="2" class="variable-value">SideeX</td> </tr> <tr class="case-globalVars"> <td colSpan="4">Global Variables</td> </tr> <tr> <td colSpan="2">Name</td> <td colSpan="2">Value</td> </tr> <tr class="global-variable"> <td colSpan="2" class="variable-name">length</td> <td colSpan="2" class="variable-value">10</td> </tr> <tr class="case-commands"> <td colSpan="4">Commands</td> </tr> <tr> <td>Command</td> <td>Target</td> <td>Value</td> <td><img src="data:image/png;base64,..."></td> </tr> <tr class="command success"> <td class="command-name">open</td> <td class="command-target">https://sideex.io/</td> <td class="command-value"></td> <td class="command-snapshot">No image</td> </tr> <tr class="command success"> <td class="command-name">clickAt</td> <td class="command-target">//h1[1]</td> <td class="command-value">461,100</td> <td class="command-snapshot"><img src="data:image/jpeg;base64,..."></td> </tr> <tr class="command success"> <td class="command-name">store</td> <td class="command-target">SideeX</td> <td class="command-value">title</td> <td class="command-snapshot">No Image</td> </tr> <tr class="command success"> <td class="command-name">storeGlobalVar</td> <td class="command-target">10</td> <td class="command-value">length</td> <td class="command-snapshot">No Image</td> </tr> </tbody> </table> <div class="time-line" id="time-line-chart">Connect to network to display the chart.</div> <div class="time-line" id="time-line-chart-time">Connect to network to display the chart.</div> <div class="time-line" id="time-line-chart-suites">Connect to network to display the chart.</div> <table id="logs"> <colgroup> <col span="1"> <col span="1"> <col span="1"> </colgroup> <thead> <tr> <td class="table-title" colspan="3">Logs</td> </tr> </thead> <tbody> <tr class="log info"> <td class="log-line">1</td> <td class="log-type">INFO</td> <td class="log-content"><span>Playing test suite Untitled Test Suite</span></td> </tr> <tr class="log info"> <td class="log-line">2</td> <td class="log-type">INFO</td> <td class="log-content"><span>Playing test case Untitled Test Case</span></td> </tr> <tr class="log info"> <td class="log-line">3</td> <td class="log-type">INFO</td> <td class="log-content"><span>Initializing: | open | https://sideex.io/ | |</span></td> </tr> <tr class="log info"> <td class="log-line">4</td> <td class="log-type">INFO</td> <td class="log-content"><span>Executing: | open | https://sideex.io/ | |</span></td> </tr> <tr class="log info"> <td class="log-line">5</td> <td class="log-type">INFO</td> <td class="log-content"><span>Initializing: | clickAt | //h1[1] | 461,100 |</span></td> </tr> <tr class="log info"> <td class="log-line">6</td> <td class="log-type">INFO</td> <td class="log-content"><span>Executing: | clickAt | //h1[1] | 461,100 |</span></td> </tr> <tr class="log info"> <td class="log-line">7</td> <td class="log-type">INFO</td> <td class="log-content"><span>Initializing: | store | SideeX | title |</span></td> </tr> <tr class="log info"> <td class="log-line">8</td> <td class="log-type">INFO</td> <td class="log-content"><span>Executing: | store | SideeX | title |</span></td> </tr> <tr class="log info"> <td class="log-line">9</td> <td class="log-type">INFO</td> <td class="log-content"><span>Initializing: | storeGlobalVar | 10 | length |</span></td> </tr> <tr class="log info"> <td class="log-line">10</td> <td class="log-type">INFO</td> <td class="log-content"><span>Executing: | storeGlobalVar | 10 | length |</span></td> </tr> <tr class="log info"> <td class="log-line">11</td> <td class="log-type">INFO</td> <td class="log-content"><span>Test case passed</span></td> </tr> </tbody> </table> <div id="my-modal"> <img id="modal-image"> <video id="modal-video" controls>Your browser doesn't support video tag.</video> </div> <script> ... </script> </body> </html> ``` ## JSON Format - It follows the standard JSON format with two spaces for indent ### Keys - version - Description: - version of this file - children - sideex - Description: - The version of SideeX where this file is generated from - Type: ```Array``` with three numbers - format - Description: - The version of file format - Type: ```Array``` with three numbers - Type: ```Object``` - title - Description - The title of the report file - Type: ```String``` - type - Description - The browser name on playback - Type: ```String``` - browser - Description - the browser version on playback - Type: ```String``` - platform - Description - the platform on playback - Type: ```String``` - lang - Description - the browser language on playback - Type: ```String``` - Optional - inputFile - Description - the file name of test case source - Type: ```String``` - Optional - suites - Description: - each element is a test suite - Type: ```Array``` with ```Object``` - Children - idText - Description: - the id text of this suite - Type: ```String``` - title - Description: - the name of this suite - Type: ```String``` - cases - Description: - the indices of the test cases belonging to this suite - Type: ```Array``` with ```Number``` - cases - Description: - each element is a test case - Type: ```Array``` with ```Object``` - Children - idText - Description: - the id of this case - Type: ```String``` - suiteIdText - Description: - the test suite id that this case belongs to - Type: ```String``` - title - Description: - the name of this case - Type: ```String``` - localVars - Description: - all local variables of this case - Type: ```Array``` with ```Object``` - Children - name - Description: - the name of this local variable - Type: ```String``` - value - Description: - the value of this local variable - Type: ```String``` - globalVars - Description: - all global variables of this case - Type: ```Array``` with ```Object``` - Children - name - Description: - the name of this global variable - Type: ```String``` - value - Description: - the value of this global variable - Type: ```String``` - records - Description: - all records of this case - Type: ```Array``` with ```Object``` - Children - name - Description: - the name of this record - Type: ```String``` - target - Description: - the target of this record - Type: ```String``` - value - Description: - the value of this record - Type: ```String``` - snapshot - Description: - the key of the screenshot of this record - Type: ```String``` - status - Description: - the status of this record - it will be `success`, `fail`, `execute`, or `default` - Type: ```String``` - time - Description: - the execution time of this record - Type: ```Number``` - Optional - logs - Description: - the logs of report - Type: ```Array``` with ```Object``` - Children - type - Description: - the type of this log - Type: ```String``` - message - Description: - the message of this log - Type: ```String``` - snapshot - Description: - all snapshots - Type: `Object` - Children: - url - Description: - the url of this snapshot - Type: ```String``` - isCompressed - Description: - whether this snapshot is compressed or not - Type: ```Boolean``` - successSuiteNum - Description: - the number of successful suites - Type: `Number` - failureSuiteNum - Description: - the number of failed suites - Type: `Number` - successCaseNum - Description: - the number of successful cases - Type: `Number` - failureCaseNum - Description: - the number of fail suites - Type: `Number` - startTime - Description: - the start time of playback - Type: `String` - endTime - Description: - the end time of playback - Type: `String` - status - Description: - the status of playback - it will be `execute` or `completed` - Type: `String` ### Example ```jsonld= { "version": { "sideex": [3, 4, 2], "format": [1, 0, 1] }, "title": "20201208 14-50-30", "type": "chrome", "browser": "Chrome 86.0.4240.198", "platform": "Windows 10 64-bit", "lang": null, "suites": [ { "title": "Untitled Test Suite", "idText": "suite-0", "cases": [0] } ], "cases": [ { "title": "Untitled Test Case", "idText": "case-0", "suiteIdText": "suite-0", "localVars": [ { "name": "title", "value": "SideeX" } ], "globalVars": [ { "name": "length", "value": "10" } ], "records": [ { "name": "open", "target": "https://sideex.io/", "value": "", "status": "success", "time": 407 }, { "name": "clickAt", "target": "//h1[1]", "value": "461,100", "snapshot": "image-2", "status": "success", "time": 130 }, { "name": "store", "target": "SideeX", "value": "title", "status": "success", "time": 59 }, { "name": "storeGlobalVar", "target": "10", "value": "length", "status": "success", "time": 81 } ] } ], "logs": [ { "type": "info", "message": "Playing test suite Untitled Test Suite" }, { "type": "info", "message": "Playing test case Untitled Test Case" }, { "type": "info", "message": "Initializing: | open | https://sideex.io/ | |" }, { "type": "info", "message": "Executing: | open | https://sideex.io/ | |" }, { "type": "info", "message": "Initializing: | clickAt | //h1[1] | 461,100 |" }, { "type": "info", "message": "Executing: | clickAt | //h1[1] | 461,100 |" }, { "type": "info", "message": "Initializing: | store | SideeX | title |" }, { "type": "info", "message": "Executing: | store | SideeX | title |" }, { "type": "info", "message": "Initializing: | storeGlobalVar | 10 | length |" }, { "type": "info", "message": "Executing: | storeGlobalVar | 10 | length |" }, { "type": "info", "message": "Test case passed" } ], "snapshot": { "image-2": { "url": "data:image/jpeg;base64,/9j/4AAQSkZJ....", "isCompressed": true } }, "successSuiteNum": 1, "failureSuiteNum": 0, "successCaseNum": 1, "failureCaseNum": 0, "startTime": "20201208 14:50:30", "endTime": "20201208 14:50:31", "status": "completed" } ```