# LOC Studio Guidebook LOC Studio V1.0.0 Release Date: 8 May 2022 (latest version) ## About LOC Studio LOC Studio is one of FST VADO's tools (Virtual Assistive Data Officer) to help users build and execute complex business/data processes. *LOC Studio will automatically log out the current user after **5 mins** of inactivity. ## Benefits - **Effective Workflow**: Allow IT and business units to have clearer and more smooth communication cycles. - **Function-as-a-Service (FaaS)**: Allow users to only focus on business logic by writing simple code to create application functionalities without considering much about the complex infrastructure. - **Event Sourcing:** Ensure the state change of a certain application/service/system to be captured by events. - **Business Activity Monitoring (BAM):** Provide real-time information about the status and results of the processes and transactions of your interest. ## Concepts - **LOC Studio**: LOC Studio is a tool for users to access LOC core functions and to deploy/update resources with a graphical user interface. - **Business Process**: A business process is a collection of business tasks and activities (provided by users), either of which generate a result that will meet a business target. - **Data Process**: In the commercial world, a data process refers to the processing of data required to run business processes in enterprises. A data process is to utilise data via LOC. It converts raw data into a business-usable form that transforms the multiple generic logics and an aggregator logic into an output. When it comes to using LOC to perform a pre-defined operation on data, a data process will be generated. - **Generic Logic**: A generic logic is the basic component to form a data process. It can be a descriptive record of an end user’s static data, dynamic trade data, or behavioural data; a piece of information or an event in the existing system; or a data change in a database. - **Aggregator Logic**: In a data process, there is only one aggregator logic which is to aggregate all the generic logics of the same data process. - **DID**: A Digital Identity (DID) is essentially a unique string (e.g. UUID/CUID/GUID, a hash, or a random string) and holds labels (to be accurate, the source and target DID). A DID can be generated or derived by logic or manual setting. - **Label Name**: A Label Name is an elementary metadata unit in LOC, named by any arbitrary string. Each label name has a unique DID which can be set up by logics or manual setting. - **Potential Labelling**: It is a schema used to set up an event. - **Event**: An emitted event is stored in the event store inside of LOC. - **Agent**: An agent is the connector when a logic proactively acquires data. - **API Route**: An API route is an API used to set up a trigger that will invoke a data process. - **HTTP**: Hypertext Transfer Protocol (HTTP) is an application-layer protocol for transmitting hypermedia documents, such as HTML. https://developer.mozilla.org/en-US/docs/Web/HTTP - **BAM**:Business Activity Monitoring (BAM) describes the processes and technologies that enhance situation awareness and enable analysis of critical business performance indicators based on real-time data. - **Event Sourcing**: The Event Sourcing defines an approach to handling operations on data that is driven by a sequence of events, each of which is recorded in an append-only store. https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing ## User Management After clicking `Users` from the left navigation bar, you can select `View all users` to display all users. ![](https://hackmd.io/_uploads/SyQaqczBq.png) In this section, you can not only view all users but also add, edit and delete them. ==Please DO NOT delete any accounts that have been active already, in case their data, such as configured logics or data processes, will be eliminated and affect the operation of the existing accounts. Instead, Please DEACTIVATE them (disable users).== ![](https://hackmd.io/_uploads/SykX0czrc.png) :::info Audience: **LOC Studio Admin** Note: Only the Admins have the right to view, create, edit, and delete users. ::: ### Create New Users - Step 1: Log into LOC Studio User Management with your `Admin account`. - Step 2: On the left menu, please select `Users`. - Step 3: On the top right, please click `Add Users`. - Step 4: Please fill in the user information as detailed as possible. (`Username` and `Email` are mandatory.) ![](https://hackmd.io/_uploads/SkqkksGrc.png) There are a few more steps to be completed. - Step 5: In the `Credentials` tab, please ensure the `Temporary` button to be switched on to set a temporary password for the user and inform the user to change the password upon login. ![](https://hackmd.io/_uploads/ByBnkjMrq.png) Don’t forget this temporary password and remember to give it to the user. The users are required to reset their password when they login to their account for the first time. ![](https://hackmd.io/_uploads/ryobnEq45.png) - Step 6: In `Role Mappings` tab, you can check the role authorisation of the selected user. ![](https://hackmd.io/_uploads/SJN7h4qNq.png) ### Remove Users - Step 1: Log into LOC Studio User Management with your `Admin account`. - Step 2: On the left menu, please select `Users`. - Step 3: Under `Actions`, please select the user you would like to remove and click `Delete`. ![](https://hackmd.io/_uploads/HJmdxszr9.png) ### Forgot Passwords - Step 1: Log into LOC Studio User Management with your `Admin account`. - Step 2: On the left menu, please select `Users`. - Step 3: Under `Actions`, please select the user you would like to reset the password and click `Edit`. - Step 4: Switch to the `Credentials` tab, under which you will find the `Reset Password` section. - Step 5: After deleting the old password, please ensure the `Temporary` button to be switched on to set a temporary password for the user and inform the user to change the password upon login. - Step 6: That user ==MUST== reset the password after login. ![](https://hackmd.io/_uploads/S1nGY5sE5.png) ## Data Discovery Introduction ### Event This repository is where users can see the relationship between each `event` or switch to graph for visualisation, helping to fulfil event sourcing. ![](https://hackmd.io/_uploads/HJdZAal8q.png) By clicking **+**, you can see more details of this particular event. ![](https://hackmd.io/_uploads/rk-a19oEc.png) Also, switching to the graph and setting up the filter of your interest allow you to see a clearer relationship of each `event` for event sourcing or BAM. (Without Filter) ![](https://hackmd.io/_uploads/rJ_ykRgUq.png) (With Filter) ![](https://hackmd.io/_uploads/HyCM1S-I5.png) ### Label Name Every `event` has exactly one `label name` and two `DIDs` (source and target). However, one `label name` can be re-used in different `events`. In this repository, you are able to filter a specific `label name` for all the `DIDs`related to that. ![](https://hackmd.io/_uploads/HkVi-co4c.png) ### DID This repository is where users can see the relationship between the `DID` and the `label name`. You can choose a specific `DID` and see all the `label names` related to it. ![](https://hackmd.io/_uploads/SkE2GqiNc.png) ### Application of Data Discovery Whenever an `event` is emitted, 1 `label name` and 2 `DIDs` (source and target) will be created as what you have set up. Additionally, you might have created multiple `events` with the same `label name`, or multiple `DIDs` related to the same `label name`. In this case, each repository has its work to do. For example, in the graph shown below, there are 4 `events` with 2 different `label names`, 4 different `source DIDs`, and 3 different `target DIDs`. We can go to [Event](###Event) and search all the `events` in this execution to get this relationship graph. In addition, you may feel interested in how many `DIDs` have been created with a specific `label name`; let's say, *labelName_1*. After searching *labelName_1* in the [Label Name](###Label-Name), you will get 3 `DIDs` (2 source and 1 target). On the other hand, you can go to [DID](###DID) to check out how many `label names` have been put onto a specific `DID`; let's say *Target_1*. In this case, you will see 2 `label names` have been put onto *Target_1* in the sense that 2 `events` share the same `target DID` (Target_1). In conclusion, the feature of [Data Discovery](##Data-Discovery-Introduction) helps our users to quickly inspect the relationship of the `events` and discover some useful information or insights on a certain `event`, `label name`, or `DID`. ![](https://hackmd.io/_uploads/BkK6ByHB5.jpg) ## Data Process Explorer Introduction After logging into LOC Studio, please click the menu of `Data Process` on the left. Afterwards, please select `DP Interactive Map`. ![](https://hackmd.io/_uploads/SyoPYc_Mc.jpg) In this section, we will be explaining the structure of `DP Interactive Map` and how to start deploying a data process via LOC Studio. ### Unit A **Unit** is default in LOC Studio at the beginning. You can regard a **Unit** as one business unit such as marketing, sales, operation, etc. Normally speaking, there is only one (business) **Unit** for a user to access. If you want to know more about the information of a certain **Unit**, please right click **Unit** and select `Unit Info`. ![](https://hackmd.io/_uploads/S1gHocOM5.jpg) #### Create a Unit For the time being, it is not supported for users to create a **Unit**. If you need to create one, please contact your Admin or FST Network. #### Edit a Unit For the time being, it is not supported for users to edit the **Unit**. If you need to edit one, please contact your Admin or FST Network. #### Remove a Unit For the time being, it is not supported for users to remove a **Unit**. If you need to remove one, please contact your Admin or FST Network. ### Project If you want to know more about the information of a certain **Project**, please right click that **Project** and select `Project Info`. #### Create a Project If you would like to create a **Project**, please right click **Unit** and select `New Project`. After filling in the **Project Name** and click **Create**, this project will be created. In each **Unit**, you can create several projects. ![](https://hackmd.io/_uploads/B1Bx6q_zq.jpg) :::info Please note that - `Project Name` is required. - `Description` is optional. - In each **Unit**, `Project Name` cannot be the same. ::: #### Edit a Project If you would like to edit a specific **Project**, please right click that **Project** and select `Project Info`. In the pop-up window, please click `Edit` to start your revision. Once completing the revision, please click `Update` to keep your latest revision; if you would like to give up this revision, please click `Cancel` to exit. ![](https://hackmd.io/_uploads/r10oyiuG9.jpg) #### Remove a Project If you would like to remove a specific **Project**, please right click that **Project** and select `Delete Project`. ![](https://hackmd.io/_uploads/BkM66uYfq.jpg) :::info Please note that once you `Delete Project`, there is NO WAY to make it back. Please think twice before your action. ::: ### Scenario If you want to know more about the information of a certain **Scenario**, please right click that **Scenario** and select `Scenario Info`. ![](https://hackmd.io/_uploads/Sky6RdtMc.jpg) #### Create a Scenario If you would like to create a **Scenario**, please right click **Project** and select `New Scenario`. After filling in the **Scenario Name** and click **Create**, this scenario will be created. In each **Project**, you can create several scenario. ![](https://hackmd.io/_uploads/rkM8Dp7L9.png) :::info Please note that - `Scenario Name` is required. - `Description` is optional. - In each **Project**, `Scenario Name` cannot be the same. ::: #### Edit a Scenario If you would like to edit a specific **Scenario**, please right click that **Scenario** and select `Scenario Info`. In the pop-up window, please click `Edit` to start your revision. Once completing the revision, please click `Update` to keep your latest revision; if you would like to give up this revision, please click `Cancel` to exit. ![](https://hackmd.io/_uploads/ryZpJKYM9.jpg) #### Remove a Scenario If you would like to remove a specific **Scenario**, please right click that **Scenario** and select `Delete Scenario`. ![](https://hackmd.io/_uploads/BJnrxttf9.jpg) :::info Please note that once you `Delete Scenario`, there is NO WAY to make it back. Please think twice before your action. ::: ### Data Process If you want to know more about the information of a certain **Data Process**, please right click that **Data Process** and select `Data Process Info`. #### Create a Data Process If you would like to create a **Data Process**, please right click **Scenario** and select `New Data Process`. After filling in the **Data Process Name** and click **Create**, this data process will be created. In each **Scenario**, you can create several data processes. ![](https://hackmd.io/_uploads/SyYaxFtzq.jpg) :::info Please note that - `Data Process Name` is required. - `Description` is optional. - `Execution Timeout` is required and defaulted as 180 seconds. If the data process is too complicated with a large amount of data, when executing the data process, it might fail because of the default execution timeout. - In each **Scenario**, `Data Process Name` cannot be the same. ::: #### Edit a Data Process If you would like to edit a specific **Data Process**, please right click that **Data Process** and select `Data Process Info`. In the pop-up window, please click `Edit` to start your revision. Once completing the revision, please click `Update` to keep your latest revision; if you would like to give up this revision, please click `Cancel` to exit. #### Remove a Data Process If you would like to remove a specific **Data Process**, please right click that **Data Process** and select `Delete Data Process`. :::info Please note that once you `Delete Data Process`, there is NO WAY to make it back. Please think twice before your action. ::: ### Logic A (generic) logic is the minimum component of a data process which must be composed with ==AT LEAST== `1 (generic) logic` and `1 aggregator logic` on LOC Studio. #### Create a (Generic) Logic If you would like to create a **(Generic) Logic**, please click `+Generic Logic` to choose where it suits you and start coding. ![](https://hackmd.io/_uploads/SJSjdp78q.png) In the `Logic Body` tab, you can either write codes yourself or use `+New Potential Labelling`, and the latter is only for creating an event. If you choose to write your own codes, here are some agent templates for your reference. ##### Event Logic ```javascript await ctx.agents.eventStore.emit([ { sourceDID: "apar-js/examples/eventEmitExample", targetDID: "saffron-eventstore", labelName: "TEST_LABEL_FROM_EXAMPLE", meta: "這是一個測試標籤", type: "default", // or "error", }, ]) ``` ##### Logic for HTTP Agents ```javascript let resp = await ctx.agents.http!.post( "https://6051775217ef.mock.pstmn.io/test123", {}, Http.ContentType.Json, new Uint8Array([123, 34, 97, 34, 58, 51, 51, 125]) ); ``` ##### Logic for DB Agents (Microsoft SQL) ```javascript const connectionInfo = { server: process.env.MSSQL_HOST, port: process.env.MSSQL_PORT, username: process.env.MSSQL_USERNAME, password: process.env.MSSQL_PASSWORD, database: process.env.MSSQL_DATABASE, trustCert: process.env.MSSQL_TRUSTCERT, }; let db = null; try { db = await ctx.agents.database?.connect({ databaseDriver: Database.Driver.Mssql, connectionString: `server=${connectionInfo.server},${connectionInfo.port};uid=${connectionInfo.username};pwd=${connectionInfo.password};database=${connectionInfo.database};TrustServerCertificate=${connectionInfo.trustCert};`, }); ``` As for using `+New Potential Labelling`, you need to firstly right click on where you would like to put in the `Logic Body` and then simply put your `Label Name`, `Meta`, `Source DID`, and `Target DID` into the pop-up window. Please note that only `Meta` optional to be filled in, the rest are required. Additionally, the potential labelling is used to emit an event. ![](https://hackmd.io/_uploads/HkNM-R8Q5.png) After completing the form, these information will be automatically transformed into codes and shown in the `Logic Body`, where you can revise further. ![](https://hackmd.io/_uploads/SkPImioN9.png) :::info Please note that `Label Name`, `Meta`, `Source DID`, and `Target DID` will be treated as string when converting from the potential labelling to the logic body. ::: #### Create an Aggregator Logic If you would like to create an **Aggregator Logic**, please click `Add Aggregator Logic` to start coding. Here are the templates of the aggregator logic for your reference. - if OK ```javascript export async function run(ctx: AggregatorContext) { const result = (await ctx.agents.sessionStorage.get("result")) as object; ctx.agents.result.finalize({ returnCode: "000", // or other success codes returnMsg: "Success", // or other success message }); } ``` - if ERROR ```javascript export async function handleError(ctx: AggregatorContext, error: RailwayError) { ctx.agents.logging.error(error.message); } ``` :::info Please note that - `Logic Name` is required. - In each **Data Process**, `Logic Name` cannot be the same. - `Description` and `Comment` are optional. - `Description` contains general words that apply to all versions of this logic even if this logic is revised. - `Comment` contains specific words to describe the version of this logic. - LOC Studio currently supports JavaScript only, but more programming languages will be added, such as TypeScript, etc. ::: #### Edit a Logic If you would like to edit a specific **Logic**, please click this icon "⋮" of that **Logic** on the top right and select the edit icon as shown below to start your revision. Once completing the revision, please click `Update` to keep your latest revision; if you would like to give up this revision, please click `Cancel` to exit. ![](https://hackmd.io/_uploads/ryHBRRUXc.jpg) #### Remove a Logic If you would like to delete a specific **Logic**, please right click that **Logic** and select the rubbish bin icon as shown below. ![](https://hackmd.io/_uploads/r1J1yywXq.jpg) :::info Please note that once you `Delete Logic`, there is NO WAY to make it back. Please think twice before your action. ::: ## API Route Configuration Introduction After logging into LOC Studio, you can find the menu of **API Route** on the left. Once completing deploying a data process, you can configure an API route to execute it. :::info - Please note that only the deployed data processes can be configured here. - The order of the linked data processes is important and will affect the result. ::: ![](https://hackmd.io/_uploads/Bkba9HNN5.png) #### Create an API Route If you would like to create an **API Route**, please click **+** right next to `API Route Explorer` to create a new folder first by filling out the information. Before creating the folder: ![](https://hackmd.io/_uploads/Sy0H5WrN9.png) After creating the folder: ![](https://hackmd.io/_uploads/rkdcFbB45.png) :::info Please note that - `Folder Name` is required. - `Description` is optional. - In each folder, `Folder Name` cannot be the same. ::: By clicking `Create API Route` on the top right, you will be able to configure an API for a specific data process. We need to stress that only the deployed data processes can be added; otherwise, the undeployed data processes cannot be selected, and hence the API route configuration cannot be completed. ![](https://hackmd.io/_uploads/BkC9WzSE5.png) :::info Please note that - `API Route Name`, `HTTP Method`, `HTTP Path`, and `Linked Data Process` are required. - `Description` is optional. - In `HTTP Path`, you only need to fill in the sub-path; the domain name is default in the system and thus there is no need to put domain name here. (eg, complete HTTP Path might be `https://api.loc-cse.fst.network/api/v1/search`; here, only `/api/v1/search` is needed to be filled in.) - `HTTP Path` cannot be the same. ::: After configuring an API, you will have this success page. By using this API and requesting it on any API platforms (such as Postman, Insomnia, etc), it will return the result as stated in the aggregator logic of your data process. ![](https://hackmd.io/_uploads/BJPeXfrV9.png) (Requesting on Postman) ![](https://hackmd.io/_uploads/H1eo6K6mL5.png) #### Edit an API Route If you would like to edit a specific **API Route**, please click that **API Route** on the top right. In the pop-up window, please click `Edit` as shown below to start your revision. Once completing the revision, please click `Update` to keep your latest revision; if you would like to give up this revision, please click `Cancel` to exit. ![](https://hackmd.io/_uploads/SythwMBEq.png) #### Remove an API Route If you would like to remove a specific **API Route**, please right click that **API Route** and select `Delete API Route`. ![](https://hackmd.io/_uploads/ByfVuGB4q.png) :::info Please note that once you `Delete API Route`, there is NO WAY to make it back. Please think twice before your action. ::: ## Quick Start The quick start section will show you how to create your first data process in LOC Studio. This data process is extremely simple, which will consume a JSON data as data source and emit an event. ### Create a User Account #### Step 1: Create a New User In order to create a data process, you'll need a user account. Logging in with the *User Management* account (provided by FST Network), on the menu of `Users`, click `Add user` to create a new account: ![](https://hackmd.io/_uploads/HJSewg2Bc.png) #### Step 2: Fill in User Information `Username` and `Email` are required: ![](https://hackmd.io/_uploads/By0cdlhrq.png) ![](https://hackmd.io/_uploads/Hkh-tx3Sc.png) > We strongly recommend filling user information as detailed as possible. #### Step 3: Set a Temporary Password In the `Credentials` tab, make sure `Temporary` is switched on in order to set a temporary password for the user: ![](https://hackmd.io/_uploads/ry0mFl3Bc.png) LOC Studio informs the user to change the password upon first login. #### Step 4: Reset the Temporary Password If `Temporary` is switched on, the user will be asked to change the temporary password after logging in for the first time: ![](https://hackmd.io/_uploads/ByzCtg2r9.png) ![](https://hackmd.io/_uploads/B1nQ5e3S5.png) > LOC Studio will automatically log out the current user after **5 mins** of inactivity. ### Create a "Greeting" Data Process Now after logging in with your new user account, we are going to show you how to create this simple data process as below: 1. User sends a HTTP request to the data process with a JSON payload, which contains a name and an age. 2. The data process emits an event containing a greeting message with user data. ![](https://hackmd.io/_uploads/rkZuZm0rc.png) This data process has only one generic logic with one aggregator logic and will emit one event. After creating and deploying it, you will be able to see its action in LOC Studio. #### Step 1: Create Project/Scenario/Data Process In order to create a data process, you need to create a **project** and a **scenario** first. - create a project Under Default Unit, you can either right click that unit or click on the top right to create a new project, with the project name of "Quick Start Project". ![](https://hackmd.io/_uploads/HyMWF1485.png) - create a scenario Under Quick Start Project, you can either right click that project or click on the top right to create a new scenario, with the scenario name of "Quick Start Scenario". ![](https://hackmd.io/_uploads/rkubok48q.png) - create a data process Under Quick Start Scenario, you can either right click that scenario or click on the top right to create a new data process, with the data process name of "Quick Start" and the default timeout of 180 seconds. ![](https://hackmd.io/_uploads/HyfzikNI5.png) Detailed instructions can be found [HERE](##Data-Process-Explorer-Introduction). #### Step 2: Create Logics As soon as setting up the process explorer in the previous step, you can now create logics for this data process: ![](https://hackmd.io/_uploads/rkX0MRXU9.png) > We name the data process `Quick Start` for the following demonstration and leave the timeout to the default of 180 seconds. > > Every data process **must have** one aggregator logic and **at least one** generic logic. Just like what we have mentioned earlier, this data process has one generic logic and one aggregator logic. Each logic has two parts: - ```if OK``` function (main logic) - ```iF Error``` function (error handling) If something goes wrong in ```run()```, LOC Studio will run ```handleError()``` instead. We will not do anything other than print out some logging messages. After all, it is always ideal to have the error-handling aspect covered in LOC Studio. Now let's start coding. ##### Generic Logic Code In the editing window click ```Logic Body```. You will find ```if OK``` and ```iF Error``` tabs: ![](https://hackmd.io/_uploads/HkIiBMRHc.png) > We will not use ```Potential Labeling``` here - it will automatically detect events in your code *if* you use the event structure that it recognises. Our customised event will not appear here though. These tabs are corresponding to the main logic and error handling functions we have mentioned above. Copy and paste the following code: ++**Generic logic - [if OK] block**++ For now, the main supported language in LOC Studio is **JavaScript**. The ```if OK``` code has to be declared as an asynchronous ```run(ctx)``` function (```ctx``` is a context object provided by LOC): ```javascript /** * * The codes in 'run' are executed when no error occurrs in Generic Logic. * */ async function run(ctx) { // a function that transforms byte array to string const UTF8ArrToStr = (aBytes) => { let utf8decoder = new TextDecoder(); return utf8decoder.decode(new Uint8Array(aBytes)); } // read and parse JSON data from the request body const payload = JSON.parse(UTF8ArrToStr(ctx.payload.http.body)); // emit an event to event store ctx.agents.eventStore.emit([ { sourceDID: "LOC_Studio", // source DID targetDID: payload.name, // target DID will be user's name labelName: `Hello, how are you, ${payload.name}?`, // event label (greeting message) meta: `${payload.age}`, // include user's age in the meta field type: "default", // default group }, ]); } ``` For example, if the user sends the following JSON data ```json { "name": "Arthur", "age": 42 } ``` This logic would send the following event: ``` sourceDID: "LOC_Studio" targetDID: "Arthur" labelName: "Hello, how are you, Arthur?" meta: 42 type: "default" ``` You might also notice that we use ```ctx.agents.eventStore.emit``` to send an event (which will be stored in the event store with user's name and age from the JSON payload). ```ctx.agents``` are built-ins so you don't need to worry about importing them from libraries. ++**Generic logic - [if Error] block**++ The ```if Error``` is another function declared as ```handleError(ctx, error)```: ```javascript /** * * The codes in 'handleError' is executed when an error occurrs in Generic Logic, * or the CURRENT running Logic just gets an error. * */ async function handleError(ctx, error) { ctx.agents.logging.error(error.message); // log the error } ``` #### Aggregator Logic Code A data process *must* have one aggregator logic which is pretty much the same as any generic logics, except that an aggregator logic cannot emit/retrieve events. The aggregator logic is used to return the final result (usually also JSON data) after the data process is executed. ![](https://hackmd.io/_uploads/By_fDZ3Bq.png) ++**Aggregator logic - [if OK] block**++ It is the same as the generic logic with the main function of a ```run(ctx)```: ```javascript /** * * The codes in 'run' are executed when no error occurrs in Aggregator Logic. * */ async function run(ctx) { ctx.agents.result.finalize({ status: "ok", taskId: ctx.task.taskId, }); } ``` This time we use agent ```ctx.agents.result.finalize``` to send back a JavaScript object (which will be converted into JSON string). The result will be like this: ```json { "status": "ok", "taskId": [some task id] } ``` We will see the actual response later. Please note that we *are NOT* sending a standard HTTP response - the fields of the JSON response can be customised as whatever you like. ++**Aggregator logic - [if Error] block**++ The error handling code for the aggregator logic is the same as before (log the error): ```javascript /** * * The codes in 'handleError' is executed when an error occurrs in Generic Logic, * or the CURRENT running Logic just gets an error. * */ async function handleError(ctx, error) { ctx.agents.logging.error(error.message); // log the error } ``` #### Step 3: Deploy the Data Processes We have finished creating the data process in [Step 2](####Step-2:-Create-Logics). Now we can deploy it. Right click on the data process and select `Deploy Data Process`. This will initiate the deployment process. ![](https://hackmd.io/_uploads/BkSOF-hBc.png) #### Step 4: Testing the Data Process A data process can be triggered, normally via an API route. However, LOC Studio allows you to invoke it directly for testing. This is also called a *single data process execution*. > Later we will see how to deploy an API route on an API platform as well. After the data process is deployed, right click it again and select `Execute Data Process`. ![](https://hackmd.io/_uploads/BylsY-nH9.png) The screen would then ask you to upload a payload. What you need to do is to create a ```payload.json``` on your computer with the following content: ++**payload.json**++ ```json { "name": "Ian", "age": 30 } ``` Make sure the JSON data contains ```name``` and ```age``` fields so that the data process can respond properly. You can change *Ian* and *30* into another name or age you prefer. Upload your payload file and click ```Execute```: ![](https://hackmd.io/_uploads/rJkS5bhHq.png) If you see something like the result below, the data process is executed successfully: ![](https://hackmd.io/_uploads/rycr5bnrq.png) When clicking the ```JSON``` button with an eye icon, you'll find the JSON response returned by the aggregator logic like this: ```json { "status": "ok", "taskId": { "executionId": "YnHhUUoj4VhHIGj3ga_7Iw", "id": "rI8U8XLPpN8psMoq1d_INA" } } ``` #### Step 5: Invoke the Data Processes via an API Route ##### Create an API Route What we did in Step 4 was just a quick test. To trigger the data process properly, we need an API route so that we (or any code) can send HTTP requests to it. > The difference between the single data process execution and API routes is that an API route can trigger *multiple* data processes. > > Also please note that only the deployed data processes can be linked to API routes. Firstly, go to the ```API Route``` tab, create an API route folder, and then click ```Create API Route``` on the upper right corner: ![](https://hackmd.io/_uploads/ByIv9-2r9.png) We set our API route as follows: - API Route Name: ```Quickstart``` - HTTP Method: ```POST``` - HTTP Path: ```/YH/Quickstart``` - Request Mode: ```Sync``` - Response Content Type: ```JSON``` - Linked Data Processes: Add the data process ```Quick Start``` > The actual API path would be something like ```https://api.loc.xxx/YH/Quickstart```. Afterwards, click ```Create```. ![](https://hackmd.io/_uploads/BkIYcZhH5.png) ##### Test the Data Process with an API Route You can use an API client tool such as **[Postman](https://www.postman.com/)** or **[Insomnia](https://insomnia.rest/)** to request URL. We will use the same payload in [Step 4](####Step-4:-Testing-the-Data-Process) (but we can simply copy and paste it in the body this time): ++**payload**++ ```json { "name": "Ian", "age": 30 } ``` Select ```POST``` and paste the full API route (including server URL). Afterwards, paste the JSON payload below. > See: [Basics of API Testing Using Postman](https://www.geeksforgeeks.org/basics-of-api-testing-using-postman/) Click ```Send``` and if all go well, you can expect to see the result like this: ![](https://hackmd.io/_uploads/B153qWhBc.png) You can see the *complete* response from the data process - notice that the response in [Step 4](####Step-4:-Testing-the-Data-Process) is actually included under the ```data``` field. #### Step 6: Inspect the Data Process in Data Discovery There is one more thing you can do with the events - to inspect the *data lineage*, which is the graphical representation of events. Click ```Event``` tab under **Data Discovery** and find the event you just sent: ![](https://hackmd.io/_uploads/SJi6j-hS9.png) You can see the ```Label Name``` of the event indeed contains the name "Ian" from our JSON payload. Now 1. copy the ```Execution ID``` of your event. 2. click ```Add filter``` on top left. 3. select the ```Field``` as ```Execution ID``` and paste your ID in ```Value```. 4. Click ```Save```. 5. Click ```Event``` again, then click the slider bar ```data``` below ```Applied Filters```. ![](https://hackmd.io/_uploads/Bk_pBJVL9.png) Now let's switch the data discovery window to ```Graph``` mode and it shows the data lineage of the event: ![](https://hackmd.io/_uploads/BJlS8yNLc.png) > You can use your mouse scrolling button to zoom in/out and drag the DIDs around. If you can click on the event label and then click the small menu icon on the right, you can inspect its details. Here you can find the age data ```30``` is in the ```meta``` field. Also: - *LOC_Studio* is the **source DID** - *Ian* is the **target DID** - the arrow "Hello, how are you, Ian?" is this event's **labal name** ![](https://hackmd.io/_uploads/SJrq41l8c.png) > If you send multiple requests to the API route with the same JSON payload, you will see multiple events appear between the same source and target in Data Discovery. --- Congratulations! You have created your first LOC data process and invoke it via an API route. This has already covered many basics in LOC Studio. ## Getting Started - McDonald's Example Imagine when you walk in McDonald's to buy your meal, the staff will key in your order. Subsequently, the kitchen will start to prepare your meal and deliver to the front desk once it is completed. Today, we are going to use LOC Studio to record this transaction (and even other customers'). Most importantly, we would like to know if every order is successfully prepared through event sourcing on LOC Studio. ### Business Process 1: staff to key in orders 1-A. (generic logic) to set order data 1-B. (generic logic) to get order data and emit an event to record orders into the event store 1-C. (aggregator logic) to return a success/error message to tell whether the execution of this logic is successful ### Business Process 2: kitchen to prepare meal 2-A. (generic logic) to get events from the 1^st^ data process and to allocate meals to the kitchen staff to produce 2-B. (generic logic) to deliver order and emit an event to record who is responsible for which delivery into the event store 2-C. (aggregator logic) to return a success/error message to tell whether the execution of this logic is successful Below please find the example of the business process analysis graph. ![](https://hackmd.io/_uploads/rkwcN7BBq.png) #### Step 1: Create Project/Scenario/Data Process In order to create a data process, you need to create a **project** and a **scenario** first. - create a project Under Default Unit, you can either right click that unit or click on the top right to create a new project, with the project name of "Fast Food Project". ![](https://hackmd.io/_uploads/rkZ63JVI9.png) - create a scenario Under Fast Food Project, you can either right click that project or click on the top right to create a new scenario, with the scenario name of "Fast Food Scenario". ![](https://hackmd.io/_uploads/B1-Hpk4U5.png) - create a data process Under Fast Food Scenario, you can either right click that scenario or click on the top right to create a new data process, with the data process name of "Fast Food_1" and the default timeout of 180 seconds. ![](https://hackmd.io/_uploads/rkwmRJEI9.png) Detailed instructions can be found [HERE](##Data-Process-Explorer-Introduction). ### Step 2: Create Logic(s) As soon as the previous data process explorer has been set up, we can now create business/data logics. To begin with the 1^st^ business/data process, there will be 2 generic logics and 1 aggregator logic to set up. #### Generic Logic - getOrder: to get order data ![](https://hackmd.io/_uploads/BJp4Vh84c.png) (sample code - If OK) ```javascript const bigData=[ { "Order": "200", "French Fries": "1", "Hamburger": "1", "Fried Chicken": "1", "Chicken Nugget": "4", "Coke": "0", "Diet Coke": "0", "Sprite": "1", "Lemonade": "0", "Black Tea": "0", "Coffee": "0", "Salad": "1", "Ice Cream": "1 " }, { "Order": "201", "French Fries": "1", "Hamburger": "1", "Fried Chicken": "1", "Chicken Nugget": "0", "Coke": "1", "Diet Coke": "0", "Sprite": "0", "Lemonade": "0", "Black Tea": "1", "Coffee": "0", "Salad": "1", "Ice Cream": "0 " }, { "Order": "202", "French Fries": "0", "Hamburger": "1", "Fried Chicken": "1", "Chicken Nugget": "6", "Coke": "0", "Diet Coke": "0", "Sprite": "0", "Lemonade": "1", "Black Tea": "1", "Coffee": "0", "Salad": "1", "Ice Cream": "1 " }, { "Order": "203", "French Fries": "1", "Hamburger": "0", "Fried Chicken": "2", "Chicken Nugget": "10", "Coke": "1", "Diet Coke": "1", "Sprite": "0", "Lemonade": "0", "Black Tea": "0", "Coffee": "0", "Salad": "0", "Ice Cream": "1 " }, { "Order": "204", "French Fries": "1", "Hamburger": "1", "Fried Chicken": "0", "Chicken Nugget": "0", "Coke": "1", "Diet Coke": "1", "Sprite": "0", "Lemonade": "1", "Black Tea": "0", "Coffee": "1", "Salad": "0", "Ice Cream": "0 " }, { "Order": "205", "French Fries": "2", "Hamburger": "0", "Fried Chicken": "4", "Chicken Nugget": "20", "Coke": "0", "Diet Coke": "10", "Sprite": "0", "Lemonade": "0", "Black Tea": "0", "Coffee": "1", "Salad": "0", "Ice Cream": "2 " }, ] let newData =bigData.map((d)=>{ let template=[] for (let property in d){ if(property ==='Order') continue let templates = {}; templates.sourceDID=property; templates.targetDID=d.Order; templates.labelName='setOrder'; templates.meta=d[property]; templates.type='default'; template.push(templates); } return template }) let data_temp =newData.flat() async function run(ctx) { await ctx.agents.sessionStorage.putJson("data", data_temp) } ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` ![](https://hackmd.io/_uploads/BkmMBnLE5.png) ![](https://hackmd.io/_uploads/rkud7xHU5.png) - getOrder_Event: to emit an event to record orders into the event store (event structure) Source DID - Item:xxx Target DID - Order:xxx Label Name - setOrder Meta - quantity of each item (sample code - If OK) ```javascript async function run(ctx) { const data = await ctx.agents.sessionStorage.get("data") data.forEach((d)=>{ ctx.agents.eventStore.emit([ { sourceDID: `Item: ${d.sourceDID}`, targetDID: `Order: ${d.targetDID}`, labelName: `setOrder:${d.labelName}`, meta: `${d.meta}`, type: 'default', }, ]); })} ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` #### Aggregator Logic - orderData: to return taskID when the data process is successfully triggered and an error message when it fails (sample code - If OK) ```javascript async function run(ctx) { ctx.agents.result.finalize({ status: "ok", taskId: ctx.task.taskId, }); } ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` Moving to the 2^nd^ business/data process, there will be 2 generic logics and 1 aggregator logic to configure as well. #### Generic Logic - prepareOrder: to get events from the 1^st^ data process and allocate meals to the kitchen staff to produce (sample code - If OK) ```javascript async function run(ctx) { const order200To205 = []; const foods = {}; for (let i = 200; i < 206; i++) { const searchReq = { queries: [ { Match: { field: 'target_digital_identity', value: `Order: ${i}`, }, }, { Match: { field: 'label_name', value: 'setOrder:setOrder', }, }, ], excludes: [], filters: [], from: 0, size: 1000, sorts: [], }; const search = await ctx.agents.eventStore.search(searchReq); const events = search?.events; order200To205.push(events) } order200To205.forEach(order => { order.forEach(detail => { if(!foods.hasOwnProperty(detail.sourceDigitalIdentity)){ foods[detail.sourceDigitalIdentity] = detail.meta } else { let amount = +foods[detail.sourceDigitalIdentity]; amount += +detail.meta foods[detail.sourceDigitalIdentity] = amount } }) }) await ctx.agents.sessionStorage.putJson('foods', { foods, }); } ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` - prepareOrder_Event: to emit an event to record who is responsible for which delivery into the event store (event structure) Source DID - Staff Target DID - Item:xxx Label Name - deliverOrder Meta - quantity of each item (sample code - If OK) ```javascript async function run(ctx) { const foods = await ctx.agents.sessionStorage.get('foods'); const staff = ['John', 'Ann', 'Emily']; for(let food in foods.foods) { if (food === 'Item: French Fries' || food === 'Item: Fried Chicken' || food === 'Item: Chicken Nugget') { await ctx.agents.eventStore.emit([ { labelName: 'deliverOrder', sourceDID: staff[0], targetDID: food, meta: `${foods.foods[food]}`, type: 'default', }, ]); } else if (food === 'Item: Hamburger' || food === 'Item: Ice Cream') { await ctx.agents.eventStore.emit([ { labelName: 'deliverOrder', sourceDID: staff[1], targetDID: food, meta: `${foods.foods[food]}`, type: 'default', }, ]); } else if (food === 'Item: Coke' ||food === 'Item: Diet Coke' || food === 'Item: Sprite' || food === 'Item: Lemonade' || food === 'Item: Black Tea' || food === 'Item: Coffee') { await ctx.agents.eventStore.emit([ { labelName: 'deliverOrder', sourceDID: staff[2], targetDID: food, meta: `${foods.foods[food]}`, type: 'default', }, ]); } } } ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` #### Aggregator Logic - preparationData: to return taskID when the data process is successfully triggered and an error message when it fails (sample code - If OK) ```javascript async function run(ctx) { ctx.agents.result.finalize({ status: "ok", taskId: ctx.task.taskId, }); } ``` (sample code - If ERROR) ```javascript async function handleError(ctx, error) { ctx.agents.logging.error(error.message); } ``` ### Step 3: Deploy Data Processes In [Step 2](#Step-2:-Create-Logic(s)), we have completed creating 2 data processes with 2 generic logics and 1 aggregator logic for each. Once they are completed, you might want to deploy these 2 data processes respectively in order for the execution later. By right clicking on the specific data process, you are allowed to deploy it. ![](https://hackmd.io/_uploads/BymrNlB8c.png) :::info Please note that - Deploying a data process is equivalent to storing it in the backend of LOC Studio. - When a data process is deployed, this data process in the menu will be marked with a green solid circle. ::: ### Step 4: Perform Single Data Process Execution In LOC Studio, there are 2 ways to execute a data process,either through the single data process execution or the execution via API routes. In this step, we are using the single data process execution. After a data process is deployed, you can right click again on that deployed data process and select `Execute Data Process`. ![](https://hackmd.io/_uploads/S1N-HgS89.png) Because, in our case, we do not need to import any data source, we can skip this and click `Execution` to continue. ![](https://hackmd.io/_uploads/Hy1DBerUc.png) When the execution completes, you will have this pop-up execution result to check out some details. For instance, you can check or download your execution result (If OK / If ERROR) per your written code snippets in the [Aggregator Logic](#Aggregator-Logic). ![](https://hackmd.io/_uploads/SyFhBeHU5.png) Moreover, if there are any events created in the data process, you can switch to `Emitted Events` to see more details. It is worth mentioning that when clicking `Repository`, you will be led to a new window of Data Discovery where the events information and data lineage can be found. For detailed explanation and introduction of Data Discovery, please refer to [Step 6](#Step-6:-Inspect-in-Data-Discovery). ![](https://hackmd.io/_uploads/r1GJIeSI9.png) ### Step 5: Execute Data Processes with API Route Besides the single data process execution stated in [Step 4](#Step-4:-Perform-Single-Data-Process-Execution), here we are going to introduce another way of executing data processes via API routes. In the beginning, you have to create an API Route folder as usual. Upon the folder is built, you can create several API routes of your interest. In this case, you can choose to create 1 API route or 2. If you choose to create 2 API routes, each API route will be linked with 1 data process. Here we are going to demonstrate how to configure with merely 1 API route which will be linked with 2 data processes we created previously. ![](https://hackmd.io/_uploads/r1aDcIYVc.png) Please remember that you need to link this API route with our aforementioned data processes (McD_1 and McD_2). Additionally, in `HTTP Path`, you are required to fill in a sub-path only (that is you can skip the domain name). ![](https://hackmd.io/_uploads/SkcP8eHIc.png) After clicking `Create`, this API route is created successfully as shown below. ![](https://hackmd.io/_uploads/H1VgvgHLc.png) The next thing you need to do is to use an API platform. Here we are going to use Postman for demonstration. By entering the request URL (domain name + sub-path), it will return a result which is exactly the same as what we have tested out in [Step 4](#Step-4:-Perform-Single-Data-Process-Execution) - the response will be a successful message. ![](https://hackmd.io/_uploads/SJmfbVqV9.png) :::info Please note that if there is an error message, the reasons could be - the URL is being requested for too long, so you need to re-send the request on Postman again. - the data process execution time is longer than your previous setup, so please change the setting. ([Reminder in Data Process](#Create-a-Data-Process)) - the code snippets in your (generic or aggregator) logics might be erroneous, so please revise them and re-deployed. Here are some of the common error messages you might encounter only for your reference. ::: ### Step 6: Inspect in Data Discovery After requesting the URL in the Postman successfully, you will also get a set of execution ID which can be put into [Data Discovery](#Data-Discovery-Introduction) to search events. :::info Please note that only when there are events written in the logics, can they be searched here. ::: After putting the execution ID into the filter, you might get this graph. ![](https://hackmd.io/_uploads/H19sToo4c.png) It showcases the relationship of the entire use case, from what food in each order to who prepares for what food. This is exactly our desired result. From this graph, not just the relationship is clearly specified, but also we could see if there is any food that has not been done, such as fried chicken, lemonade, etc. It is useful for event sourcing. On top of this use case, you might also want to try out the subsequent events and business/data processes or to refine them. For example, you can take prices into account and plan some more for billing.