Agent List

This API document is designed for JavaScript developers using LOC Studio and/or LOC CLI.

LOC provides a range of agents (built-in JavaScript drivers) for data processes so that you can access internal session storage data, events or external APIs, databases at run-time.

• Generic Logic
  • HTTP Request and Response
  • Session Storage
  • Event Store
  • Database
  • File Storage
  • SMTP
  • Local Storage
  • Error Logging
• Aggregator Logic
  • Result
  • Session Storage
  • Error Logging

Be noted that most LOC agents are asynchronous, so it is recommended to use await to tackle them properly.

ctx and ctx.task

ctx is the context object injected by LOC into generic/aggregator logics at run-time.

You can access task-related information from ctx.task:

Attributes	Type
currentLogic	{ permanentIdentity: name; name: string; revision: number }
dataProcess	{ permanentIdentity: name; name: string; revision: number }
executedLogics	{ permanentIdentity: name; name: string; revision: number }[]
startAt	string (Date)
taskId	{ executionId: string; id: string }

Example:

const taskId = ctx.task.taskId.id;
const executionId = ctx.task.taskId.executionId;

Generic Logic

Parse Input

ctx.payload.http

ctx.payload.http has the following attributes:

Attributes	Type	Example
apiGatewayIdentityContext	{ id: string; name: string }
apiIdentityContext	{ id: string; name: string }
body	number[]
headers	{ [name: string]: unknown; }	{"content-type":"application/json"}
host	string
method	string	"POST"
path	string	/hello/greeting
query	string	?param=data...
requestId	string
scheme	string	https
version	string	"HTTP/1.1"

Parse POST JSON Payload

You can read the request body from API routes like this:

// a helper function to convert byte array to string
const UTF8ArrToStr = (aBytes) => {
    const utf8decoder = new TextDecoder();
    return utf8decoder.decode(new Uint8Array(aBytes));
}

// read request body and header
const body = JSON.parse(UTF8ArrToStr(ctx.payload.http.body));
const headers = ctx.payload.http.headers;

For instance, if you have this JSON input that is going to be inserted into the request body,

{
    // JSON body
    "name": "Arthur",
    "age": 42
}

After using the code snippets mentioned above, you can access them like this:

// a helper function to convert byte array to string
const UTF8ArrToStr = (aBytes) => {
    const utf8decoder = new TextDecoder();
    return utf8decoder.decode(new Uint8Array(aBytes));
}

const body = JSON.parse(UTF8ArrToStr(ctx.payload.http.body));

let name = body?.name;  // extract name from JSON body
let age = body?.age;  // extract age from JSON body

The JSON input will be stored in a constant variable body after you invoke the API route with that request body. Hence, whilst querying body.name and body.age, you can expect the returns to be "Arthur" as a string and 42 as a number, respectively.

Parse GET QueryString

For example, if you send a GET request with the following QueryString:

https://api.fst.xxx/<api_route>?name=Arthur&age=42

You can extract the parameters like this:

const query = ctx.payload.http.query;

const searchParams = new URLSearchParams(query);

let name = searchParams.get("name");  // extract name from QueryString
let age = searchParams.get("age");  // extract age from QueryString

HTTP Agent

ctx.agents.http.get(url, headers, contentType, body, config=null)

Send a HTTP GET request.

Parameters	Type	Description
url	string	URL
headers	{ header1: value1; … }	HTTP Headers
contentType	string	"None", "PlantText", "Json" or "Form"
body	Uint8Array or null	Request body (optional only for get())
config (optional)	object	{ acceptInvalidCerts: true/false }

There are a series of agents corresponding to HTTP methods (the parameters are exactly the same):

Agents	HTTP Methods
ctx.agents.http.get()	GET
ctx.agents.http.post()	POST
ctx.agents.http.patch()	PATCH
ctx.agents.http.put()	PUT
ctx.agents.http.delete()	DELETE

All of the HTTP agents will return a Http.Response object with the following attributes:

Attributes	Type	Description
status	number	HTTP status code
headers	{ header1: value1; … }	Response headers
body	Uint8Array	Response body

Example:

const data = {
    key1: "data1",
    key2: "data2",
    ...
}

// send a HTTP POST request and get the response
const response = await ctx.agents.http.post(
    "https://myserver/myservice",  // URL
    {},  // no headers needed to be set
    "Json",  // content type
    new TextEncoder().encode(JSON.stringify(data))  // body
);

// assuming the service return JSON as well;
// this is as same as using UTF8ArrToStr in the previous section
const body = JSON.parse(new TextDecoder().decode(response.body));

if (body.status === 200) {  // if success
    ctx.agents.logging.info(body?.name);  // extract name from JSON
    // ...
}

With the code snippets above, you are allowed to store the data from HTTP into a constant variable body for future use.

See HTTP Agent Request Examples for more code examples.

Session Storage

Read and write temporary data between logics during the same execution task of a data process.

Please note that the data kept via this way (session storage) can only be accessed by the same data process. The data would also be cleaned up once the data process finishes the task.

If you want to keep data between tasks, use local storage instead.

If you feel like exchanging the data between other data processes, we suggest you use event emission to keep the data and event search to query the events.

ctx.agents.sessionStorage.putJson(key, value)

Put JSON data in the session storage.

Parameters	Type	Description
key	string	key of session data
value	any	JSON object

Example:

const data = [
    {
        field1: "field1",
        field2: "field2",
        // ...
    },
    // ...
];

// store the data in session storage in JSON format
await ctx.agents.sessionStorage.putJson("mydata", data);

ctx.agents.sessionStorage.putString(key, value)

Put a string in the session storage.

Parameters	Type	Description
key	string	key of session data
value	string	string data

Example:

const name = "Author";
const age = 42;

// store the data in session storage as string
await ctx.agents.sessionStorage.putString("name", name);
await ctx.agents.sessionStorage.putString("age", String(age));

ctx.agents.sessionStorage.putByteArray(key, value)

Put a byte array in the session storage.

Parameters	Type	Description
key	string	key of session data
value	Uint8Array	byte array

ctx.agents.sessionStorage.get(key)

Read data by key from the session storage.

Parameter	Type	Description
key	string	key of session data

Example:

const data = await ctx.agents.sessionStorage.get("mydata");

data will be the same type as what you put in the session storage.

ctx.agents.sessionStorage.remove(key)

Remove (delete) a data from session storage.

Parameter	Type	Description
key	string	Data key

Event Store

Send and query events between data processes.

See concepts to know more about LOC events.

Also, you CANNOT use event agents in Aggregator logics. Instead, you can only use them in generic logics.

ctx.agents.eventStore.emit(events)

Emit event(s) into the event store.

Parameter	Type	Description
events	object array	events

Example:

const events = [
    // event 1
    {
        sourceDID: "source name xxx",
        targetDID: "target name xxx",
        labelName: "event name xxx",
        meta: "",  // data payload
        type: "default",  // event group
    },
    // event 2
    {
        // ...
    }
];

await ctx.agents.eventStore.emit(events);

The maximum length of meta is 2¹⁵ bytes. You can convert a JSON object to a string using JSON.stringify() and store it in meta:

meta: JSON.stringify(data),

ctx.agents.eventStore.search(request)

Search and read event(s) from the event store.

Parameter	Type	Description
request	object array	request parameters

Example:

// a helper function to create a match condition
const createMatch = (field, value) => {
    return { Match: { field, value } };
};

// search for events that event name equals to "event name"
const requests = {
    // you can search multiple fields
    queries: [
        createMatch(
            "label_name", 
            "event name xxx"
        ),
        createMatch(
            "source_digital_identity", 
            "Source DID xxx"
        ),
        // ...
    ],
    excludes: [],
    filters: [],
    from: 0,
    size: 1000,
    sorts: [],
};

const query = await ctx.agents.eventStore.search(requests);
const events = query?.events

// iterate through events
events.forEach(event => {
    ctx.agents.logging.info(event.executionId);
    ctx.agents.logging.info(event.label.name);
    ctx.agents.logging.info(event.meta);
    ...
});

The query field name here is different with the ones you have used in ctx.agents.eventStore.emit(). See Event Field Comparison Table for reference.

events is an array of Event objects. Each Event has the following attributes:

Attributes	Type	Description
idataProcessIdentityContext	{ name: string; permanentIdentity: string; revision: number; }
executionId	string	Execution ID
label	{ id: number; name: string }	Event ID and name
logicIdentityContext	{ id: number; name: string }
meta	string	Meta data
sequence	number	Sequence Number
sourceDigitalIdentity	string	Source DID
targetDigitalIdentity	string	Target DID
taskId	string	Task ID
timestamp	string
type	string	Event group

ctx.agents.eventStore.searchWithPattern(request)

Search sequences of events, for which 1^st event has to fit 1^st condition, 2^nd event has to fit 2^nd condition…so on.

Parameter	Type	Description
request	object array	request parameters

Example:

// a helper function for creating a search condition
const createCondition = (field, value, type) => {
    const operators = ["Eq", "NotEq"];
    if (!operators.includes(type)) type = "Eq"; // default operator: Eq (equal)
    return { [type]: { field, value } };
}

// create sequence search pattern
const requests = {
    sequences: [
        // sequence 1 event
        {
            conditions: [
                // label name equals "event name xxx"
                createCondition(
                    "label_name",
                    "event name xxx",
                    "Eq"
                ),
            ],
            sharedFields: [],
            type: "any",
        },
        // sequence 2 event
        {
            conditions: [
                // source DID equals "source DID xxx"
                createCondition(
                    "source_digital_identity",
                    "source DID xxx",
                    "Eq"
                ),
                // and target DID do not equal "target DID xxx"
                createCondition(
                    "target_digital_identity",
                    "target DID xxx",
                    "NotEq"
                ),
            ],
            sharedFields: [],
            type: "any",
        },
    ],
}

const query = await ctx.agents.eventStore.searchWithPattern(requests);
const sequences = query?.sequences;

// iterate through sequences
sequences.forEach(sequence => {
    
    // each sequence has multiple events
    sequence.events?.forEach(event => {
        
        ctx.agents.logging.info(event.taskId);
        ctx.agents.logging.info(event.label.name);
        ctx.agents.logging.info(event.meta);
        // ...
        
    });
});

Since we are search sequences of events, you MUST provide at least two sequence patterns (each pattern contains at least one condition).

The events above is an array similar to the one in the ctx.agents.eventStore.search() example.

Search Pattern Operators

Operators	Description
"Eq"	equal to (==)
"NotEq"	not equal to (!=)

Database

ctx.agents.database.connect({databaseDriver, connection=null})

Create a database object.

Parameters	Type	Description
databaseDriver	string	Name of the driver. See DB Driver List
connection	object	DB Connection parameters

MySQL Example (click to see more)

// import database definitions
const database = Saffron.Database;

// MySQL database connection parameters
const connectionInfo = {
    host: "127.0.0.1",
    port: 3306,
    username: "admin",
    password: "xxxx",
    database: "myDB",
};

// connect to database
const db = await ctx.agents.database.connect({
    databaseDriver: "MySQL",
    connection: new database.MySqlParameters(connectionInfo),
});

// querying db...

// disconnect when done
await db?.disconnect();

MSSQL Example (click to see more)

// import database definitions
const database = Saffron.Database;

// MySQL database connection parameters
const connectionInfo = {
    host: "127.0.0.1",
    port: 3306,
    username: "admin",
    password: "xxxx",
    database: "myDB",
    trustCert: true,
};

// connect to database
const db = await ctx.agents.database.connect({
    databaseDriver: "MSSQL",
    connection: new database.MssqlParameters(connectionInfo),
});

// querying db
let sql = `SELECT * FROM {table name}`;
let resp = await db?.query(sql, []);
await ctx.agents.sessionStorage.putJson("resp", resp);

// disconnect when done
await db?.disconnect();

For more examples of connecting with other databases, please refer to Local Database Testing with Simple Runtime.

DB Driver List (database.connect() Parameters)

LOC currently provides the following built-in database drivers:

Database	databaseDriver	connection
MySQL	"MySQL" or database.Driver.MySql	new database.MySqlParameters (connectionInfo)
PostgresSQL	"Postgres" or database.Driver.Postgres	new database.PostgresParameters(connectionInfo)
MS SQL Server	"MSSQL" or database.Driver.Mssql	new database.MssqlParameters(connectionInfo)

connectionInfo Parameters

Parameter	Type	DB
host	string	All (default: lodalhost)
port	number	All (default: MySQL - 3306, Postgres - 1521, MSSQL - 1433)
database	string	All
username	string	All
password	string	All
trustCert (optional)	boolean	MSSQL
options (optional)	string	Postgres
connectTimeout (optional)	number	Postgres
keepalives (optional)	boolean	Postgres
keepalivesIdle (optional)	number	Postgres

Please be noted that database.connect() will check the connection object. If you use an connection object which contains incorrect or unsupported parameters, an "missing connection information" error will be thrown.

db.query(rawSql, params)

Execute a SQL query.

Parameter	Type	Description
rawSql	string	SQL prepared statement
params	array	SQL query parameters

Example:

// query with SQL prepared statement (MSSQL)
const resp = await db.query(
    "SELECT * FROM table1 WHERE col_1 = ?;",
    [12345]
);  // "?" will be replaced with "12345"

SQL prepared statement separates parameters from the SQL statement in order to stop SQL injection. Please be noted that SQL syntax in prepared statement may differ on different databases.

db.query() returns a Database.QueryResults objcet with the following attributes:

Attributes	Type	Description
columns	{ name: string; type: string }[]	Selected columns name and data type
rows	{ key1: any; key2 : any; … }[]	Selected rows

db.execute(rawSql, params)

Execute a SQL statement.

Example:

// insert a new row into table with SQL prepared statement
await db.execute(
    "INSERT INTO table1 (id, col_1...) VALUES (?, ?...);",
    [12345, 'value 1'...]
);

db.disconnect()

Disconnect the database.

db.beginTransaction()

db.commitTransaction()

db.rollbackTransaction()

Execute/rollback a SQL statement in a transaction.

These transaction functions are based on Transact-SQL thus can only be used for MSSQL.

Example:

try {
    
    await db.beginTransaction();  // begin transaction

    // update table with SQL prepared statement
    await db.execute(
        "UPDATE table1 SET status = ? WHERE id = ?;",
        ['Accepted', 12345]
    );
    
    // ...other SQL actions

    await db.commitTransaction();  // commit transcation

} catch (err) {
    // rollback transaction if there are error
    await db.rollbackTransaction()  
}

File Storage

Access a remote file via HTTP, FTP (File Transfer Protocol) or SMB (Server Message Block) protocols.

Note: the file storage agent does not invoke SMB protocol directly. Instead, you need to provide the connection info of your SMB service (including Amazon S3) to the FST Network team. These info will be configured in your Kubernetes environment. The LOC core will handle the login and serves as a file broker between data processes and the SMB server.

A S3 URL looks like this:

s3://<access-key-id>:<secret-access-key>@<bucket>/<key>?region=<region>&endpoint-url=<endpoint>`

Where <secret-access-key> should be percent-encoding.

For more information, please contact FST Network.

ctx.agents.fileStorage.simpleGet(url)

Read a remote file. (For HTTP it's a GET request.)

Parameter	Type	Description
url	string	HTTP/FTP/SMB URL

The returned value is a Uint8Array.

Example:

// a FTP url
const url = "ftp://myuser:mypass@<host>/<dir>/test_file.txt";
const receivedData = await ctx.agents.fileStorage.simpleGet(url);

// decode to unicode string
const data = new TextDecoder().decode(receivedData);

ctx.agents.fileStorage.simplePut(url, data)

Write data into a remote file. (For HTTP it's a PUT request.)

Parameters	Type	Description
url	string	HTTP/FTP/SMB URL
data	Uint8Array	Content to be written into the file

Example:

const url = "ftp://myuser:mypass@<host>/<dir>/test_file.txt";
const data = "...";

// encode data to Uint8Array
const encodedData = new TextEncoder().encode(data);

await ctx.agents.fileStorage.simplePut(url, encodedData);

ctx.agents.fileStorage.delete(url)

Delete a remote file. (For HTTP it's a DELETE request.)

Parameter	Type	Description
url	string	HTTP/FTP/SMB URL

ctx.agents.fileStorage.list(url)

List all files under a directory. (Not available for HTTP file requests.)

Parameter	Type	Description
url	string	FTP/SMB URL

Returns Array<FileStorage.FileType>. A FileType has the following attributes:

Attributes	Type	Description
type	string	"file", "directory" or "symbolicLink"
file	string	name of the file/dir/symbolic link

Example:

const url = "ftp://myuser:mypass@<host>/<dir>/test_file.txt";
const fileList = await ctx.agents.fileStorage.list(url);

// iterate through all items in the list
fileList.forEach((item) => {
    ctx.agents.logging.info(`${item.name}: ${item.type}`);
});

ctx.agents.fileStorage.createDirAll(url)

Create a new directory at specified location.

Parameter	Type	Description
url	string	FTP/SMB URL

SMTP

Send a email via a SMTP server.

ctx.agents.smtp.connect(host, username, password)

Connect to a SMTP server and returns a smtp object.

Parameter	Type	Description
host	string	SMTP host URL
username	string	SMTP username
password	string	SMTP password

Example:

const host = "smtp.xxx.com";
const usermail = "[e-mail]";
const password = "xxxxx";

const smtp = await ctx.agents.smtp.connect(host, usermail, password);

smtp.send(mail)

Send a email with the SMTP object.

const mail_body = `U get my mail???
Reply asap pls`;

// create a new mail object
let mail = new Saffron.Smtp.Mail();

// required fields
mail.setSender(usermail, "user name");  // mail sender (your email and name)
mail.setReceivers("[e-mail]", "receiver 1 name");  // mail receiver(s)
mail.setReceivers("[e-mail]", "receiver 2 name");  // you can add multiple receivers
mail.setSubject("Test mail");  // mail title
mail.setBody(mail_body);   // mail body

// optional fields
mail.setReplyTo("[e-mail]", "reply name");  // reply mail address
mail.setCC("[e-mail]", "cc name");  // cc
mail.setBCC("[e-mail]", "bcc name");  // blind cc

// send the mail
await smtp.send(mail);

For setSender(), setReceivers() and similar methods, the second name parameter can be omitted:

mail.setSender(usermail);
mail.setReceivers("[e-mail]");
// ...

Local Storage

Local Storage is very similar to session storage. The difference is that the data can be persistent between tasks of the same data process (session data will be purged after each task).

Just like session storage, a local storage data saved by one data process cannot be accessed by other data processes. The data would also be purged if you re-deploy the data process.

Local Storage can ONLY be used in generic logics (not available in aggregator logics).

ctx.agents.localStorage.putJson(key, value, timeout)

Write a JSON data into local storage.

Parameters	Type	Description
key	string	Data key
value	object	JSON data
timeout (optional)	number	Time limit (seconds) for persisting the data. Default: null (unlimited)

Example:

const data = {
        field1: "field1",
        field2: "field2",
        // ...
    };

await ctx.agents.localStorage.putJson("mydata", data);

ctx.agents.localStorage.putString(key, value, timeout)

Write a string into local storage.

Parameters	Type	Description
key	string	Data key
value	string	String
timeout (optional)	number	Time limit (seconds) for persisting the data. Default: null (unlimited)

ctx.agents.localStorage.putByteArray(key, value, timeout)

Write a byte array into local storage.

Parameters	Type	Description
key	string	Data key
value	Uint8Array	Byte array
timeout (optional)	number	Time limit (seconds) for persisting the data. Default: null (unlimited)

ctx.agents.localStorage.get(key)

Read a data from local storage.

Parameter	Type	Description
key	string	Data key

Example:

const data = await ctx.agents.localStorage.get("mydata");

ctx.agents.localStorage.remove(key)

Remove (delete) a data from local storage.

Parameter	Type	Description
key	string	Data key

Error Logging

ctx.agents.logging.error(value)

ctx.agents.logging.info(value)

Log a error or general message:

Parameter	Type	Description
value	string of object	Log message

The message can either be a string or a JSON object.

Example:

ctx.agents.logging.error("error message");

ctx.agents.logging.error({
    errorMessage: "error message",
});

Be warned that ctx.agents.logging cannot parse non-JSON objects and will throw error. You can convert the object to string first:

ctx.agents.logging.error(JSON.stringify(some_object);

Log level

Generally error() (for reporting errors) and info() (for general messages) are enough for most situations. But you can also use warn() for something that is not an error but still requires attention:

Agent	Log level/severity
ctx.agents.logging.error(value)	Highest
ctx.agents.logging.warn(value)
ctx.agents.logging.info(value)	Lowest

Log outputs can only be seen from the LOC Kubernetes environment or via Local Simple Runtime Execution.

Aggregator Logic

Result

ctx.agents.result.finalize(value)

Write a customised result of the data process task, and the API route will aggregate the task results of the execution this time.

Parameter	Type	Description
value	object	string or JSON object

Example:

ctx.agents.result.finalize({
    status: "ok",  // this is NOT the HTTP status code but a custom field
    taskId: ctx.task.taskId,
    message: "xxx",
    // ...
});

The result would be included as part of the actual API response:

{
    "_status":200,
    "_metadata":{
        "executionId":"Ynth0FrP6mbq21bfUahtLg",
        "status":"success",
        "expiration":"2022-05-14T07:12:16.729755958Z"
    },
    "data":{  // the results outputed by finalize()
        "status":"ok",
        "taskId":{
            "executionId":"Ynth0FrP6mbq21bfUahtLg",
            "id":"_3xVQoFQ7l5dCW03vbPZOw"
        },
        "message": "xxx",
        // ...
    }
}

The data process would always return "_status":200 even though there are errors occurred.

If you see "_status":202, it may be that the data process/API route are just deployed and are not ready yet. If you see this repeatly, it may be something went wrong in the LOC itself so the data process can never finish its task.

Session Storage

See: Session Storage

Error Logging

See: Error Logging

---

tags: LOC Studio LOC CLI