owned this note
owned this note
Published
Linked with GitHub
# Retail OS
To star the Retail OS, we need User, Product Catalog in our repository then only we can process any DAG.
1. Recommendation
2. Forecasting
3. Inventory Optimisation
### Step 0: Prerequisite
##### Entity
1. SKU
3. Article
3. Merchant
4. Warehouse
5. Customer
6. Segment
7. Brand
8. Building
9. Company
10. Location
/entity/common/grid_{6, 7, 8, 9}
/entity/common/pin
/entity/common/city
/entity/common/locality
/entity/common/district
##### Recommendation DAG (till feature repository)
#### Features for recommendation DAG
1. /feature/retail/brand_customer/quantity_sum/time_window_type_week
2. /feature/retail/brand_customer/value_sum/time_window_type_week
3. /feature/retail/category_customer/quantity_sum/time_window_type_week
4. /feature/retail/category_customer/value_sum/time_window_type_week
5. /feature/retail/sku_customer/quantity_sum/time_window_type_week
6. /feature/retail/sku_customer/value_sum/time_window_type_week
7. /feature/retail/brand_customer/z_quantity_sum/time_window_type_week
8. /feature/retail/brand_customer/z_value_sum/time_window_type_week
9. /feature/retail/category_customer/z_quantity_sum/time_window_type_week
10. /feature/retail/category_customer/z_value_sum/time_window_type_week
11. /feature/retail/sku_customer/z_quantity_sum/time_window_type_week
12. /feature/retail/sku_customer/z_value_sum/time_window_type_week
13. /feature/retail/sku_customer/quantity_sum/time_window_type_month
14. . /feature/retail/sku_customer/value_sum/time_window_type_month
15. /feature/retail/brand_customer/quantity_sum/time_window_type_month
16. /feature/retail/brand_customer/value_sum/time_window_type_month
17. /feature/retail/category_customer/quantity_sum/time_window_type_month
18. /feature/retail/category_customer/value_sum/time_window_type_month
19. /feature/retail/brand_customer/z_quantity_sum/time_window_type_month
20. /feature/retail/brand_customer/z_value_sum/time_window_type_month
21. /feature/retail/category_customer/z_quantity_sum/time_window_type_month
22. /feature/retail/category_customer/z_value_sum/time_window_type_month
23. /feature/retail/sku_customer/z_quantity_sum/time_window_type_month
24. /feature/retail/sku_customer/z_value_sum/time_window_type_month
#### Events Needed
1. Add2Cart
2. Purchase (delete to invetory)
3. InventoryUpdate/Replenish (add to invetory)
4. HomePageView
6. Category PageView
6. Search
### Step 1:
we can use this link https://cloud.google.com/retail/docs/reference/rest/v2/projects.locations.catalogs.userEvents as reference to create the event protos.
```
add-to-cart: Products being added to cart.
category-page-view: Special pages such as sale or promotion pages viewed.
detail-page-view: Products detail page viewed.
home-page-view: Homepage viewed.
promotion-offered: Promotion is offered to a user.
promotion-not-offered: Promotion is not offered to a user.
purchase-complete: User finishing a purchase.
search: Product search.
shopping-cart-page-view: User viewing a shopping cart.
enum RetailEventType {
RETAIL_ADD_TO_CART_EVENT = 0;
RETAIL_HOME_PAGE_VIEW_EVENT = 1;
RETAIL_SEARCH_EVENT = 2;
...
}
// TxnEvent
message RetailOrderEvent {
message RetailAdd2CartEvent add_to_cart_event = 1;
message RetailPDPEvent pdp_event = 2;
}
{
"eventType": string,
"visitorId": string,
"sessionId": string,
"eventTime": string,
"experimentIds": [
string
],
"attributionToken": string,
"productDetails": [
{
object (ProductDetail)
}
],
"completionDetail": {
object (CompletionDetail)
},
"attributes": {
string: {
"text": [
string
],
"numbers": [
number
],
"searchable": boolean,
"indexable": boolean
},
...
},
"cartId": string,
"purchaseTransaction": {
object (PurchaseTransaction)
},
"searchQuery": string,
"filter": string,
"orderBy": string,
"offset": integer,
"pageCategories": [
string
],
"userInfo": {
object (UserInfo)
},
"uri": string,
"referrerUri": string,
"pageViewId": string,
"entity": string
}
```
### Step 2:
```
message Item2ItemRecommendationRequest {
uint32 item_type_id = 1; // optional
uint64 item_id = 2;
int recommendation_count = 3;
}
message Item2ItemRecommendationResponse {
repeated ItemScorePair item_score_pair = 1;
}
message ItemScorePair {
uint64 item_id = 1;
float score = 2;
}
message UserPersonalizedRecommendationRequest {
unit32 user_type_id = 1; // optional
unit64 user_id = 2;
UserProfile profile = 3;
int recommendation_count = 4;
}
message UserProfile {
unit32 user_type_id = 1; // optional
unit64 user_id = 2;
repeated UserProfileDataPoint profile_data = 3;
}
message UserProfileDataPoint {
uint32 item_type_id = 1; // optional
uint64 item_id = 2;
RetailEventType event_type = 3;
Timestamp event_time = 4;
}
message UserPersonalizedRecommendationResponse {
repeated ItemScorePair item_score_pair = 1;
}
```
### Sample purchase event
```
{
"PurchaseId": "118052700",
"AddedDate": "2023-01-22 08:00:39.001",
"StoreId": "4114845",
"CustomerId": "10980910",
"InvoiceAmount": "328137.3",
"line-items": [
{
"ProductSKUId": "10151184",
"Quantity": "65",
"SalePrice": "2241",
"TotalAmount": "145665"
},
{
"ProductSKUId": "10173686",
"Quantity": "28",
"SalePrice": "1911.6",
"TotalAmount": "53524.8"
},
{
"ProductSKUId": "10124315",
"Quantity": "55",
"SalePrice": "1243.8",
"TotalAmount": "68409"
},
{
"ProductSKUId": "10037121",
"Quantity": "55",
"SalePrice": "1100.7",
"TotalAmount": "60538.5"
},
{
"ProductSKUId": "10151184",
"Quantity": "65",
"SalePrice": "2241",
"TotalAmount": "145665"
},
{
"ProductSKUId": "10173686",
"Quantity": "28",
"SalePrice": "1911.6",
"TotalAmount": "53524.8"
},
{
"ProductSKUId": "10124315",
"Quantity": "55",
"SalePrice": "1243.8",
"TotalAmount": "68409"
},
{
"ProductSKUId": "10037121",
"Quantity": "55",
"SalePrice": "1100.7",
"TotalAmount": "60538.5"
}
]
}
```
*Request parameters*
| Request Parameter | Description | Valid Values | Default Value
|-------------|-------------------|-----------------|-----------------
| itemId | Seed item Id | string
| recommendationCount | Number of recommended items to return | 1-100 | 10
> **Sample Request:**
>```
>GET https://<service_name>.azurewebsites.net/api/models/e16198c0-3a72-4f4d-b8ab-e4c07c9bccdb/recommend?itemId=70322
>x-api-key: your_api_key
>```
> **Sample Response:**
>
>```
>HTTP/1.1 200 OK
>Content-Type: application/json; charset=utf-8
>
>[{
> "recommendedItemId": "46846",
> "score": 0.45787626504898071
>},
>{
> "recommendedItemId": "46845",
> "score": 0.12505614757537842
>},
>...
>{
> "recommendedItemId": "41607",
> "score": 0.049780447036027908
>}]
>```
## Get Personalized Recommendations
*POST /api/models/\{modelId\}/recommend*
Gets personalized recommendations for the model specified, given a set of recent usage events for a particular user.
An optional user id may also be specified, in which case the usage events of that user, extracted during model training from the input usage files, will also be considered.
> Note: If a user id is provided, additional usage events may still be provided in the request body, representing a more recent user activity
*Request parameters*
| Request Parameter | Description | Valid Values | Default Value
|---------------------|----------------------------------------------|--------------|-----------------
| userId | The id of a user to get recommendations for. The user id will be ignored unless the model was trained with a *enableUserToItemRecommendations* set to **true**.<br/>See [Model Training Parameters Schema](#model-training-parameters-schema) for more info. | string | null
| recommendationCount | Number of recommended items to return | 1-100 | 10
The request body must be an array of [Get Recommendations Usage Events](#get-recommendations-usage-event).
The response body will be a JSON array of [Recommendation Objects](#recommendation-object-schema).
> **Sample Request:**
>```
>POST https://<service_name>.azurewebsites.net/api/models/e16198c0-3a72-4f4d-b8ab-e4c07c9bccdb/recommend?userId=user032669023
>x-api-key: your_api_key
>
>[
> {
> "itemId": "ItemId123",
> "eventType": "Click",
> "timestamp": "2017-01-31T23:59:59"
> },
> {
> "itemId": "ItemId456",
> "eventType": "Purchase"
> },
> {
> "itemId": "ItemId789",
> "weight": 2.3
> },
> {
> "itemId": "ItemId135"
> }
>]
>```
> **Sample Response:**
>
>```
>HTTP/1.1 200 OK
>Content-Type: application/json; charset=utf-8
>
>[{
> "recommendedItemId": "46846",
> "score": 0.45787626504898071
>},
>{
> "recommendedItemId": "46845",
> "score": 0.12505614757537842
>},
>...
>{
> "recommendedItemId": "41607",
> "score": 0.049780447036027908
>}]
>```
https://github.com/Microsoft/Product-Recommendations/blob/master/doc/api-reference.md
(TxnEvent)
Order >> Qunantize >> Enrich >> event
-> f1 //customer_brand_profile (pivot + aggregate) -- > FeatureStore
-> f2 //customer_category_profile -- > FeatureStore
-> f3 //
>> recommendation engine DAG batch trigger, user profile change, customer-purchase, profile change --event --> inference (recommendation model deployed and available for inferencing)
recommendation model available : train process
DAG : profile based similar product
recommendation engine DAG >> Inference Store (userID, product, week)
Search Engine > Query "Atta"
customer ID : 1
Query : Atta
PersonalizationService (Telos Process) << Inference Store + Feature
request : 1, Atta
```
key : user_id
value :
f1:
f2:
f3:
```
# Retail flow involving ML/AI models
## 1. Profile recommendation
### API details
even
#### Recipe
model_name :
#### Fetch : Rajinder
introduce the window concept, make customer X (brand, category, sku) X {WINDOW} = {value, quantity=volume}
```
/feature/retail/customer/preference/category/value
/feature/retail/customer/preference/category/volume
/feature/retail/customer/preference/sku/value
/feature/retail/customer/preference/sku/volume
/feature/retail/customer/preference/brand/value
/feature/retail/customer/preference/brand/volume
```
**Service Name : FetchProcess (Java) take queue based input and produces traning.csv**
## fetch - flow :
1. will be consuming an event which will be having the model related info i.e
{
modelDesignSchemaId : 1234
}
2. Using the above modelDesignSchemaId the schema will be fetched from redis cache
3. Spring batch job will be initiated for the fetched model.
Batch job tasks:
a. Using row conditions fetch the record keys
b. using the record key and column condition combination fetch the features and attributes.
c. using the pre-processor condition calculate the derived feature and will generate the final training csv.
4. The generated csv will be uploaded to the blob storage. /file name formate Training_modelId_timestamp.csv
5.In the last step an event will be generated on queue where python lib is listening having details about model id and uploaded file i.e
{
modelDesignSchemaId : 1234
filePath:'uploaeded/file/name' //Training_modelId_timestamp.csv
}
There should be 6 CSVs, this needs to be translated to FetchCritiera Config (TODO : Mono to do that)
columnCriteria --> categoryID
rowCritera --> customerID
actualValue --> Value/Volume
customer_category_value.csv
customer_category_volume.csv
customer_sku_value.csv
customer_sku_volume.csv
customer_brand_value.csv
customer_brand_volume.csv
#### ModelManagerService (Aparna Tiwari)
Daemon service listening to **'training-topic'**, and for every **ModelTrainingTrigggerEvent** in the following topic it does the following.
1. Downloads the dataset
2. Creates a dataset PVC
3. launch traning-container service with dataset PVC mounted and ModelDesignSchema (receipe) mounted
```protobuf=
message ModelTrainingTrigggerEvent {
ModelDesignSchema design_schema; // full ModelDesign will come
repated string dataset_artifact_url; // path to gcs blob storage
}
```
#### Model Training (Yashdeep)
Shortlived background job specifically for model training run as a container. Its primary purpose it to use the ModelDesignSchema and Dataset for training the model (Cutomer Profile Recommendation). Its assumed that the ModelDesignSchema and Dataset are mounted in the container. Once training is complete it publishes an **ModelInferenceEvent** event to **'model-inference-topic'**
Input and output Fields specifically used from ModelDesignSchema for Training process
```json
//TBD - csv file generated from model should have limited preference values or all the values
ModelDesignSchema
input:
customer_sku // customer_sku.csv
/retail/entity/customer
/retail/entity/sku
brand_id
category_id
/feature/retail/customer_sku/lifetime/quantity
/feature/retail/customer_sku/lifetime/value
customer_brand // customer_brand.csv
/retail/entity/customer
/retail/entity/brand
/feature/retail/customer_brand/lifetime/quantity
/feature/retail/customer_brand/lifetime/value
customer_category // customer_category.csv
/retail/entity/customer
/retail/entity/category
/feature/retail/customer_category/lifetime/quantity
/feature/retail/customer_category/lifetime/value
output:
// inference_customer_sku_preference.csv
// customer_id | sku_preference
// c1 | {s1: 0.91, s2: 0.82}
// c2 | {s1: 0.91, s2: 0.82}
// c3 | {s1: 0.91, s2: 0.82}
customer_sku_preference:
token : /inference/retail/customer_sku_prefence
/retail/entity/customer
dict
// inference_customer_brand_preference.csv
// customer_id | brand_preference
// c1 | {b1: 0.91, b2: 0.82}
customer_brand_preference:
/retail/entity/customer
dict
// inference_customer_category_preference.csv
// customer_id | category_preference
// c1 | {c1: 0.91, c2: 0.82}
customer_category_preference:
/retail/entity/customer
dict
hyper-pameters
count : 10
Type : Recommendation
Framework : CustomScript
```
Output event published to **model-inference-topic**
```protoc
message ModelInferenceEvent {
ModelDesignSchema design_schema; // full ModelDesign token will come
repated string inference_artifact_url; // path to blob storage
}
```
```
name: /train/retail/customer/recommendation
input:
modelDesignSchema.json
recipe.json
customer_category_value.csv
customer_category_volume.csv
customer_sku_value.csv
customer_sku_volume.csv
customer_brand_value.csv
customer_brand_volume.csv
output:
/feature/retail/customer/1: {
"sku": {
"sku1": {
"value": 10,
"volume": 900
},...
},
"category": {
"category1": {
"value": 10,
"volume": 900
},...
},
"brand": {
"brand1": {
"value": 10,
"volume": 900
},...
}
}
```
#### Inference Repository/Store
```
1. Read from model-inference-topic and get ModelInferenceEvent
message ModelInferenceEvent {
ModelDesignSchema design_schema;
repated string inference_artifact_url; //"customer_sku_prefence.csv" , "customer_brand_prefence.csv" , "customer_category.csv"
}
2. ModelDesignSchema output section should contain these file with the inference key contained in these files.
File names should correspond the key in the output section.
3. Using inference token in model design output section -
Inference schema will be fetched from schema cache which will be used to save inferences in inference store.
4. Download csv File
5. Convert CSV file data in Inference format -
{
"1515" : c1, //RecordKey uint32
inference_id: 1,
record_key : Base64Encoded(record_key),
inference_value: Base64Encoded(quanity),
inference_value_primittive: 0,
timestamp : // TBD for now current time stamp
}
```
GRPC call to make CreateInference request and support to save below kind of object.
/feature/retail/customer: {
"sku": {
"sku1": {
"value": 10,
"volume": 900
},...
},
"category": {
"category1": {
"value": 10,
"volume": 900
},...
},
"brand": {
"brand1": {
"value": 10,
"volume": 900
},...
}
}
#### Personalization Service (Java)
Personalization Service is a new service which we need to develop, it will talk to inference store via GetInference and get the customer profile for given customerID and then apply personalization logic to generate the reponse listed.
```
name: /inference/retail/customer/personalization_recommendation
input:
/entity/retail/customer
query = ex. Atta
output:
/feature/retail/customer: {
"sku": {
"sku1": {
"value": 10,
"volume": 900
},...
},
"category": {
"category1": {
"value": 10,
"volume": 900
},...
},
"brand": {
"brand1": {
"value": 10,
"volume": 900
},...
}
}
```
#### JSON Entity Schema Definitions:
##### 1. SKU Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Unique identifier for the SKU"
},
"product_id": {
"type": "string",
"description": "Identifier of the associated product"
},
"name": {
"type": "string",
"description": "Name or title of the SKU"
},
"description": {
"type": "string",
"description": "Description of the SKU"
},
"price": {
"type": "number",
"description": "Price of the SKU"
},
"currency": {
"type": "string",
"description": "Currency code (e.g., USD, EUR)"
},
"quantity_available": {
"type": "integer",
"description": "Quantity of the SKU available in stock"
},
"attributes": {
"type": "object",
"description": "Additional attributes specific to this SKU",
"additionalProperties": true
},
"weight": {
"type": "number",
"description": "Weight of the SKU in kilograms"
},
"dimensions": {
"type": "object",
"properties": {
"length": {
"type": "number",
"description": "Length of the SKU in centimeters"
},
"width": {
"type": "number",
"description": "Width of the SKU in centimeters"
},
"height": {
"type": "number",
"description": "Height of the SKU in centimeters"
}
},
"description": "Dimensions of the SKU"
},
"brand": {
"type": "string",
"description": "Brand or manufacturer of the SKU"
},
"barcode": {
"type": "string",
"description": "Barcode or UPC for the SKU"
}
},
"required": ["id", "product_id", "name", "price", "currency", "quantity_available"],
"additionalProperties": false
}
Example : {
"id": "1234567890",
"product_id": "9876543210",
"name": "iPhone 13 Pro Max",
"description": "The iPhone 13 Pro Max is the largest and most powerful iPhone ever.",
"price": 1199,
"currency": "USD",
"quantity_available": 10,
"attributes": {
"color": "Midnight Green",
"storage": "512GB"
},
"weight": 0.803,
"dimensions": {
"length": 16.08,
"width": 7.81,
"height": 0.767
},
"brand": "Apple",
"barcode": "1901980588267"
}
```
##### 1.Actual SKU Entity Schema
```
{
"token": {
"vertical": "Retail",
"type": "TELOS_ENTITY_TOKEN",
"id": {
"key": "/entity/retail/sku",
"u32": 1765
},
"knowledge": {
"entity": {
"name": {
"path": [
"sku"
]
}
}
}
},
"bizId": {
"primary": {
"vertical": "Retail",
"type": "TELOS_BIZ_TOKEN",
"id": {
"key": "/bizid/retail/sku/productskuid",
"u32": 1069
},
"knowledge": {
"bizId": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"productskuid"
]
}
}
}
},
"secondary": {
"element": {
"/bizid/retail/sku/productskuid": {
"vertical": "Retail",
"type": "TELOS_BIZ_TOKEN",
"id": {
"key": "/bizid/retail/sku/productskuid",
"u32": 1070
},
"knowledge": {
"bizId": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"productskuid"
]
}
}
}
}
}
}
},
"attribute": {
"attribute": {
"/attribute/retail/sku/mrp": {
"attributeToken": {
"vertical": "Retail",
"type": "TELOS_ATTRIBUTE_TOKEN",
"id": {
"key": "/attribute/retail/sku/mrp",
"u32": 1100
},
"knowledge": {
"attribute": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"mrp"
]
}
}
}
},
"attributeType": "KNOWLEDGE_ATTRIBUTE_TYPE_STATIC",
"quantitySchema": {
"quantitySchemaToken": {
"vertical": "Retail",
"type": "TELOS_QUANTITY_TOKEN",
"id": {
"key": "/quantity/retail/amount",
"u32": 371
},
"core": {
"quantity": {
"name": {
"path": [
"amount"
]
}
}
}
},
"quantityType": "QUANTITY_TYPE_ATOMIC",
"atomic": {
"atomicType": "QUANTITY_ATOMIC_TYPE_NUMERIC",
"numeric": {
"numericType": "NUMBER_TYPE_F64",
"range": {
"minimum": {
"f64": -10000000
},
"maximum": {
"f64": 10000000
},
"includeMinimum": true,
"includeMaximum": true
}
}
}
},
"qualifierSchemaMap": {}
},
"/attribute/retail/sku/weight": {
"attributeToken": {
"vertical": "Retail",
"type": "TELOS_ATTRIBUTE_TOKEN",
"id": {
"key": "/attribute/retail/sku/weight",
"u32": 1102
},
"knowledge": {
"attribute": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"weight"
]
}
}
}
},
"attributeType": "KNOWLEDGE_ATTRIBUTE_TYPE_STATIC",
"quantitySchema": {
"quantitySchemaToken": {
"vertical": "Retail",
"type": "TELOS_QUANTITY_TOKEN",
"id": {
"key": "/quantity/retail/amount",
"u32": 371
},
"core": {
"quantity": {
"name": {
"path": [
"amount"
]
}
}
}
},
"quantityType": "QUANTITY_TYPE_ATOMIC",
"atomic": {
"atomicType": "QUANTITY_ATOMIC_TYPE_NUMERIC",
"numeric": {
"numericType": "NUMBER_TYPE_F64",
"range": {
"minimum": {
"f64": -10000000
},
"maximum": {
"f64": 10000000
},
"includeMinimum": true,
"includeMaximum": true
}
}
}
},
"qualifierSchemaMap": {}
},
"/attribute/retail/sku/description": {
"attributeToken": {
"vertical": "Retail",
"type": "TELOS_ATTRIBUTE_TOKEN",
"id": {
"key": "/attribute/retail/sku/description",
"u32": 1103
},
"knowledge": {
"attribute": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"description"
]
}
}
}
},
"attributeType": "KNOWLEDGE_ATTRIBUTE_TYPE_STATIC",
"quantitySchema": {
"quantitySchemaToken": {
"vertical": "Retail",
"type": "TELOS_QUANTITY_TOKEN",
"id": {
"key": "/quantity/retail/text",
"u32": 370
},
"core": {
"quantity": {
"name": {
"path": [
"text"
]
}
}
}
},
"quantityType": "QUANTITY_TYPE_ATOMIC",
"atomic": {
"atomicType": "QUANTITY_ATOMIC_TYPE_TEXT",
"text": {
"textType": "TEXT_QUANTITY_TYPE_SIMPLE"
}
}
},
"qualifierSchemaMap": {}
},
"/attribute/retail/sku/colour": {
"attributeToken": {
"vertical": "Retail",
"type": "TELOS_ATTRIBUTE_TOKEN",
"id": {
"key": "/attribute/retail/sku/colour",
"u32": 1101
},
"knowledge": {
"attribute": {
"entityType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"name": {
"path": [
"colour"
]
}
}
}
},
"attributeType": "KNOWLEDGE_ATTRIBUTE_TYPE_STATIC",
"quantitySchema": {
"quantitySchemaToken": {
"vertical": "Retail",
"type": "TELOS_QUANTITY_TOKEN",
"id": {
"key": "/quantity/retail/colour",
"u32": 384
},
"core": {
"quantity": {
"name": {
"path": [
"colour"
]
}
}
}
},
"quantityType": "QUANTITY_TYPE_ATOMIC",
"atomic": {
"atomicType": "QUANTITY_ATOMIC_TYPE_SYMBOLIC",
"symbolic": {
"type": "QUANTITY_ATOMIC_SYMBOL_TYPE_U64",
"symbolCollection": {
"collectionToken": {
"vertical": "Retail",
"type": "TELOS_COLLECTION_TOKEN",
"id": {
"key": "/collection/retail/colour",
"u32": 370
},
"core": {
"collection": {
"name": {
"path": [
"colour"
]
}
}
}
},
"collectionType": "COLLECTION_TYPE_SYMBOL",
"symbol": {
"set": {
"str": {
"element": {
"tan": true,
"maroon": true,
"lavender": true,
"black": true,
"yellow": true,
"blue": true,
"azure": true,
"lime": true,
"fuchsia": true,
"salmon": true,
"olive": true,
"orange": true,
"pink": true,
"mint green": true,
"turquoise": true,
"cyan": true,
"grey": true,
"purple": true,
"skyblue": true,
"indigo": true,
"magenta": true,
"violet": true,
"ivory": true,
"plum": true,
"gold": true,
"red": true,
"silver": true,
"orchid": true,
"teal": true,
"green": true,
"white": true
}
}
},
"strIndex": {
"tan": "371",
"maroon": "372",
"lavender": "373",
"black": "374",
"yellow": "375",
"blue": "376",
"azure": "377",
"lime": "378",
"fuchsia": "379",
"salmon": "380",
"olive": "381",
"orange": "382",
"pink": "383",
"mint green": "384",
"turquoise": "385",
"cyan": "386",
"grey": "387",
"purple": "388",
"skyblue": "389",
"indigo": "390",
"magenta": "391",
"violet": "392",
"ivory": "393",
"plum": "394",
"gold": "395",
"red": "396",
"silver": "397",
"orchid": "398",
"teal": "399",
"green": "400",
"white": "401"
}
}
}
}
}
},
"qualifierSchemaMap": {}
}
}
},
"predicate": {
"predicate": {
"/predicate/retail/sku/belongs_to_category": {
"subject": {
"vertical": "Retail",
"type": "TELOS_ENTITY_TOKEN",
"id": {
"key": "/entity/retail/sku",
"u32": 1765
},
"knowledge": {
"entity": {
"name": {
"path": [
"sku"
]
}
}
}
},
"predicate": {
"vertical": "Retail",
"type": "TELOS_PREDICATE_TOKEN",
"id": {
"key": "/predicate/retail/sku/belongs_to_category",
"u32": 514
},
"knowledge": {
"predicate": {
"subjectType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"objectType": {
"key": "/entity/retail/category",
"u32": 1761
},
"name": {
"path": [
"belongs_to_category"
]
}
}
}
},
"object": {
"vertical": "Retail",
"type": "TELOS_ENTITY_TOKEN",
"id": {
"key": "/entity/retail/category",
"u32": 1761
},
"knowledge": {
"entity": {
"name": {
"path": [
"category"
]
}
}
}
},
"qualifierSchemaMap": {},
"predicateType": {
"oneToMany": true
}
},
"/predicate/retail/sku/has_brand": {
"subject": {
"vertical": "Retail",
"type": "TELOS_ENTITY_TOKEN",
"id": {
"key": "/entity/retail/sku",
"u32": 1765
},
"knowledge": {
"entity": {
"name": {
"path": [
"sku"
]
}
}
}
},
"predicate": {
"vertical": "Retail",
"type": "TELOS_PREDICATE_TOKEN",
"id": {
"key": "/predicate/retail/sku/has_brand",
"u32": 515
},
"knowledge": {
"predicate": {
"subjectType": {
"key": "/entity/retail/sku",
"u32": 1765
},
"objectType": {
"key": "/entity/retail/brand",
"u32": 1764
},
"name": {
"path": [
"has_brand"
]
}
}
}
},
"object": {
"vertical": "Retail",
"type": "TELOS_ENTITY_TOKEN",
"id": {
"key": "/entity/retail/brand",
"u32": 1764
},
"knowledge": {
"entity": {
"name": {
"path": [
"brand"
]
}
}
}
},
"qualifierSchemaMap": {},
"predicateType": {
"oneToMany": true
}
}
}
}
}
```
##### 2. Article Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the article."
},
"title": {
"type": "string",
"description": "The title of the article."
},
"description": {
"type": "string",
"description": "A description of the article."
},
"productType": {
"type": "string",
"description": "The product type of the article."
},
"brand": {
"type": "string",
"description": "The brand of the article."
},
"gtin": {
"type": "string",
"description": "The GTIN of the article."
},
"mpn": {
"type": "string",
"description": "The MPN of the article."
},
"variantGroupId": {
"type": "string",
"description": "The variant group ID of the article."
},
"productUri": {
"type": "string",
"description": "The URI of the product page for the article."
},
"imageUrl": {
"type": "string",
"description": "The URL of the image for the article."
},
"price": {
"type": "number",
"description": "The price of the article."
},
"availability": {
"type": "string",
"description": "The availability of the article."
},
"condition": {
"type": "string",
"description": "The condition of the article."
},
"attributes": {
"type": "object",
"description": "A map of attributes for the article."
},
"categories": {
"type": "array",
"description": "A list of categories for the article."
},
"product_id": {
"type": "integer",
"description": "The product ID for the article."
},
"variant_id": {
"type": "integer",
"description": "The variant ID for the article."
},
"body_html": {
"type": "string",
"description": "The HTML for the article's body."
},
"published_at": {
"type": "datetime",
"description": "The date and time the article was published."
},
"updated_at": {
"type": "datetime",
"description": "The date and time the article was last updated."
},
"author": {
"type": "string",
"description": "The author of the article."
},
"image": {
"type": "string",
"description": "The URL of the image for the article."
},
"tags": {
"type": "array",
"description": "A list of tags for the article."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the article."
}
},
"required": ["id"]
}
Example :
{
"id": "1234567890",
"title": "The Best Laptops for Students in 2023",
"description": "This article reviews the best laptops for students in 2023, based on factors such as price, performance, and portability.",
"productType": "Laptop",
"brand": "Apple",
"gtin": "0190198074911",
"mpn": "M2700LL/A",
"variantGroupId": "1234567890",
"productUri": "https://www.apple.com/shop/product/M2700LL/A/macbook-air-m2-chip-13-inch-retina-display-laptop",
"imageUrl": "https://www.apple.com/content/dam/apple/in/home/shop/macbook-air-m2/macbook-air-m2-hero-midnight-202306.jpg",
"price": 999,
"availability": "In stock",
"condition": "New",
"attributes": {
"processor": "M2 chip",
"memory": "8GB",
"storage": "256GB",
"display": "13.3-inch Retina display"
},
"categories": [
"Laptops",
"MacBooks"
],
"product_id": 1234567890,
"variant_id": 9876543210,
"body_html": "<p>The MacBook Air is a great choice for students because it's lightweight and portable, yet still offers powerful performance. The new M2 chip is even faster than the previous generation, and the long battery life means you can go all day without having to worry about running out of power.</p>",
"published_at": "2023-08-04T12:00:00.000Z",
"updated_at": "2023-08-04T12:00:00.000Z",
"author": "John Doe",
"tags": [
"laptops",
"students",
"MacBook Air"
],
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 3. Merchant Entity
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the merchant."
},
"name": {
"type": "string",
"description": "The name of the merchant."
},
"description": {
"type": "string",
"description": "A description of the merchant."
},
"logoUri": {
"type": "string",
"description": "The URL of the logo for the merchant."
},
"url": {
"type": "string",
"description": "The URL of the merchant's website."
},
"contactInfo": {
"type": "object",
"properties": {
"email": {
"type": "string",
"description": "The email address of the merchant."
},
"phone": {
"type": "string",
"description": "The phone number of the merchant."
},
"address": {
"type": "object",
"properties": {
"street1": {
"type": "string",
"description": "The first line of the merchant's address."
},
"street2": {
"type": "string",
"description": "The second line of the merchant's address."
},
"city": {
"type": "string",
"description": "The city of the merchant's address."
},
"state": {
"type": "string",
"description": "The state of the merchant's address."
},
"postalCode": {
"type": "string",
"description": "The postal code of the merchant's address."
},
"country": {
"type": "string",
"description": "The country of the merchant's address."
}
}
}
}
},
"shippingPolicies": {
"type": "array",
"description": "A list of the merchant's shipping policies."
},
"returnPolicies": {
"type": "array",
"description": "A list of the merchant's return policies."
},
"paymentMethods": {
"type": "array",
"description": "A list of the merchant's accepted payment methods."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the merchant."
}
},
"required": ["id", "name"]
}
shippingPolicies: A list of the merchant's shipping policies.
returnPolicies: A list of the merchant's return policies.
paymentMethods: A list of the merchant's accepted payment methods.
metafield: A map of metafields for the merchant.
Example :
{
"id": "1234567890",
"name": "Amazon",
"description": "Amazon is an American multinational technology company that specializes in e-commerce, cloud computing, digital streaming, and artificial intelligence. It is one of the Big Five American information technology companies, along with Google, Microsoft, Apple, and Meta. As of 2023, it is the world's largest online retailer by revenue.",
"logoUri": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Amazon_logo.svg/1200px-Amazon_logo.svg.png",
"url": "https://www.amazon.com/",
"contactInfo": {
"email": "customer-service@amazon.com",
"phone": "1-866-216-1072",
"address": {
"street1": "410 Terry Avenue North",
"city": "Seattle",
"state": "WA",
"postalCode": "98109",
"country": "US"
}
},
"shippingPolicies": [
{
"name": "Free Shipping on Orders Over $25",
"description": "Orders over $25 ship for free in the United States."
},
{
"name": "Two-Day Shipping",
"description": "Orders placed with Amazon Prime ship for free in two days."
}
],
"returnPolicies": [
{
"name": "30-Day Free Returns",
"description": "Items purchased on Amazon can be returned for free within 30 days of purchase."
}
],
"paymentMethods": [
"Visa",
"MasterCard",
"American Express",
"Discover",
"PayPal"
],
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 4) Warehouse Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Unique identifier for the warehouse"
},
"name": {
"type": "string",
"description": "Name or label for the warehouse"
},
"location": {
"type": "object",
"properties": {
"address": {
"type": "string",
"description": "Street address of the warehouse"
},
"city": {
"type": "string",
"description": "City where the warehouse is located"
},
"state": {
"type": "string",
"description": "State or province of the warehouse"
},
"postal_code": {
"type": "string",
"description": "Postal code or ZIP code of the warehouse"
},
"country": {
"type": "string",
"description": "Country where the warehouse is situated"
}
},
"description": "Physical location of the warehouse"
},
"capacity": {
"type": "integer",
"description": "Total storage capacity of the warehouse"
},
"inventory": {
"type": "array",
"items": {
"type": "object",
"properties": {
"product_id": {
"type": "string",
"description": "Unique identifier of the product in the warehouse"
},
"quantity": {
"type": "integer",
"description": "Quantity of the product in stock at the warehouse"
}
},
"required": ["product_id", "quantity"]
},
"description": "Current inventory in the warehouse"
}
},
"required": ["id", "name", "location", "capacity"],
"additionalProperties": false
}
capacity: The capacity of the warehouse in square feet.
type: The type of warehouse, such as fulfillment center or distribution center.
status: The status of the warehouse, such as active or inactive.
Example :
{
"id": "1234567890",
"name": "Amazon Warehouse, Seattle",
"location": {
"address": "410 Terry Avenue North",
"city": "Seattle",
"state": "WA",
"postal_code": "98109",
"country": "US"
},
"capacity": 1000000,
"inventory": [
{
"product_id": "1111111111",
"quantity": 100
},
{
"product_id": "2222222222",
"quantity": 200
},
{
"product_id": "3333333333",
"quantity": 300
}
],
"imageUrl": "https://example.com/amazon-warehouse-seattle.jpg"
}
```
##### 5) Customer Entity
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the customer."
},
"firstName": {
"type": "string",
"description": "The customer's first name."
},
"lastName": {
"type": "string",
"description": "The customer's last name."
},
"email": {
"type": "string",
"description": "The customer's email address."
},
"phone": {
"type": "string",
"description": "The customer's phone number."
},
"shippingAddress": {
"type": "object",
"properties": {
"street1": {
"type": "string",
"description": "The first line of the customer's shipping address."
},
"street2": {
"type": "string",
"description": "The second line of the customer's shipping address."
},
"city": {
"type": "string",
"description": "The city of the customer's shipping address."
},
"state": {
"type": "string",
"description": "The state of the customer's shipping address."
},
"postalCode": {
"type": "string",
"description": "The postal code of the customer's shipping address."
},
"country": {
"type": "string",
"description": "The country of the customer's shipping address."
}
}
},
"billingAddress": {
"type": "object",
"properties": {
"street1": {
"type": "string",
"description": "The first line of the customer's billing address."
},
"street2": {
"type": "string",
"description": "The second line of the customer's billing address."
},
"city": {
"type": "string",
"description": "The city of the customer's billing address."
},
"state": {
"type": "string",
"description": "The state of the customer's billing address."
},
"postalCode": {
"type": "string",
"description": "The postal code of the customer's billing address."
},
"country": {
"type": "string",
"description": "The country of the customer's billing address."
}
}
},
"orders": {
"type": "array",
"description": "A list of the customer's orders."
},
"loyaltyStatus": {
"type": "string",
"description": "The customer's loyalty status, such as bronze, silver, or gold."
},
"tags": {
"type": "array",
"description": "A list of tags associated with the customer."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the customer."
}
},
"required": ["id"]
}
loyaltyStatus: The customer's loyalty status.
tags: A list of tags associated with the customer.
Example :
{
"id": "1234567890",
"firstName": "John",
"lastName": "Doe",
"email": "john.doe@example.com",
"phone": "(123) 456-7890",
"shippingAddress": {
"street1": "123 Main Street",
"city": "Anytown",
"state": "CA",
"postalCode": "91234",
"country": "US"
},
"billingAddress": {
"street1": "456 Elm Street",
"city": "Anytown",
"state": "CA",
"postalCode": "91234",
"country": "US"
},
"orders": [
{
"id": "9876543210",
"orderDate": "2023-08-04T12:00:00.000Z",
"total": 100.00,
"items": [
{
"product_id": "1111111111",
"quantity": 1,
"price": 50.00
},
{
"product_id": "2222222222",
"quantity": 2,
"price": 25.00
}
]
}
],
"loyaltyStatus": "silver",
"tags": ["vip", "customer"],
"imageUrl": "https://example.com/john-doe.jpg"
}
```
##### 6) Segment Entity Scehma
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the segment."
},
"name": {
"type": "string",
"description": "The name of the segment."
},
"description": {
"type": "string",
"description": "A description of the segment."
},
"filters": {
"type": "array",
"items": {
"type": "object",
"properties": {
"field": {
"type": "string",
"description": "The field to filter on."
},
"operator": {
"type": "string",
"description": "The operator to use for the filter."
},
"value": {
"type": "string",
"description": "The value to filter on."
}
}
}
},
"segmentType": {
"type": "string",
"description": "The type of segment, such as customer segment or product segment."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the segment."
}
},
"required": ["id"]
}
filters: An array of filters to apply to the segment.
segmentType: The type of segment, such as customer segment or product segment.
Example :
{
"id": "1234567890",
"name": "VIP Customers",
"description": "This segment includes all customers who have spent more than $1000 in the past year.",
"filters": [
{
"field": "orders.total_amount",
"operator": "greaterThan",
"value": 1000
}
],
"segmentType": "customer",
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 7) Brand Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the brand."
},
"name": {
"type": "string",
"description": "The name of the brand."
},
"description": {
"type": "string",
"description": "A description of the brand."
},
"logoImage": {
"type": "string",
"description": "The URL of the brand's logo image."
},
"website": {
"type": "string",
"description": "The URL of the brand's website."
},
"socialMediaLinks": {
"type": "object",
"description": "A map of the brand's social media links."
},
"products": {
"type": "array",
"description": "A list of the brand's products."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the brand."
}
},
"required": ["id"]
}
Example :
{
"id": "1234567890",
"name": "Apple",
"description": "Apple Inc. is an American multinational technology company that specializes in consumer electronics, computer software, and online services. Apple is one of the Big Five American information technology companies, along with Alphabet (Google), Amazon, Meta (Facebook), and Microsoft.",
"logoImage": "https://upload.wikimedia.org/wikipedia/commons/thumb/f/fa/Apple_logo_%282015%29.svg/1200px-Apple_logo_%282015%29.svg.png",
"website": "https://www.apple.com/",
"socialMediaLinks": {
"facebook": "https://www.facebook.com/apple",
"twitter": "https://twitter.com/apple",
"instagram": "https://www.instagram.com/apple"
},
"products": [
{
"id": "9876543210",
"name": "iPhone 13 Pro Max",
"description": "The iPhone 13 Pro Max is the largest and most powerful iPhone ever."
},
{
"id": "1111111111",
"name": "MacBook Air M2",
"description": "The MacBook Air M2 is the latest version of Apple's popular ultra-thin laptop."
}
],
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 8) Building Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the building."
},
"name": {
"type": "string",
"description": "The name of the building."
},
"address": {
"type": "object",
"properties": {
"street1": {
"type": "string",
"description": "The first line of the building's address."
},
"street2": {
"type": "string",
"description": "The second line of the building's address."
},
"city": {
"type": "string",
"description": "The city of the building's address."
},
"state": {
"type": "string",
"description": "The state of the building's address."
},
"postalCode": {
"type": "string",
"description": "The postal code of the building's address."
},
"country": {
"type": "string",
"description": "The country of the building's address."
}
}
},
"image": {
"type": "string",
"description": "The URL of the building's image."
},
"capacity": {
"type": "integer",
"description": "The capacity of the building in square feet."
},
"type": {
"type": "string",
"description": "The type of building, such as office building, retail building, or warehouse."
},
"status": {
"type": "string",
"description": "The status of the building, such as active or inactive."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the building."
}
},
"required": ["id"]
}
image: The URL of the building's image.
capacity: The capacity of the building in square feet.
Example :
{
"id": "1234567890",
"name": "Empire State Building",
"address": {
"street1": "350 Fifth Avenue",
"city": "New York",
"state": "NY",
"postalCode": "10118",
"country": "US"
},
"image": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/df/Empire_State_Building_2010.jpg/1200px-Empire_State_Building_2010.jpg",
"capacity": 2,768,591,
"type": "Office",
"status": "Active",
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 9) Company Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the company."
},
"name": {
"type": "string",
"description": "The name of the company."
},
"description": {
"type": "string",
"description": "A description of the company."
},
"logoImage": {
"type": "string",
"description": "The URL of the company's logo image."
},
"website": {
"type": "string",
"description": "The URL of the company's website."
},
"socialMediaLinks": {
"type": "object",
"description": "A map of the company's social media links."
},
"products": {
"type": "array",
"description": "A list of the company's products."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the company."
}
},
"required": ["id"]
}
logoImage: The URL of the company's logo image.
socialMediaLinks: A map of the company's social media links.
Example :
{
"id": "1234567890",
"name": "Amazon",
"description": "Amazon is an American multinational technology company that specializes in e-commerce, cloud computing, digital streaming, and artificial intelligence. It is one of the Big Five American information technology companies, along with Google, Microsoft, Apple, and Meta. As of 2023, it is the world's largest online retailer by revenue.",
"logoImage": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Amazon_logo.svg/1200px-Amazon_logo.svg.png",
"website": "https://www.amazon.com/",
"socialMediaLinks": {
"facebook": "https://www.facebook.com/amazon",
"twitter": "https://twitter.com/amazon",
"instagram": "https://www.instagram.com/amazon"
},
"products": [
{
"id": "9876543210",
"name": "Amazon Echo Dot",
"description": "A smart speaker that lets you control your smart home devices, play music, and get information."
},
{
"id": "1111111111",
"name": "Kindle Paperwhite",
"description": "An e-reader that lets you read books, magazines, and newspapers on a glare-free screen."
}
],
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
##### 10) Location Entity Schema
```
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "The unique identifier for the location."
},
"name": {
"type": "string",
"description": "The name of the location."
},
"address": {
"type": "object",
"properties": {
"street1": {
"type": "string",
"description": "The first line of the location's address."
},
"street2": {
"type": "string",
"description": "The second line of the location's address."
},
"city": {
"type": "string",
"description": "The city of the location's address."
},
"state": {
"type": "string",
"description": "The state of the location's address."
},
"postalCode": {
"type": "string",
"description": "The postal code of the location's address."
},
"country": {
"type": "string",
"description": "The country of the location's address."
}
}
},
"coordinates": {
"type": "object",
"properties": {
"latitude": {
"type": "number",
"description": "The latitude of the location."
},
"longitude": {
"type": "number",
"description": "The longitude of the location."
}
}
},
"hoursOfOperation": {
"type": "array",
"description": "A list of the location's hours of operation."
},
"image": {
"type": "string",
"description": "The URL of the location's image."
},
"types": {
"type": "array",
"description": "A list of the location's types."
},
"metafield": {
"type": "object",
"description": "A map of metafields for the location."
}
},
"required": ["id"]
}
Example :
{
"id": "1234567890",
"name": "Starbucks, 123 Main Street",
"address": {
"street1": "123 Main Street",
"city": "Anytown",
"state": "CA",
"postalCode": "91234",
"country": "US"
},
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243684
},
"hoursOfOperation": [
{
"dayOfWeek": "Monday",
"openTime": "6:00 AM",
"closeTime": "10:00 PM"
},
{
"dayOfWeek": "Tuesday",
"openTime": "6:00 AM",
"closeTime": "10:00 PM"
},
{
"dayOfWeek": "Wednesday",
"openTime": "6:00 AM",
"closeTime": "10:00 PM"
},
{
"dayOfWeek": "Thursday",
"openTime": "6:00 AM",
"closeTime": "10:00 PM"
},
{
"dayOfWeek": "Friday",
"openTime": "6:00 AM",
"closeTime": "10:00 PM"
},
{
"dayOfWeek": "Saturday",
"openTime": "8:00 AM",
"closeTime": "6:00 PM"
},
{
"dayOfWeek": "Sunday",
"openTime": "8:00 AM",
"closeTime": "6:00 PM"
}
],
"image": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/d3/Starbucks_Coffee_logo.svg/1200px-Starbucks_Coffee_logo.svg.png",
"types": ["coffee_shop", "cafe"],
"metafield": {
"marketingCampaignId": "1234567890"
}
}
```
#### JSON Event Schema Definitions -(Draft) :
##### Purchase (delete to invetory) Event :
```
{
"type": "object",
"properties": {
"user_id": {
"type": "string",
"description": "The unique identifier for the user."
},
"order_id": {
"type": "string",
"description": "The unique identifier for the order."
},
"items": {
"type": "array",
"description": "A list of the items purchased."
},
"total_price": {
"type": "number",
"description": "The total price of the order."
},
"timestamp": {
"type": "string",
"format": "date-time",
"description": "The timestamp of the Purchase event."
},
"context": {
"type": "object",
"description": "A map of contextual data about the Purchase event."
}
},
"required": ["user_id", "order_id", "items", "total_price", "timestamp"]
}
Example :
{
"user_id": "1234567890",
"order_id": "9876543210",
"items": [
{
"product_id": "1111111111",
"product_name": "iPhone 13 Pro Max",
"product_image": "https://upload.wikimedia.org/wikipedia/commons/thumb/b/b8/IPhone_13_Pro_Max_display.jpg/1200px-IPhone_13_Pro_Max_display.jpg",
"quantity": 1,
"price": 1099.00
},
{
"product_id": "2222222222",
"product_name": "Apple Watch Series 8",
"product_image": "https://upload.wikimedia.org/wikipedia/commons/thumb/e/e0/Apple_Watch_Series_8.jpg/1200px-Apple_Watch_Series_8.jpg",
"quantity": 1,
"price": 399.00
}
],
"total_price": 1498.00,
"timestamp": "2023-11-07T17:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
}
```
#### For more Recommendation events ,please refer the [xls](https://rilcloud-my.sharepoint.com/:x:/g/personal/chandiprasad_s_ril_com/EVHbHfr7S3pCoiWFSbjK-fEB-YPOf06CzLsvZVxSD31L9Q?CID=a12e92db-3972-33b6-b762-a748561e9eda) .
## 2. Demand Forecasting (TODO : Amandeep please fill it based on learning from 1)
## 3. Inventory Optimisation (TODO : Amandeep please fill it based on learning from 1)
# Execution Plan
Below are the tasks w.r.t all the workflows with their corresponding ID's in azure
| ID | Title | Assigned To | Dependency | Core/Domain Telos | Status |
| ------------------------------------------------------------------ | -------------------------------------------------- | ---------------- | ---------------- | ----------------- | --- |
| [25173](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25173/) | Schema Definitions for Entities | Chandiprasad S | | Core TelOS | Done |
| [25174](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25174/) | Json Entity schema to TelOS schema mapping | Chandiprasad S | Priti Bhimsariya | Core TelOS | Done |
| [25175](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25175/) | Onboarding Entity schema to Central schema service | Chandiprasad S | Priti Bhimsariya | Core TelOS | Done |
| [25176](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25176/) | Schema Definitions for Events | Chandiprasad S | | Core TelOS | Done |
| [25177](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25177/) | Json Event schema to TelOS schema mapping | Chandiprasad S | Priti Bhimsariya | Core TelOS | Done |
| [25178](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25178/) | Onboarding Event schema to Central schema service | Chandiprasad S | Priti Bhimsariya | Core TelOS | Done |
| [25179](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25179/) | Synthetic Data Generation | Chandiprasad S | Dharmendra | TelOS | Done |
| [25180](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25180/) | Simulator Event Ingestion | Rajan Punchouty | | Core TelOS | |
| [25181](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25181/) | Define schema for Model, Feature, Inference, Label | Priti Bhimsariya | | Core TelOS | |
| [25182](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25182/) | Define the DAG | Mona Sharma | | Core TelOS | |
| [25183](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25183/) | Define the configuration for Quantize | Priti Bhimsariya | | Core TelOS | |
| [25184](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25184/) | Define the configuration for Enrich | Priti Bhimsariya | | Core TelOS | |
| [25185](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25185/) | Define the configuration for Pivot | Mona Sharma | | Core TelOS | |
| [25186](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25186/) | Define the configuration for Aggregate (reduce) | Mona Sharma | | Core TelOS | |
| [25187](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25187/) | Feature Repository Service | Karan Kesarwani | | Core TelOS | Done |
| [25188](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25188/) | Trigger Configuration for all Models | Saket Verma | | Core TelOS | |
| [25189](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25189/) | Scheduler Service (should deploy Fetch & Train) | Saket Verma | | Core TelOS | |
| [25190](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25190/) | Fetch Service (Configuration & generate a CSV) | Rajender7 Kumar | | Core TelOS | |
| [25191](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25191/) | Generate a Training container | Naresh Ganesan | | Core TelOS | |
| [25192](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25192/) | Model Docker (Up & Running) | Naresh Ganesan | | Core TelOS | |
| [25193](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25193/) | Store the Model data to Inference Store | Naresh Ganesan | | Core TelOS | |
| [25194](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25194/) | Inference Store API/Service | Karan Kesarwani | | Core TelOS | Inprogress |
| [25195](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25195/) | Model Training Service Orchestration | Naresh Ganesan | | Core TelOS | |
| [25196](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25196/) | Personalization Java Service | Rajan Punchouty | | Domain TelOS | |
| [25197](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25197/) | Product Documentation for all 3 workflows | Vankayalapati P | | | |
| [25224](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25224/) | Replenishment DAG | Mona Sharma | | Core TelOS | |
| [25226](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25226/) | Allocation Optimizer Service | Naresh Ganesan | | Domain TelOS | |
| [25228](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25228/) | Real Time Inventory check | Karan Kesarwani | | Domain TelOS | |
| [25229](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25229/) | Safety stock check service | Karan Kesarwani | | Domain TelOS | |
| [25231](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25231/) | Supply Optimizer Service | Priti Bhimsariya | | Domain TelOS | |
| [25232](https://devops.jio.com/AICOE/TelOS/_workitems/edit/25232/) | Real Time Inventory Update Service | Karan Kesarwani | | Domain TelOS | |
Below should run for Forecasting and Recommendation:
1. Trigger config for all the models - Saket
2. Scheuler service - Saket
3. Fetch Service (config and generated a CSV) - Rajinder
4. Listing to queue to Fetch Service output and generate a Traning Container - Naresh
5. Model Docker up and running - Naresh
6. Model is store the data to inference store - Naresh
7. Inference Store API/Service - Karan
1. Scheduler - Cron Job
2. Fetch
3. RecommendationModel
Manyank : CentralSchema service will create an ModelDesign Object and make a call to ModelStore API to store the json listed in this doc.
ModelStore - Chandi - need to change
cron stored - redis - Saket
Scheduler : Saket
1. read from Redis (model-id, design) https://devops.jio.com/AICOE/TelOS/_git/telos?path=%2Finfra%2Fscheduler&version=GBdevelop
2. launch the fetch instance cotainer (design.criteria)
3. fetch will write to traning-topic and event named ModelTrainingTrigggerEvent
4. fetch will upload all x files to blob store
5. fetch container will die
Fetech Service - Rajinder /Nitin
Preprocess : Nitin
"preprocessor": {
"multivariate": {
"preprocess": [
{
"expression": "(/ (- variable:/feature/retail/brand_customer_sku/value/time_window_type_week constant:/quantity/retail/mean) constant:/quantity/retail/standard_deviation)",
"output": "/feature/healthcare/brand_customer_sku/Z_avg_mrp/time_window_type_week",
"constants": {
"/quantity/retail/mean": {
"doubleData": 212.5680919
},
"/quantity/retail/standard_deviation": {
"doubleData": 543.5968978
}
}
},
{
"expression": "(/ (- variable:/feature/retail/brand_customer_sku/value/time_window_type_month constant:/quantity/retail/mean) constant:/quantity/retail/standard_deviation)",
"output": "/feature/retail/brand_customer_sku/Z_value/time_window_type_month",
"constants": {
"/quantity/retail/mean": {
"doubleData": 151.565344655345
},
"/quantity/retail/standard_deviation": {
"doubleData": 379.5876069
}
}
}
}
output of ModelManager/traning container
model-batch-inference-topic - produce
event : ModelBatchInferenceEvent :
{
string model_id;
repated string dataset_artifact_url; // path to k8s
}
Fetch has written a event to topic 'traning-topic'
Event : ModelTrainingTrigggerEvent
{
ModelDesignSchema design_schema; // full ModelDesign token will come
repated string dataset_artifact_url; // path to blob storage
}
Aparna : ModelManagerSevice : https://devops.jio.com/AICOE/TelOS/_git/telos?path=%2Fcortex%2Fmodel-manager%2Fmodel_manager&version=GBdevelop
- read from traning-topic
- download the dataset
- created a PVC
- launch traning-container service with PVC mount
https://devops.jio.com/AICOE/TelOS/_git/telos?path=%2Fcortex%2Fmodel-manager%2Fmodel_training&version=GBdevelop
- output of the traning will be uploaded to minio
- No status will get published for killing the infra
- Publish an event to 'model-inference-topic'
ModelInferenceEvent
{
ModelDesignSchema design_schema; // full ModelDesign token will come
repated string dataset_artifact_url; // path to blob storage
}
ModelDesign
input :
customer_sku
/retail/entity/customer
/retail/entity/sku
brand_id
category_id
/feature/retail/customer_sku/lifetime/quantity
/feature/retail/customer_sku/lifetime/value
customer_brand
/retail/entity/customer
/retail/entity/brand
/feature/retail/customer_brand/lifetime/quantity
/feature/retail/customer_brand/lifetime/value
customer_category
/retail/entity/customer
/retail/entity/category
/feature/retail/customer_category/lifetime/quantity
/feature/retail/customer_category/lifetime/value
output:
customer_sku_prefence:
token : /inference/retail/customer_sku_prefence
/retail/entity/customer
dict
customer_brand_prefence:
/retail/entity/customer
dict
customer_category:
/retail/entity/customer
dict
hype-pameters
count : 10
Type : Recommendation
Framework : Custom
criteria :
input :
TIMEBASED : 7 days | NO CRITERIA
customer_sku >> customer_sku.csv
entity/entity/customer
/retail/entity/sku
/retail/entity/brand_id
/retail/entity/category_id
/feature/retail/customer_sku/lifetime/quantity
/feature/retail/customer_sku/lifetime/value
TIMEBASED : 7 days | NO CRITERIA
customer_brand >> customer_brand.csv
/retail/entity/customer
/retail/entity/brand
/feature/retail/customer_brand/lifetime/quantity
/feature/retail/customer_brand/lifetime/value
TIMEBASED : 7 days | NO CRITERIA
customer_category >> customer_category.csv
/retail/entity/customer
/retail/entity/category
/feature/retail/customer_category/lifetime/quantity
/feature/retail/customer_category/lifetime/value
prep
Create Feature Schema : /feature/retail/customer_sku/lifetime/quantity
record_key = /entity/customer
value = /quantity/atomic/numeric
Create Inference Schema : /inference/retail/customer_sku_prefence
record_key = customer
value = /quantity/compond/basket/sku_preference
key = customer_id
map
key = sku_id
value = float
recommendation - Sidhharth : r.py Matrix Factorni
input :
customer_sku.csv
customer_brand.csv
customer_category.csv
output :
customer_sku_prefence.csv
customer_brand_prefence.csv
customer_category.csv
demand forecasting - Tushar : d.py
Yashdeep will take r.py and d.py and make code changes in traning sdk
InferenceRepository: Karan
1. read model-inference-topic and get ModelInferenceEvent
2. files convert it to telosformat.
customer_sku_prefence.csv
c1 : customer_id
c2: dict
c1: /retail/entity/customer
c2: /quantity/compond/basket/sku_preference
key = customer_id
map
key = sku_id
value = score
customer_brand_prefence.csv
customer_category.csv
/inference/retail/customer_sku_prefence : CreateInferenceAPI Call
key = /retail/entity/customer
value : {
sku1 : {
value : .10
},
...
..
}
Java Service PersonalizationService REST API
request :
{
customer_id : 1
query : atta,
}
resonse :
{
[,,,,,]
}
Spring
- Controller
- /
- customer_id
--> inference_store
TempJson =- readInference["/inference/retail/customer_sku_prefence", "", "", Citeria = C11]
t1 & t2 & t3 = profile json
// logic - Krishna
return ["shantibhog atta", "xzy atta"]
## Final
```
message Program {
string vertical = 1
string client = 2;
float version = 3;
ProgramCollections collections = 5;
ProgramQuantities quantities = 6;
ProgramUnits units = 7;
ProgramEntities entities = 10
ProgramEvents events = 11
ProgramFunctions functions = 12;
ProgramFeatures features = 14;
ProgramModels models = 13;
ProgramInferences inferences = 15;
ProgramDecisions decisions = 16;
ProgramMetrics metrics = 17;
}
message ProgramCollections {
map<string, CollectionSchema> collection = 1;
}
message ProgramQuantities {
map<string, QuantitySchema> quantity = 2;
}
message ProgramUnits {
map<string, UnitSchema> unit = 3;
}
message ProgramEntities {
map<string, EntitySchema> entity = 1
}
message ProgramEvents {
map<string, EventSchame> event = 1
}
message ProgramFunctions {
map<string, FunctionSchema> function = 1;
}
message ProgramModels {
map<string, ModelSchema> model = 1;
}
message ProgramFeatures {
map<string, FeatureSchema> feature = 1;
}
message ProgramInferences {
map<string, InferenceSchema> inference = 1;
}
message ProgramDecisions {
map<string, DecisionSchema> inference = 1;
}
message ProgramDecisions {
map<string, DecisionSchema> inference = 1;
}
message ProgramMetrics {
map<string, MetricSchema> inference = 1;
}
```
Customer --> APP --> Add2Cart (CustomerID, ProductSKUID, Quantity)
TransactionEvent -- Add2CartEvent
Maggi - 1 > event1 Add2Cart - key = maagi, value = 1 >> event quantity 1
Maggin- 6 > event2 maggi, value = 5 >> event quantity 5
Maggin- 4 > event3 (DeleteFromCart) >> quantity = -2
pivot - aggegate
enrich = price, brand, category
/customer_cart/productSKU_ID/quantity/value
customer X Cart X Product SKUD
1 X abcd X maggi 1 t1
maggi. 10 t2
maggi. 4 t3
aggegate
customer X Cart X Product SKUD
1 X abcd X maggi 6
Reduce = Group By latest first/last - feature
'add2cart-topic' - lifetime
event1 -t1
event2 - t1+24
event3 - t1+72
redis
customer X cart-id
{
event1/2/3 updated
}
Reduce Java
/custom/average_cart
/customer_sku/add2cart_quantity/value
/customer_sku/add2cart_quantity/value
APP >> Add2Cart = Change Data Capture Kind
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T17:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "1",
"Quantity": "1",
},
{
"ProductSKUId": "2",
"Quantity": "10",
}
]
}
```
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T17:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "1",
"Quantity": "4",
},
{
"ProductSKUId": "2",
"Quantity": "10",
},
{
"ProductSKUId": "3",
"Quantity": "10",
}
]
}
```
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T17:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "1",
"Quantity": "4",
},
{
"ProductSKUId": "2",
"Quantity": "10",
},
{
"ProductSKUId": "3",
"Quantity": "10",
},
{
"ProductSKUId": "1",
"Quantity": "4",
},
{
"ProductSKUId": "2",
"Quantity": "10",
},
{
"ProductSKUId": "3",
"Quantity": "10",
}
]
}
```
Pivot Add2Cart Event Only Take Latest Snapshot
unique timestamp
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T17:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "1",
"Quantity": "1",
},
{
"ProductSKUId": "2",
"Quantity": "10",
}
]
}
```
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T23:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "1",
"Quantity": "2",
},
{
"ProductSKUId": "5",
"Quantity": "10",
}
]
}
```
```
{
"eventID" : "randomKey = timebased"
"CustomerId": "1234567890",
"cart_id": "cart456",
"AddedDate": "2023-11-07T23:30:55.000Z",
"context": {
"device_type": "mobile",
"browser": "Chrome",
"ip_address": "192.168.1.1"
}
"cart-items": [
{
"ProductSKUId": "6",
"Quantity": "1",
},
{
"ProductSKUId": "7",
"Quantity": "10",
}
]
}
```
Google Drive
Gist
Clipboard
Sign in with Wallet
