# Retail B2B Store Warehouse order USECASE

For retail B2B usecase we have store purchase orders.

## Overview

Demand forecasting is done for grocery products in JioMart Partner Fulfillment Centers (Warehouses in TelOS). The forecast is done at the Warehouse-SKU level on weekly basis.

## Schema Onboarding

Schema onboarding will happen via telos workbench(UI). Different kind of schemas needed to onboard:
1. Collections –> Any ordinal/text data should be defined as a set of defined value.

| Collection name | Symbolic/Ordinal | items |
| ---------------- | ---------------- | --------------------------------------------------------------- |
| transaction_type | Symbolic | UPI,CASH,CARD,WALLET |
| zone | Symbolic | East, West, North, South |
| seasons | Ordinal | winter, spring, summer, moonsoon, autumn |
| week_type | Ordinal | weekday, weekend |
| days_of-week | Ordinal | monday, tuesday, wednesday, thursday, friday , saturday, sunday |

<br>

2. Quantity –> All data types needed in the system.

| Quantity | datatype | range |
| ---------- | -------- | ----------------- |
| amount | float | 0 to (float.Max) |
| count | integer | 0 to (int.Max) |
| percentage | float | 0 to 100 |
| is_free | binary | 0 or 1 |
| date | date | dd:mm:yy hh:mm:ss |

3. Entity –> All static data and relationship between different entities will be onboarded.

| Entity | bizid | attributes |
| --------------- | ------------------ | ---------|
| sku | productskuid | |
| store | storeid | |
| warehouse | fc_id | zone|
| category | category_name ||
| class | class_name | |
| group | group_name | |
| department | department_name | |
| pincode | pincode | |
| brand | brandid | |
|article| fc_productid|

| Subject | predicate | Object|
| --------------- | ------------------ | --------|
| sku | belongs_to | category|
| sku | belongs_to | class |
| sku | belongs_to | group|
| sku | belongs_to |department|
| sku | has_brand| brand |
| sku | has_article | article |
| store | is_in | pincode |

4. Event –> POS Transaction event and KeyValue events for aggregation.
Example: Event with Keys as (store * sku) and (warehouse * sku)
> [name=supriyopaul] Example tables for event data for better uderstanding

| Event | Keys | Values |
| --------------- | ------------------ | ---------|
| POS_Transaction | store, sku | transaction_type, amount, count, date |
| | warehouse, sku | transaction_type, amount, count, date |

5. Functions -> Ingestion and feature engineering functions schema will be defined with the help of incoming and outgoing events schema.

<br>

## Knowledge Onboarding

Knowledge data should get onboarded into knowledge graph once schema is created. Biz-ids will get saved to id-garden which is backed by Redis and names will get saved to name-garden(elasticsearch) and all attributes, qualifiers, predicates will be stored in ArangoDb.

<br>

## FUNCTIONS :

### Ingestion :
/function/retail/b2b_sales_order_ingestion
Ingestion will consume raw purchase-order Json coming on Kafka topic and will convert it to standardized event proto.

Steps:
1. Identify incoming Json keys among biz-id, attributes, facets or property with the help of ingestion config.
2. Fetch telos ids for string biz-ids (sku, store, warehouse).

<br><br>


<br><br>

### Compute :
/function/retail/b2b_sales_order_compute
Compute function is responsible for modifying the incoming event by adding new properties or attributes to it. For any kind of arithmetic, logarithmic operation compute will evaluate it using expression engine sdk.
In Demand forecasting use case, we are calculating total sales_price , mrp and discount percentage for each line item.

compute {
total_sales_revenue= count x sales_price
total_mrp = count x mrp
"discount" = ("mrp" -"saleprice")*100/"mrp"
}


<br><br>

### Quantise :
/function/retail/b2b_sales_order_quantise
Quantisation function is responsible for deriving any temporal, spatial , binning kind of information from the existing incoming fields in the events.
For example, from order_date field we can derived further temporal fields like week_type, week_day, month, season etc.

<br>


### Enrich :
/function/retail/b2b_sales_order_enrich
Enrich function is responsible for filling any missing knowledge related data in the incoming events, like predicates and attributes of the entities.
It makes a query to the knowledge Lake to fetch required fields for the incoming entities.
For example, category, brand, pincode are not there in the incoming sales_order event. Enrich fetch it for every sku ( category, brand), store (pincode).

<br>


### Pivot :
/function/retail/b2b_sales_order_pivot
Pivot function is responsible for splitting one event to multiple event based on the key config. The result will be n events having different record keys with same or different value set.
In Demand forecasting use case, we are creating record key -> warehouse * sku, store * sku, store * brand, store * category etc.


<br>
<br>

### Reduce :
/function/reduce/retail/order/warehouse*sku
Reduce function is responsible for doing real time in-memory processing using kafka streams. Incoming events will get aggregated over different time-windows resulting in multiple sink topics.
In Demand forecasting use case, we are having weekly time windows on warehouse * sku as a key. Aggregation we are doing is SUM, AVERAGE, PERCENTAGE, MOVING_AVERAGE.

<br>


<br><br>

## Feature Repository:

Features created by reduce function will get saved to feature repo (Cassandra) as a time series data. The record keys i.e., warehouse and sku are partitioned for fast search and insert. Also, data is sorted by created date which is a cluster key so that finding top N records are efficient.
** Feature Repo will be used for training as well as inferencing of models.

#### List of features getting stores in feature repository in time-series form:

1. /feature/retail/warehouse_x_sku/weekly/count/sum
2. /feature/retail/warehouse_x_sku/weekly/discount/percentage
3. /feature/retail/warehouse_x_sku/weekly/sales_price/average
4. /feature/retail/warehouse_x_sku/weekly/mrp/average
5. /feature/retail/warehouse_x_sku/weekly/amount/sum
6. /feature/retail/warehouse_x_sku/weekly/lag_1/amount
7. /feature/retail/warehouse_x_sku/weekly/lag_2/amount
8. /feature/retail/warehouse_x_sku/weekly/lag_4/amount
9. /feature/retail/warehouse_x_sku/weekly/lag_5/amount
10. /feature/retail/warehouse_x_sku/weekly/moving_average_1/amount
11. /feature/retail/warehouse_x_sku/weekly/moving_average_2/amount

#### Feature repository data storage:

| warehouse | sku | feature | value | timestamp |
| --------- | ----- | ------------------- | ----- | ---------- |
| 10002 | 40001 | count_sum | 50 | 14/05/2023 |
| 10002 | 40001 | discount_perc | 13.5 | 14/05/2023 |
| 10002 | 40001 | sales_price_average | 3450 | 14/05/2023 |
| 10002 | 40001 | amount_sum | 4000 | 14/05/2023 |
| 10002 | 40002 | count_sum | 50 | 14/05/2023 |
| 10002 | 40002 | discount_perc | 13.5 | 14/05/2023 |
| 10002 | 40002 | sales_price_average | 3450 | 14/05/2023 |
| 10002 | 40002 | amount_sum | 4000 | 14/05/2023 |
| 10002 | 40001 | count_sum | 50 | 07/05/2023 |
| 10002 | 40001 | discount_perc | 13.5 | 07/05/2023 |
| 10002 | 40001 | sales_price_average | 3450 | 07/05/2023 |
| 10002 | 40001 | amount_sum | 4000 | 07/05/2023 |

#### Different fetch queries available on feature repository:

1. Fetch Latest(1) features( discount_perc, sales_price_average) for warehouse (10002).
2. Fetch Latest features( discount_perc, sales_price_average) for warehouse (10002) and sku (40001).
3. Fetch Latest N features( discount_perc, sales_price_average) for warehouse (10002).
4. Fetch features( discount_perc, sales_price_average) for warehouse (10002) for date range (01/04/2023 - 30/04/2023)
5. Fetch features( discount_perc) for warehouse (10002) where value > 40.
6. Any custom query in form of CQL (Cassandra Query Language).

### TBD

Usecase:

Any Fetch operation - leads to query operation

Usecase: Training Fetch
Model: Demand forecasting for warehouse * sku

Data required:
warehouse*sku - record_key, (window - weekly)

/feature/retail/warehouse_x_sku/weekly/count/sum
/feature/retail/warehouse_x_sku/weekly/discount/percentage
/feature/retail/warehouse_x_sku/weekly/sales_price/average
/feature/retail/warehouse_x_sku/weekly/mrp/average
/feature/retail/warehouse_x_sku/weekly/amount/sum
/feature/retail/warehouse_x_sku/weekly/lag_1/amount
/feature/retail/warehouse_x_sku/weekly/lag_2/amount
/feature/retail/warehouse_x_sku/weekly/lag_4/amount
/feature/retail/warehouse_x_sku/weekly/lag_5/amount
/feature/retail/warehouse_x_sku/weekly/moving_average_1/amount
/feature/retail/warehouse_x_sku/weekly/moving_average_2/amount

/attribute/retail/warehouse/location/lat_long
/attribute/retail/warehouse/capacity
/predicate/retail/warehouse/state
/predicate/retail/sku/category
/predicate/retail/sku/brand

Training Fetch data: The above data points needs to be fetched from feature and knowledge for last week.

@startuml
entity "Warehouse" as warehouse {
  + fc_id : uint64
  --
  location : spatial
  capacity : uint32
  state : symbolic
  zone: symbolic
}

entity "SKU" as sku {
  + productskuid : uint64
  --
  category : uint64
  class : uint64
  group : uint64
  department : uint64
  brand : uint64
  article : uint64
}

entity "Warehouse x SKU" as Warehouse_x_SKU {
  + fc_id : uint64
  + sku_id : uint64
  + timestamp : temporal
  --
  window : temporal
  count_sum : f32
  discount_percentage : f32
  sales_price_average : f32
  mrp_average : f32
  amount_sum : f32
  lag_1_amount : f32
  lag_2_amount : f32
  lag_4_amount : f32
  lag_5_amount : f32
  moving_average_1_amount : f32
  moving_average_2_amount : f32
}

warehouse ||--o{ Warehouse_x_SKU : has
sku ||--o{ Warehouse_x_SKU : has
@enduml