# Video Template
###### tags: `whilter`
> validate() method need to be implemented to check mlt file as well as template metadata. it has two parts - validate mlt file sturcture and template metadata
> ML process will generated dynamic content, that need to be validated
## Input Request
### API Request
```json
{
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"user-id": "<user-id>",
"reference-record-id": "<record-id-for-enrich>",
"qr-code" : [
"qr-code-text-1"
],
"phrase-text" : [
"Rajan",
"A Happy New Year"
],
"video-text" : [
"30% off", "Mumbai", "Delhi"
],
"single-value-chart" : [
12, 18.84, 175, 4.7
],
"image-uri" : [
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image1.jpeg",
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image2.jpeg"
],
"video-uri" : [
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/videos/video1.mp4"
]
}
```
### Email Request
Images and videos will come as attachment. Natural of name of files will be used for indexing images and videos
The table will be similar to below but without **Notes** column
#### HTML table in body of email
| Key | Value | Notes |
| ----------- | ----------- |----------- |
| client-id | Domino | **Required** - - It is needed but can be derived from source email. So optional for email table |
| template-id | template-001 | **Required** - It is needed but can be derived for specific client e.g. default template. So optional for email table |
| user-id | 97284763839 | **Required** it can be email address or other client specific primary key |
| reference-record-id | uuid-for-enrich | **Optional** if data need to be feteched from client system|
| qr-code | some-pnr-text | **Optional** |
| phrase-text | "Rajan", "A Happy New Year" | **Optional** |
| video-text | "30% off", "Mumbai", "Delhi" | **Optional** |
| single-value-chart | 12, 18.84, 175, 4.7 | **Optional** |
| email-destination | "rajan@racloop.com" | **Optional** one of destination is mandatory - email, whatsapp, sms |
| whatsapp-destination | 97284763839 | **Optional** one of destination is mandatory - email, whatsapp, sms |
| sms-destination | 97284763839 | **Optional** one of destination is mandatory - email, whatsapp, sms |
### Whatsapp Request
Images and videos will come as attachment. Natural of name of files will be used for indexing images and videos
#### Key value in body of whatsapp message
| Key | Value | Notes |
| ----------- | ----------- |----------- |
| client-id | Domino | **Required** - - It is needed but can be derived from source email. So optional for email table |
| template-id | template-001 | **Required** - It is needed but can be derived for specific client e.g. default template. So optional for email table |
| phrase-text | "Rajan", "A Happy New Year" | **Optional** |
| video-text | "30% off", "Mumbai", "Delhi" | **Optional** |
| single-value-chart | 12, 18.84, 175, 4.7 | **Optional** |
* user-id : same number of sender
* whatsapp-destination : whatsapp number of sender
### Schema
All input data should have definition or schema associated with input request.
> During phase 1 each schema will be added manually to redis.
```json
[
{
"schema-id" : "qr-code-xxxx-1",
"name" : "Indigo PNR",
"data-type" : "DATA_TYPE_STRING",
"output-type" : "OUTPUT_TYPE_QR_CODE",
"metadata" : {
"primary-label" : "PNR",
"qr-code-width" : 100,
"qr-code-height" : 100
}
},
{
"schema-id" : "phrase-xxxx-1",
"name" : "phrase-1",
"data-type" : "DATA_TYPE_STRING",
"output-type" : "OUTPUT_TYPE_PHRASE_TEXT",
"metadata" : {
"template" : "Hello ${name}, Wishing you ${festival}",
"reference-video-path" : "uri",
"reference-audio-text" : "Hello Shabu, Wishing you a happy new year",
"reference-audio-path" : "uri",
"model_path_voice_clone" : "uri",
"model_path_lip_sync" : "uri"
}
},
{
"schema-id" : "image-xxxx-1",
"name" : "image data",
"data-type" : "DATA_TYPE_URI",
"output-type" : "OUTPUT_TYPE_IMAGE",
"metadata" : {
"tags" : ["person", "bike", "smiling"]
}
},
{
"schema-id" : "video-xxxx-1",
"name" : "video-data",
"data-type" : "DATA_TYPE_URI",
"output-type" : "OUTPUT_TYPE_VIDEO",
"metadata" : {
"tags" : ["person", "bike", "smiling"]
}
},
{
"schema-id" : "video-text1-xxxx-1",
"name" : "video-text",
"data-type" : "DATA_TYPE_STRING",
"output-type" : "OUTPUT_TYPE_VIDEO",
"metadata" : {
"text-max-length" : 12
}
},
{
"schema-id" : "chart-xxxx-1",
"name" : "Erythrocyte Sedimentation Rate",
"data-type" : "DATA_TYPE_DOUBLE",
"output-type" : "OUTPUT_TYPE_CHART",
"metadata" : {
"chart-type" : "gauge",
"primary-label" : "ESR",
"secondary-label" : "Modified Westergren",
"units" : "mm/hr",
}
"range" : [
{
"lowValue" : 0,
"highValue" : 20,
"name" : "Low",
"audio" : [
{
"language" : "en",
"audio-uri" : "s3_url_of_audio"
},
{
"language" : "hi",
"audio-uri" : "s3_url_of_audio"
}
]
},
{
"lowValue" : 20,
"highValue" : 70,
"name" : "Medium",
"audio" : [
{
"language" : "en",
"audio-uri" : "s3_url_of_audio"
},
{
"language" : "hi",
"audio-uri" : "s3_url_of_audio"
}
]
},
{
"lowValue" : 70,
"highValue" : 100,
"name" : "High",
"audio" : [
{
"language" : "en",
"audio-uri" : "s3_url_of_audio"
},
{
"language" : "hi",
"audio-uri" : "s3_url_of_audio"
}
]
}
]
},
{
"schema-id" : "pi-xxxx-1",
"name" : "Population Demography",
"data-type" : "DATA_TYPE_DOUBLE",
"output-type" : "OUTPUT_TYPE_CHART",
"metadata" : {
"chart-type" : "pi",
"primary-label" : "Population Demography",
"classes" : ["red", "green", "blue"]
}
},
{
"schema-id" : "bar-xxxx-1",
"name" : "Metro Population",
"data-type" : "DATA_TYPE_DOUBLE",
"output-type" : "OUTPUT_TYPE_CHART",
"metadata" : {
"chart-type" : "bar",
"primary-label" : "Metro Population (in Million)",
"classes" : ["Delhi", "Mumbai", "Chennai"]
"x-label" : "Cities",
"y-label" : "Population"
}
},
{
"schema-id" : "line-xxxx-1",
"name" : "Delhi Population (in Million)",
"data-type" : "DATA_TYPE_DOUBLE",
"output-type" : "OUTPUT_TYPE_CHART",
"metadata" : {
"chart-type" : "line",
"primary-label" : "Delhi Population (in Million)",
"x-label" : "Year",
"y-label" : "Population"
}
}
]
```
1. **schema-id :** Primary key
2. **name :** Name of schema. Can be used in display e.g. chart name
3. **data-type :** Data type of input request data
4. **output-type :** Define output - QR, Chart, Phrase etc
5. **metadata :** More information about data
## Template Metadata
> During phase 1 it will be added manually to redis
```json
{
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"template_url" : "s3 url",
"language-code" : "en",
"tts-voice" : "name-as-per-google",
"chart-metadata" : {
"color-palette" :[ "#F44336", "#448AFF", "#7B1FA2"],
"background-color" : "#FFFFFF",
"chart-groups" : [
{
"group-id" : "group-xxxx-01",
"group-name" : "Vitamin Profile",
"group-description" : "Vitamins perform many essential roles in your body and maintain your overall health.",
"charts" : ["chart-xxxx-1", "chart-xxxx-2"],
"notes" : [
{
"type" : "diet",
"value" : "A balanced diet can take care of all the vitamins needed by your body"
},
{
"type" : "doctor-info",
"value" : "Consult your doctor before taking any vitamin supplements"
}
]
}
]
},
"request-mapping" : [
{
"output_type" : "OUTPUT_TYPE_QR_CODE",
"index" : 0,
"melt-producer" : "producer3",
"schema-id" : "qr-xxxx-1",
"constant-duration" : true
},
{
"output_type" : "OUTPUT_TYPE_IMAGE",
"index" : 0,
"melt-producer" : "producer4",
"schema-id" : "image-xxxx-1",
"constant-duration" : true
},
{
"output_type" : "OUTPUT_TYPE_IMAGE",
"index" : 1,
"melt-producer" : "producer5",
"schema-id" : "image-xxxx-1",
"constant-duration" : true
},
{
"output_type" : "OUTPUT_TYPE_VIDEO",
"index" : 0,
"melt-producer" : "producer6",
"schema-id" : "video-xxxx-1",
"constant-duration" : true
},
{
"output_type" : "OUTPUT_TYPE_PHRASE_TEXT",
"index" : 0,
"melt-producer" : "producer8",
"schema-id" : "phrase-xxxx-1",
"constant-duration" : false,
"index" : 0
},
{
"output_type" : "OUTPUT_TYPE_PHRASE_TEXT",
"index" : 1,
"melt-producer" : "producer8",
"schema-id" : "phrase-xxxx-1",
"constant-duration" : false,
"index" : 1
},
{
"output_type" : "OUTPUT_TYPE_CHART",
"index" : 1,
"schema-id" : "chart-xxxx-1",
"multiple" : true,
"reference-audio-producer" : "producer14",
"placeholder-producer" : "producer12"
}
],
"enrich-mapping" : [
{
"output_type" : "OUTPUT_TYPE_CHART",
"index" : 1,
"schema-id" : "chart-xxxx-1",
"multiple" : true,
"reference-audio-producer" : "producer14",
"placeholder-producer" : "producer12"
}
],
"destination-config": [
{
"type" : "email",
"email-template" : "path to html",
"smtp-host" : "smtp.racloop.com",
"port" : 21,
"user-key" : "",
"user-secret" : ""
},
{
"type" : "whatsapp",
"template-id" : "text template - TBD"
},
{
"type" : "sms",
"template-id" : "text template - TBD",
"sms-url" : "http://name-of-host.com ",
"user-key" : "",
"user-secret" : ""
}
]
}
```
**language-code :** ISO 639.2 https://www.loc.gov/standards/iso639-2/php/code_list.php
**tts-voice :** https://cloud.google.com/text-to-speech/docs/voices
**color-palette :** different colors used in charts
**mapping :** producer mapping for melt
1. **request-index :** identify data element in request
2. **melt-producer :** producer reference in melt file
3. **schema-id :** schema definition in redis
4. **index :** in case of chart (multiple charts on same screen) and phrase - different input will have same destination producer.
5. **constant-duration :** fixed duration producer in melt
6. **multiple :** generated content at multiple places (charts and phrase replacement)
## :memo: Template (.mlt file)
```xml
<?xml version='1.0' encoding='utf-8'?>
<mlt LC_NUMERIC="C" producer="main_bin" version="7.13.0" root="C:/Users/Suhani/Desktop/Project- PVR/Project">
<profile frame_rate_num="25" sample_aspect_num="1" display_aspect_den="16" colorspace="709" progressive="1" description="Vertical Suhani" display_aspect_num="9" frame_rate_den="1" width="1080" height="1920" sample_aspect_den="1"/>
<consumer f="mp4" g="15" channels="2" crf="23" target="C:/Users/Suhani/Videos/PVR829.2.mp4" mlt_service="avformat" real_time="-1" threads="0" vcodec="libx264" ab="160k" movflags="+faststart" preset="veryfast" acodec="aac" in="0" out="509"/>
<producer id="producer0" in="00:00:00.000" out="00:00:36.560">
<property name="length">915</property>
<property name="eof">pause</property>
<property name="resource">C:/Users/Suhani/Downloads/qr.png</property>
<property name="ttl">25</property>
<property name="aspect_ratio">1</property>
<property name="progressive">1</property>
<property name="seekable">1</property>
<property name="format">1</property>
<property name="meta.media.width">225</property>
<property name="meta.media.height">225</property>
<property name="mlt_service">qimage</property>
<property name="kdenlive:clipname">QR ~Replace</property>
<property name="kdenlive:duration">00:00:05.000</property>
<property name="kdenlive:folderid">-1</property>
<property name="kdenlive:clip_type">2</property>
<property name="kdenlive:id">2</property>
<property name="kdenlive:file_size">3825</property>
<property name="kdenlive:file_hash">fc91d55ffaca12cb622909d8f1f2ac6d</property>
</producer>
<producer id="producer1" in="00:00:00.000" out="00:00:39.560">
<property name="length">990</property>
<property name="eof">pause</property>
<property name="resource">C:/Users/Suhani/Desktop/Project- PVR/Logo/PVR-logo.png</property>
<property name="ttl">25</property>
<property name="aspect_ratio">1</property>
<property name="progressive">1</property>
<property name="seekable">1</property>
<property name="format">2</property>
<property name="meta.media.width">300</property>
<property name="meta.media.height">300</property>
<property name="mlt_service">qimage</property>
<property name="kdenlive:clipname">PVR</property>
<property name="kdenlive:duration">00:00:05.000</property>
<property name="kdenlive:folderid">-1</property>
<property name="kdenlive:clip_type">2</property>
<property name="kdenlive:id">3</property>
<property name="kdenlive:file_size">6411</property>
<property name="kdenlive:file_hash">e0c8fa5405c28125588aab37ba639751</property>
</producer>
</mlt>
```
## Example Pipeline (API)
### Client Request
```json
{
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"user-id": "<user-id>",
"reference-record-id": "<record-id-for-enrich>",
"qr-code" : [
"qr-code-text-1"
],
"phrase-text" : [
"Rajan",
"A Happy New Year"
],
"video-text" : [
"30% off", "Mumbai"
],
"single-value-chart" : [
12, 18.84
],
"image-uri" : [
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image1.jpeg",
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image2.jpeg"
],
"video-uri" : [
"https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/videos/video1.mp4"
]
}
```
### Event on ingestion output topic
The above request will be converted to Event object.
```json
{
"request-uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"user-id": "<user-id>",
"reference-record-id": "<record-id-for-enrich>",
"event-created-time" : "2023-11-25 11:30",
"client-process-status" : {
"uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"status" : "PIPELINE_STATUS_REQUEST_LOGGED"
},
"dag" : [
{"enrich" : true },
{"compute" : true },
{"qr-code" : true },
{"chart" : true},
{"downloader" : true},
{"recognition" : true},
{"tts" : true},
{"voice-clone" : true},
{"lip-sync" : true},
{"melt" : true},
{"notification" : true}
],
"ingress_type" : "INGRESS_TYPE_API",
"client_request_data" : [
{
"index" : 0,
"schema-id" : "qr-code-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_STRING"
"string_value" : "qr-code-text-1"
}
},
{
"index" : 0,
"schema-id" : "phrase-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_STRING"
"string_value" : "Rajan"
}
},
{
"index" : 1,
"schema-id" : "phrase-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_STRING"
"string_value" : "A Happy New Year"
}
},
{
"index" : 0,
"schema-id" : "video-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_STRING"
"string_value" : "30% off"
}
},
{
"index" : 1,
"schema-id" : "video-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_STRING"
"string_value" : "Mumbai"
}
},
{
"index" : 0,
"schema-id" : "chart-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_DOUBLE"
"double_value" : 12
}
},
{
"index" : 1,
"schema-id" : "chart-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_DOUBLE"
"double_value" : 18.84
}
},
{
"index" : 0,
"schema-id" : "user_image_1",
"data" : {
"data-type" : "DATA_TYPE_URI"
"string_value" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image1.jpeg"
}
},
{
"index" : 1,
"schema-id" : "user_image_1",
"data" : {
"data-type" : "DATA_TYPE_URI"
"string_value" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image2.jpeg"
}
},
{
"index" : 0,
"schema-id" : "user_video_1",
"data" : {
"data-type" : "DATA_TYPE_URI"
"string_value" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/video1.mpeg"
}
}
]
}
```
### Event on enrich output topic
```json
{
"uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"user-id": "<user-id>",
"reference-record-id": "<record-id-for-enrich>",
"event-created-time" : "2023-11-25 11:30:30",
"client-process-status" : {
"uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"status" : "PIPELINE_STATUS_ENRICH_COMPLETE",
"update_time" : "2023-11-25 11:31:00"
},
"ingress_type" : "INGRESS_TYPE_API",
"client_request_data" : ["same as above"],
"enrich_data" : {
"reference-record-id": "<record-id-for-enrich>",
"reference-data" : [
{
"index" : 0,
"schema-id" : "chart-xxxx-1",
"data" : {
"data-type" : "DATA_TYPE_DOUBLE"
"double_value" : 175
}
}
]
}
}
```
### Event on compute output topic
Compute will do most of calculation. It will take client request data, enriched data, template metadata and mlt file to create event that is ready for processsing
```json
{
"uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"client-id": "<Client Id>",
"template-id": "<template-uuid>",
"user-id": "<user-id>",
"reference-record-id": "<record-id-for-enrich>",
"event-created-time" : "2023-11-25 11:30:30",
"client-process-status" : {
"uuid" : "0a6eff19-6483-438b-9fe5-8a83b22d2cfd",
"status" : "PIPELINE_STATUS_COMPUTE_COMPLETE",
"update_time" : "2023-11-25 11:31:30"
},
"ingress_type" : "INGRESS_TYPE_API",
"client_request_data" : ["same as above"],
"enrich_data" : ["same as above"],
"template_data" : [
{
"output_type" : "OUTPUT_TYPE_QR_CODE",
"producer-id" : "producer3",
"data" : "qr-code-text-1"
},
{
"output_type" : "OUTPUT_TYPE_PHRASE_TEXT",
"producer-id" : "producer8-0",
"text-template" : "Hello ${name}, Wishing you ${festival}",
"phrase-text" : ["Rajan", "a happy new year"],
"user-text" : "Hello Rajan, Wishing you a happy new year",
"reference_video_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/videos/reference-video1.mp4",
"reference-audio-text" : "Hello Shabu, Wishing you a happy new year",
"reference_audio_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/audio/reference-video1.mp3",
"model_path_voice_clone" : "uri",
"model_path_lip_sync" : "uri"
},
{
"output_type" : "OUTPUT_TYPE_VIDEO_TEXT",
"producer-id" : "producer12",
"value" : "30% off"
},
{
"output_type" : "OUTPUT_TYPE_CHART",
"producer-id" : "producer14-0",
"group-id" : "group-xxxx-01",
"title" : "Diabetes Monitoring",
"subtitle" : "Diabetes is a condition where your blood glucose or sugar is too high. Untreated diabetes (high blood sugar) can silently (without any observable symptoms) damage your blood vessels, heart, kidney, eyes etc. These tests help diagnose diabetes and give some estimation of your future risk of developing diabetes.",
"chart" : [
{
"schema-id" : "xxxx-1",
"chart-type" : "gauge",
"title" : "Glycosylated Hemoglobin (HbA1c)",
"subtitle" : "HPLC",
"units" : "%",
"value" : 50
},
{
"schema-id" : "xxxx-1",
"title" : "Test Name - 1",
"subtitle" : "Optional",
"chart-type" : "gauge",
"value" : 50
}
],
"audio" : {
"producer-id" : "producer18-0",
"output_type" : "OUTPUT_TYPE_AUDIO",
"bucket_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/audio/audio-01.wav"
}
},
{
"output_type" : "OUTPUT_TYPE_IMAGE",
"producer-id" : "producer20",
"bucket_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image1.jpeg"
},
{
"output_type" : "OUTPUT_TYPE_IMAGE",
"producer-id" : "producer21",
"bucket_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/images/image2.jpeg"
},
{
"output_type" : "OUTPUT_TYPE_VIDEO",
"producer-id" : "producer24",
"bucket_path" : "https://amplify-whilter-dev--193849-deployment.s3.ap-south-1.amazonaws.com/videos/video1.mp4"
}
]
}
```
### Event on compute QR code output topic
Rest of the event will remain same. Only **local_file_path** get added for generated qr code.
```json
{
"template_data" : [
{
"producer-type" : "PRODUCER_TYPE_QR_CODE",
"producer-id" : "producer3",
"data" : "qr-code-text-1",
"local_file_path" : "/root/workspace/template-1/request-uuid/producer1/qr.png"
}
]
}
```
### Event on chart output topic
Rest of the event will remain same. Only **local_file_path** get added after generation of chart image.
```json
{
"template_data" : [
{
"producer-type" : "PRODUCER_TYPE_CHART",
"producer-id" : "producer14-0",
"title" : "Kidney Function Test",
"subtitle" : "some text",
"local_file_path" : "/root/workspace/template-1/request-uuid/producer14/chart-1.png",
"chart" : [
{
"chart-id" : "xxxx-1",
"title" : "Test Name - 1",
"subtitle" : "Optional",
"chart-type" : "gauge",
"value" : 50
},
{
"chart-id" : "xxxx-2",
"title" : "Test Name - 2",
"subtitle" : "Optional",
"chart-type" : "pi",
"value" : [50, 20, 70]
}
],
"audio" : {
"producer-id" : "producer18-0",
"text" : "chart discription"
}
},
{
"producer-type" : "PRODUCER_TYPE_CHART",
"producer-id" : "producer14-1",
"title" : "Kidney Function Test",
"subtitle" : "some text",
"local_file_path" : "/root/workspace/template-1/request-uuid/producer14/chart-2.png",
"chart" : [
{
"chart-id" : "xxxx-1",
"title" : "Test Name - 3",
"subtitle" : "Optional",
"chart-type" : "bar",
"value" : [50, 20, 70, 30]
},
{
"chart-id" : "xxxx-2",
"title" : "Test Name - 4",
"subtitle" : "Optional",
"chart-type" : "line",
"value" : [21, 26, 28, 30]
}
],
"audio" : {
"producer-id" : "producer18-1",
"text" : "chart discription"
}
}
}
```
Sign in with Wallet
Connect another wallet