# Industrial Computer Vision at the Edge
This page lists publically available information on practical applications of computer vision machine learning solutions at the edge, with an industrial context.

Let's start with a few definitions. There is lots of hype in this space. Here we are focusing on pragmatic and useful use cases.

## Definitions
- Artificial Intelligence (AI) - think mimicing human intelligence activities. NOT the 'rise of the machines' that subjugate humans.
- Machine Learning (ML) - This is a good STEM or science based description of the tools, algorithms and technologies
- Deep Learning (DL) - No predefined features like traditional feature based ML (think chess board and game playing where we predefine the approach, goal and rules of the game) . DL is where the model decides what is of interest.
- Computer Vision (CV) - visual based ML
- Natural Language Processing (NLP) - text based ML
Large Language Models (LLM) - Recent advances in how much data and how it can be ingested massively parallel into DL models has led the current interest around AI and ML. Here we're limiting ourselves to CV at the edge.
- Advances are happening quickly. Acronyms, terms, models, data set size and scope etc are being obsoleted on a daily basis. But the fundamentals don't change. The machines are getting better at learning, so I'll use the term ML broadly.

## CV Use Cases
CV Use cases that have are of interest to industrial customers include:
- Inspect packaging and palleting
- Workplace safety
- Defect detection
- Real time process monitoring

## NLP Use Cases
Not applicable here. We're focussing on CV.

## Generative AI Use Cases
Not applicable here. While we aren't looking to 'generate' new visual outputs re industrial CV; we may consider Generative AI as a means to generate synthetic training data. But out of scope for this post.

# Ethics, Constraints and Limitations
- Ethics around the use of AI and the data used, consumed or generated or often vague, controversial or confusing. We'll limit ourselves to use cases that aren't ethically controversial.
- Constraints include access to training data specific to your use case (aka Ground Truth) and ML inference in a practical production grade way. (Model Training versus ML Inference)
- Limitations - Some industrial use cases require other data (other than CV data) to augment CV ML.
- MLOps is important; and if you don't consider MLOps then it can be challenging to get out of a simple experiment and into production delivering business value. Traditional software development processes are still important but the the blending of Data Engineering, ML and DevOps is a bigger picture; often referred to as MLOps. https://en.wikipedia.org/wiki/MLOps

# Demo
This section is a summary of the steps to demo several use cases. The detail is in the links.

The demos using Amazon Sagemaker and setting up AWS Panorama are not live. It takes tens of minutes or sometimes hours to train models. We can see how things are built, deployed and configured by the verbose output in the Jupyter notebooks. We can also skip back and forth to answer QandA.

## Industrial automation at Tyson with computer vision, AWS Panorama, and Amazon SageMaker
This AWS Machine Learning Blog post is in two parts.
- Part one https://aws.amazon.com/blogs/machine-learning/industrial-automation-at-tyson-with-computer-vision-aws-panorama-and-amazon-sagemaker/
- Part two https://aws.amazon.com/blogs/machine-learning/computer-vision-based-anomaly-detection-using-amazon-lookout-for-vision-and-aws-panorama/

### Usage and Limitations
- Describes a production grade CV use case for an industrial customer. Part one is an inventory counting use case and part two uses CV anomoly detection for predictive maintenance.
- The model and infrastructure used in these examples is customer IP, so not accessible for reuse
- Key data about scaling and the business value of this approach is described. Useful in building a CV business case.

## AWS Panorama Immersion Day
This hands on workshop is publically available. It allows you to do the prep work to build and deploy a CV ML inference model to a virtual edge device (an EC2 instance) or to a physical AWS Panorama device.

https://catalog.workshops.aws/panorama-immersion-day/en-US

Here we have built:
- Amazon Sagemaker Environment using Amazon Cloudformation
- We've accessed our Jupyter Notebook in Amazon Sagemaker
- We've run all the steps sequentially in the Jupyter Notebook
- Initially we've setup the underlying compute powering Amazon Sagemaker, updated libraries and downloaded the videos we are running ML inference on.
- Then we can run one or multiple versions of our model training, validation (inference) and deployment from our use case notebook
- When finished we can stop the Amazon Sagemaker compute, version control changes and reduce idle costs
- If needed we can tear everything down and rebuild.

### Usage and Limitations
This workshop:
- uses an AWS Cloudformation template to build the Amazon Sagemaker Jupyter Notebook instance. The Jupyter notebook provides a verbose step by step guided journey.
- The AWS Cloudformation template allows you to quickly build, tear down and clone the 'scaffolding' pieces needed to experiment.
- There are four labs in the notebook and these can be used to scaffold your POC and to swap out CV models
- For scaling to production and multiple devices there are more efficient ways to automate. But this is a good place to begin your edge CV experiment.

## Labeling mixed-source, industrial datasets with Amazon SageMaker Ground Truth
Like any big data use case, obtaining, generating or cleansing input data takes effort. In the ML space the term 'ground truth' is commonly used to describe our 'trusted' or 'source of truth' for training or validating our CV ML solutions. In this AWS Machine Learning Blog post we have a comprehensive and step by step overview of prepping quality data for training and validating.

https://aws.amazon.com/blogs/machine-learning/labeling-mixed-source-industrial-datasets-with-amazon-sagemaker-ground-truth/

### Usage and Limitations
- This example focusses on welding quality control. Most practical industrial use cases are unique to you and your business. This might be useful internally or you might productise and share / sell / distribute your IP. Something to consider from a new busines context, perhaps.
- This example is two years old (circa Jan 2021) so there have been new features and services that make this type of task easier. If this of interest then we can leverage additional AWS SMEs.

## AWS Panorama Edge Appliance and SDK
Here I'll briefly talk about AWS and Partner Edge Hardware
- AWS Panorama - edge ML CV appliance that is designed to scale securely in production environments. The appliance is a zero trust, ruggedised appliance with GPU inference. It's only configurable from within an AWS environment. There is no hardware access other than to provision.
- AWS Panorama - also has an SDK which allows more flexibility on how you operate your edge hardware
- AWS Technology partners also have hardware options

The following presentations (not publically accessible) may be used to answer QandA:
- AWS Panorama – Network and Security
- AWS Panorama deep dive