# DATA WAREHOUSE

## Definition
- **Products and Association Rules**: a set of patterns or relationships that can be discovered in large datasets. These rules identify the co-occurrence or correlations between different items in a dataset.
- Association rules are used to identify relationships between items in a dataset. They represent if-then statements with 2 parts:
- **Antecedent**: A set of items in the data
- **Consequent**: A set of items FOUND in combination Antecedent in the data
- **Metrics**: (First 2 are the main)
- **Support**:  the total number of transactions in which the itemset occurs.
$Support(X => Y)= \frac{\text{Transactions have X and Y}}{\text{Total Transactions}}$

- **Confidence**: the conditional probability of occurrence of consequent given the antecedent
$Confidence(X => Y)= P(Y|X)=\frac{\text{Transactions have X and Y}}{\text{Transactions have X}}$

- **Lift**:  the rise in probability of having {Y} on the cart with the knowledge of {X} being present over the probability of having {Y} on the cart without any knowledge about presence of {X}
- \> 1: X increase the chance of buying Y
- < 1: Otherwise
- = 1: unknown

$Lift(X => Y)= \frac{P(Y|X)}{P(Y)}=\frac{\frac{\text{Transactions have X and Y}}{\text{Transactions have X}}}{\frac{\text{Transactions have Y}}{\text{Total Transactions}}}$
## Use Cases
- By using association rules, businesses can gain insights into customer behavior, identify cross-selling opportunities, optimize product placements, and improve overall marketing strategies.
- **Market basket analysis** Association rules can identify complementary products or services that can be recommended to customers to increase sales. Retailers can collect data about purchasing patterns, recording purchase data as item barcodes are scanned by point-of-sale systems. Machine learning models can look for co-occurrence in this data to determine which products are most likely to be purchased together. The retailer can then adjust marketing and sales strategy to take advantage of this information. One example of association rule mining in retail stores is the famous “beer and diapers” rule. This rule states that customers who buy diapers are more likely to buy beer than the average customer. This rule was reportedly discovered by analyzing the sales data of a supermarket chain in the US. The possible explanation for this rule is that young fathers who are sent to buy diapers for their babies may also buy beer for themselves as a reward or a stress relief.
- **Customer behavior analysis:** Association rules can reveal patterns in customer behavior, such as preferences or purchasing habits, which can be used for personalized marketing and product recommendations. Developers can collect data on how consumers use a website they create. They can then use associations in the data to optimize the website user interface -- by analyzing where users tend to click and what maximizes the chance that they engage with a call to action, for example.
- **Medical diagnosis:** Association rules can be used to analyze patient data and identify potential relationships between symptoms, diseases, and treatments. Doctors can use association rules to help diagnose patients. There are many variables to consider when making a diagnosis, as many diseases share symptoms. By using association rules and machine learning-fueled data analysis, doctors can determine the conditional probability of a given illness by comparing symptom relationships in the data from past cases. As new diagnoses get made, machine learning models can adapt the rules to reflect the updated data. Examples of association rules that can be mined from medical diagnostic data can inlcude: Patients who have chest pain and high blood pressure are more likely to have heart disease than the average patient, patients who have diabetes are more likely to need insulin than the average patient or patients who have cancer and undergo chemotherapy are more likely to experience hair loss than the average patient.
- **Fraud detection:** Association rules can help identify suspicious patterns or correlations in data that may indicate fraudulent activities. Some examples of association rules that can be mined from fraud detection data may include: Transactions that involve online purchases with high amounts from foreign countries are more likely to be fraudulent than the average transaction, claims that involve the same parties within a short time interval are more likely to be fraudulent than the average claim and transactions that involve credit cards with low balances and multiple merchants are more likely to be fraudulent than the average transaction.
## Case Study: Amazon

As one of the largest e-commerce companies, Amazon heavily relies on association rules to enhance its product recommendations. By analyzing vast amounts of customer data, including browsing history and purchase behavior, Amazon leverages association rules to suggest related products to customers. Amazon uses association rule mining for various purposes, such as:
- **Recommending products to customers based on their previous purchases or browsing history**. For example, if a customer buys a laptop, Amazon can recommend a mouse and a keyboard based on the association rules that involve laptops and accessories.
- **Offering discounts or promotions on products that are frequently bought together by customers**. For example, if a customer buys a book, Amazon can offer a discount on another book that is related to the first one based on the association rules that involve books and genres.
- **Placing products that are often bought together near each other in the website layout or on the same web page**. For example, if a customer searches for a camera, Amazon can display related products such as memory cards, tripods, or lenses based on the association rules that involve cameras and equipment.

## How can Data Warehouse and DSS be used with Association Rules

- **Data Warehouse:** Store transactional data, customer information, and other relevant data needed for association rule mining. With a properly designed data warehouse, businesses can extract and transform transaction data into a format suitable for association rule mining algorithms. This accelerates the process of deriving valuable association rules from the data.

- **Decision Support System (DSS):** Can utilize association rules to support decision-making related to product recommendations, marketing strategies, and inventory management. For example, a DSS can generate product recommendations based on association rules to help businesses cross-sell or upsell products to customers. It can provide insights on which products are frequently purchased together and suggest strategies to optimize product placement or marketing campaigns. Additionally, a DSS can help in inventory management by identifying associations between products and assisting in supply chain management decisions, such as stock replenishment and forecasting demand.

- By integrating association rules generated from the data warehouse into a DSS, businesses can make data-driven decisions and take advantage of the discovered patterns and relationships between products. This enables businesses to optimize their strategies, increase sales, and enhance customer satisfaction.

## Problem Description
- **User Story**: I am the manager of a bike shop which sells bicycles, components, sports clothing and accessories. My goal is to maximize the shop's profit and improve overall customer experience. To achieve that, I need a system which can analyze the sales data of all products. From the analysis, the system can suggest an optimal way to manage the product inventory, hence motivating customers to purchase additional products during their shopping experience. I also want to the system to suggest promotion strategy to further encourage customer to spend on our products.
- **System Requirements**
- The system can analyze historical sales data and customer purchase patterns to identify associations between products.
- The system should generate reports or visualizations to help the manager measure the performance and effectiveness of the inventory recommendation system.
- The system should suggest product’s placement based on their association rules while taking into consideration the current inventory, pricing, and shelf space constraints.
- The system should provide suggestions for promotional activities such as discounts, bundle deals, limited-time offers, or loyalty programs
- The system should be flexible to adapt with changes in customer behavior and market trends
- The system should have an intuitive user interface that allows the manager to easily access and interpret the recommendations.
- The system should provide the option to customize the association rules and fine-tune the recommendation algorithm based on the specific needs and context of the bike shop

## Technology Survey
- Association Rules Mining Algorithms
- AIS
- FP-Growth
- Input:
- Output:
- Flow:
- Apriori
- All subsets of a frequent itemset must be frequent(Apriori property).
- If an itemset is infrequent, all its supersets will be infrequent.
- Input: Dataset, minimum support count, minimum confidence
- Output: A set of association rules
- Flow:
- Each iteration will generate itemsets with a predefined number, remove one whose support is lower the the minimum support count => Construct a table
- The subset of itemset in each iteration must be included in previous iteration tables
- Stop when all generated itemset have support < minimum support count
- After last iteration, define rules
- S is a subset of I
- Rule: S -> (I-S)
- Calculate Confidence
- Software to build DWH + DSS
- RDBMS
- ETL Tools
- BI & Analytics Tools
- Data Mining and Statistical Analysis Tools:
- OLAP Tools
- Related Work
- "A Decision Support System for Product Inventory Management Using Association Rule Mining" by Smith et al. (2018)
- "Association Rule-Based Product Recommendation System for Online Retailers" by Johnson et al. (2016)

- A Decision Support System for Product Inventory Management
- ABC classification:
- Redesigning a Retail Store Based on Association Rule Mining
- The paper describe an approach to redesign the store layout based on association rules. From a dataset of transactions last 6 months of a supermaket, authors first categorized products. They then applied Apriori and Éclat to deduce the association rules between categories. From the association rules, the paper suggests changing the store layout by letting categories that has positive corelation close to each other. It also suggests the different approach to redessign the store layout: the higher the support value of the category, and the less space it takes up, the closer to the door it should be placed. Two approaches are then evaluated by the distance traveled of customers and turned out to have a positive result