Imagine you’re booking food on a food delivery platform. As you add a Pizza dish to your cart, the app suggests a perfect pairing—say, Cheesy Dip or Cold Drink. This isn’t guesswork. It’s driven by intelligent recommendations powered by association rule mining.
In this blog, we’ll walk through how we implemented a cart-level menu recommendation system using the Apriori algorithm, leveraging historical order data from the platform. This system runs per restaurant, ensuring personalized and relevant suggestions.
Problem Statement
While users can order food during their train journeys, helping them discover relevant item combinations at the cart level can:
-
Improve user experience by showing tailored and popular combos.
-
Reduces decision fatigue by suggesting logical add-ons.
-
Increase average order value via cross-sells.
Technical Process
1. Data Pipeline
Data Sources:
- Transactional Data: Orders placed in the last 3 months (from MySQL).
- Menu Metadata: Active items, food types (veg/non-veg), and item popularity scores.
Key Steps:
- Data Extraction:
- Gather the transaction orders data for the last 3 months from all the restaurants.
- Filters: Active restaurant/items only, orders with ≥2 items (to enable association analysis).
- Data Transformation:
- Pivot Tables: Convert transactional data into a boolean matrix format (order_id × item_id) for association rule mining.
- Item Metadata Enrichment: Merge with menu data (food type, item name, popularity) for post-processing.
2. Association Rule Mining (Apriori Algorithm)
Purpose:
Identify items frequently bought together using historical transaction patterns.
Workflow:
- Frequent Itemset Generation:
- Apriori Algorithm (mlxtend library): Discovers item combinations with ≥1% support (i.e., appearing in ≥1% of orders).
- Example: If “Samosa” and “Tea” appear together in 3% of orders, they form a frequent itemset.
- Rule Extraction:
- Metrics:
- Confidence: It assesses the likelihood that an item Y is purchased when item X is purchased. It provides insight into the strength of the association between two items. Confidence tells us how often items go together. Likelihood of consequent being bought if antecedent is in the cart (e.g., 80% confidence that Tea is bought with Samosa).
- Lift: Lift evaluates how much more likely two items are to be purchased together compared to being purchased independently. A lift greater than 1 suggests a strong positive association. Lift shows how strong the connection is between items. (“Bread and butter are much more likely to be bought together than by chance.”)
- Constraints:
- Rules limited to 1 antecedent → 1 consequent (simpler recommendations).
3. Post-Processing & Filtering
Dietary Compatibility:
- Veg/Non-Veg Rules: Ensure non-veg items are not recommended for veg items.
- Example: A “Paneer Tikka” (veg) will only suggest veg items.
Food-Type Fallback:
- For items with no Apriori rules (e.g., new or low-sales items):
- Food-Type Compatibility: Recommend items from the same food category (e.g., “North Indian” → “Dal Makhani”).
- Popularity Score: Prioritize items with higher historical sales.
4. Hybrid Recommendation Engine
Combines two strategies for full coverage:
- Apriori-Powered Rules: Direct item-to-item associations (high confidence).
- Food-Type/Popularity Rules: Fallback for items without associations.
Example Workflow:
- If a user adds “Pav Bhaji” (no Apriori rules):
- Recommend the top 2 items from the “Snacks” category with the highest (popularity × food-type support).
- Data Storage & Retrieval:
- MySQL: Stores transactional data (orders, order_items), menu items, and restaurant metadata.
- SQLAlchemy: Connects Python to MySQL for query execution.
- Data Processing:
- Pandas: Merges datasets, filters orders, and transforms data (pivot tables).
- NumPy: Handles matrix operations.
- Machine Learning:
- MLxtend: Implements the Apriori algorithm and association rule extraction.
- Logging & Monitoring:
- Python Logging: Tracks errors (e.g., failed restaurant processing) and performance metrics.
- Deployment:
- Batch Processing: Restaurants are processed individually to avoid memory overload.
- MySQL Connector/Python: Updates the items table with recommendations in JSON format.
Key Challenges & Solutions
- Sparse Data for Small Restaurants:
- Solution: Fallback to food-type/popularity recommendations.
- Dietary Rule Enforcement:
- Solution: Post-merge filtering using item_type(veg/non-veg) metadata.
- Scalability:
- Solution: restaurant-level parallelization
Outcome
- Real-Time Readiness: Recommendations are precomputed and stored in MySQL as JSON (e.g., recommended_items column).
- Cart-Level Integration: The app fetches pre-generated suggestions from the database when users add an item to their cart.
- Dynamic Updates: Runs daily/weekly to incorporate new order data.
Future Enhancements
- Collaborative Filtering: Uses patterns from similar users or orders to recommend pairings for personalization.
- Real-Time Rules: Use streaming engines (Apache Kafka/Spark) for live updates.
- Deep Learning: Train embeddings for item co-occurrence (e.g., Word2Vec).
- Contextual Filtering: Add contextual filters based on time of the day, order size, etc.
- Real-time feedback loop: User clicks on suggested items and add-to-cart behavior helps refine what gets shown.
This system balances rule-based logic with machine learning to deliver scalable, context-aware recommendations, enhancing user engagement on the platform.
