Marketing Campaign Recommendation System
Published:
Overview
Designed and deployed a sophisticated recommendation system to optimize customer targeting during digital marketing campaigns for a data infrastructure and analytics startup.
Challenge
Marketing teams struggled with:
- Low conversion rates in digital campaigns
- Inefficient customer targeting strategies
- Lack of personalization at scale
- No systematic approach to measure treatment effects
Solution
Technical Architecture
Built an end-to-end ML pipeline incorporating:
1. Data Engineering
- Ingested and processed 50M+ customer interaction events
- Built scalable ETL pipelines handling 3M customer profiles
- Implemented feature engineering for behavioral signals
2. Machine Learning Models
- Developed causal inference models to estimate treatment effects
- Implemented uplift modeling to identify high-propensity customers
- Used meta-learners (T-learner, S-learner, X-learner) for heterogeneous treatment effects
3. MLOps Infrastructure
- Containerized models using Docker
- Orchestrated workflows with Apache Airflow
- Managed experiments and model versioning with MLflow
- Deployed on AWS EC2 with Kubernetes for auto-scaling
- Implemented CI/CD pipelines via GitLab
4. Production Deployment
- Real-time scoring API using FastAPI
- A/B testing framework for continuous optimization
- Monitoring dashboards with Grafana
Technology Stack
- Languages: Python, SQL
- ML Libraries: Scikit-learn, XGBoost, CausalML
- Data: AWS Redshift, PostgreSQL
- Orchestration: Airflow, Kubernetes
- MLOps: Docker, MLflow, DVC, GitLab CI/CD
- Cloud: AWS (EC2, S3, Redshift)
Impact
- 30% increase in conversion rates for targeted campaigns
- 3 million customers scored in production
- 50 million interaction events processed
- Established scalable MLOps best practices across the organization
- Reduced time-to-production for new models from weeks to days
Key Learnings
Causal Inference Matters: Using uplift modeling vs traditional prediction models revealed significant improvements in identifying persuadable customers
Production-First Mindset: Building with deployment in mind from day one avoided costly refactoring
Experiment-Driven Culture: A/B testing framework enabled continuous iteration and measurable impact
Skills Demonstrated
- End-to-end ML pipeline development
- Causal inference and experimentation
- MLOps and production engineering
- Cloud infrastructure (AWS)
- Cross-functional collaboration with marketing teams