Multi-Objective Ad Recommendation System

This project builds a production-style baseline recommendation system for information-feed ads, with one-click execution and four objective modes:

1. CTR optimization
2. Order optimization
3. Session dwell optimization
4. Multi-objective optimization (click + order + dwell)

1. Problem Statement

Given user_id and current request time when a user opens the APP, rank candidate ads to maximize business outcomes under different objective settings.

Available data:

• data/user_app_behavior.xlsx
• data/user_order.xlsx
• data/ads.xlsx
• data/product.xlsx

2. End-to-End Solution Overview

2.1 System Flow Diagram

flowchart LR
    A["Raw Event and Master Data"] --> B["Data Cleaning and Timestamp Parsing"]
    B --> C["Display Level Label Construction"]
    C --> D["Leakage Safe Feature Engineering"]
    D --> E["Temporal Train Val Test Split"]
    E --> F["CTR Model Training"]
    E --> G["Order Model Training"]
    E --> H["Dwell Model Training"]
    F --> I["Objective Specific Scoring"]
    G --> I
    H --> I
    I --> J["Rank Candidate Ads"]
    J --> K["Offline Evaluation and Artifact Export"]

Loading

2.2 One-Click Runtime Flow

flowchart TD
    A["python3 main.py"] --> B["Profile Dataset"]
    B --> C["Train All Models"]
    C --> D["Auto Select Demo User"]
    D --> E["Generate Top K Recommendations"]
    E --> F["Save Bundle Metrics and Recommendation CSV"]

Loading

3. Data Modeling and Labeling Strategy

Training unit:

• one row per display event

Labels:

• label_click: user clicked same ad within 30 minutes
• label_order_24h: user placed any order within 24 hours after display
• label_order_7d: long-window order label for analysis
• label_dwell_sec: remaining session duration after display

Why this design:

• aligns recommendation ranking to exposure decisions
• supports direct objective switching without rebuilding the sample table
• keeps modeling and evaluation consistent across scenarios

4. Feature Engineering Design

Feature groups:

• context features: hour, weekday, is_weekend
• session dynamics: session_event_index, session_elapsed_sec
• user history: prior displays, clicks, historical CTR
• ad history: prior displays, clicks, historical CTR
• user-ad interaction history: pair-level display/click/CTR stats
• transaction history: prior orders, average items per order, time since last order
• ad metadata: creative_type, main_color, design_style, has_ad_profile

Leakage control:

• all history features use only prior events
• split is time-based (not random)
• as-of joins use sorted keys to preserve temporal causality

5. Objective-Specific Recommendation Logic

Scenario A: CTR Optimization

• target: maximize immediate click probability
• model: p_ctr = P(click | user, ad, context)
• score: score_ctr = p_ctr

Scenario B: Order Optimization

• target: maximize downstream orders
• model: p_order = P(order_24h | user, ad, context)
• score: score_order = p_ctr * p_order

Reason:

• this combines engagement likelihood and conversion likelihood, reducing instability from sparse conversion labels.

Scenario C: Dwell Optimization

• target: maximize session continuation
• model: pred_dwell_sec
• score: score_dwell = 0.35 * p_ctr + 0.65 * norm(pred_dwell_sec)

Reason:

• dwell-only ranking can drift from ad relevance; CTR term acts as a relevance guardrail.

Scenario D: Multi-Objective Optimization

• target: optimize click, order, and dwell jointly
• score:
  • score_multi = w_ctr*norm(p_ctr) + w_order*norm(p_order) + w_dwell*norm(pred_dwell_sec)
  • default weights: w_ctr=0.4, w_order=0.4, w_dwell=0.2

Reason:

• explicit utility weighting gives product and business teams a controllable tradeoff mechanism.

6. Modeling and Training Stack

Models:

• CTR: HistGradientBoostingClassifier
• Order: HistGradientBoostingClassifier
• Dwell: HistGradientBoostingRegressor on log1p(dwell)

Preprocessing:

• categorical: most-frequent imputation + ordinal encoder with unknown handling
• numeric: median imputation
• class imbalance: balanced sample weights for binary tasks

Split strategy:

• strict temporal split with default ratios 70/15/15

7. Evaluation Framework

7.1 Model-Level Quality

• CTR and Order: ROC-AUC, PR-AUC, LogLoss, positive rate
• Dwell: RMSE, MAE

7.2 Ranking-Level Quality

• NDCG@5 and NDCG@10 on click and order labels
• top1 policy simulation:
  • top1_click_rate
  • top1_order_rate_24h
  • top1_avg_dwell_sec
  • top1_multi_utility

7.3 Evaluation Flow Diagram

flowchart LR
    A["Validation and Test Predictions"] --> B["Model Metrics"]
    A --> C["Objective Scores"]
    C --> D["NDCG at K"]
    C --> E["Top1 Policy Simulation"]
    B --> F["metrics.json"]
    D --> F
    E --> F

Loading

8. Code Architecture

• main.py: CLI entrypoint and one-click orchestration
• src/recsys/config.py: path and training config dataclasses
• src/recsys/data.py: loading, cleaning, labels, features, temporal split
• src/recsys/modeling.py: model builders and fit/predict helpers
• src/recsys/scoring.py: objective score functions
• src/recsys/evaluation.py: offline metric suite
• src/recsys/pipeline.py: profile/train/recommend orchestration

9. Usage

9.1 One-Click Execution

python3 main.py

Equivalent explicit mode:

python3 main.py run-all --data-dir data --artifact-dir artifacts --objective multi --top-k 20

Option-only style also works:

python3 main.py --artifact-dir artifacts --objective order --top-k 30

9.2 Granular Execution

Profile only:

python3 main.py profile --data-dir data

Train only:

python3 main.py train --data-dir data --artifact-dir artifacts

Recommend only:

python3 main.py recommend --artifact-dir artifacts --user-id 6176 --timestamp "2021-11-21 12:00:00" --objective multi --top-k 20

10. Artifacts

• artifacts/model_bundle.joblib
• artifacts/metrics.json
• artifacts/val_predictions.csv
• artifacts/test_predictions.csv
• artifacts/one_click_recommendations_<user>_<objective>.csv

11. Production Upgrade Path

1. Add propensity logging and IPS or DR estimators for counterfactual policy evaluation.
2. Replace independent models with multi-task architecture such as MMoE or PLE.
3. Introduce constrained re-ranking with KPI guardrails and pacing logic.
4. Add online experiment framework with CUPED and guardrail monitoring.
5. Add nearline feature updates for fresher user state at serving time.

12. Known Limitations

• Order labels are user-level outcomes and do not establish direct ad-to-order causality.
• Offline policy metrics are still correlational without explicit propensity logs.
• This baseline does not include latency benchmarks or serving infrastructure.

These limitations are documented intentionally to define a clear next iteration plan.