CS7180: Decompose and Relight - Intrinsic Image Separation with Automatic Light Detection

Authors: Vaibhav Thalanki, Oliver Fritsche, Sai Manichandana Devi Thumati
Institution: Khoury College of Computer Sciences, Northeastern University

Overview

This project implements an intrinsic image decomposition pipeline inspired by Garces et al. (2012) that separates photographs into reflectance (surface color) and shading (illumination) components. The pipeline uses SLIC superpixel segmentation with KMeans clustering and solves a sparse linear system balancing reflectance consistency and edge-aware shading smoothness.

Features

• Intrinsic Decomposition: Separates images into reflectance and shading using clustering-based approach
• Sepia Tone Transform: Applies artistic sepia effect to reflectance component
• Automatic Light Detection: Point-light simulation with automatic light source detection (Lopez-Moreno et al. 2010)
• Directional Lighting: Simulates lighting from right edge
• Photorealistic Composition: Linear RGB space composition with automatic exposure adjustment

Pipeline

1. Decompose input image into reflectance (R) and shading (S)
2. Apply sepia stylization to R
3. Relight S using point-auto or directional mode
4. Recompose in linear space with automatic exposure correction

Results

Decomposition Components

Original Reflectance

Original Shading

Transform and Relighting

Sepia-transformed Reflectance

Relit Shading (Point-Auto)

Final Result

Final Combined Output (Point-Auto Relighting)

Usage

pip install -r ./requirements.txt

python run_pipeline.py <image_path> [options]

Options:

• --mode {point_auto,directional_right} - Relighting mode (default: point_auto)
• --out_dir PATH - Output directory (default: ./outputs)
• --clusters K - Number of KMeans clusters (default: 10)
• --n_segments N - Number of SLIC superpixels (default: 600)
• --sepia FLOAT - Sepia intensity 0-1 (default: 0.85)

Example:

python run_pipeline.py ./inputs/input_image3.jpg --mode point_auto --out_dir ./outputs

The script generates decomposition and relighting results in the specified output directory.

Project Structure

intrinsic/          # Decomposition module
relight/            # Relighting modules (point, directional, compose)
utils/              # Color space utilities
outputs/            # Generated results
inputs/            # 5 chosen input images

Parameters

• Superpixels: 600 segments, compactness 10.0
• KMeans clusters: K=10
• Smoothness: λ=0.2, σ=0.1
• Sepia intensity: t=0.85

Limitations

Since this is a classical (non-learning) approach, new shadows do not form when introducing artificial light sources. Modern deep learning methods address this limitation by learning shadow formation implicitly.

Dataset

Images sourced from Intrinsic Images in the Wild (Bell, Bala, and Snavely 2014), a large-scale public dataset with crowdsourced material property annotations.