https://github.com/hpotechius/ColorTransferLab and https://potechius.com/ColorTransferLab

ColorTransferLib

ColorTransferLib is a library dedicated to color transfer, style transfer, and colorization, featuring a diverse range of published algorithms. Some methods have been re-implemented, while others are integrated from public repositories.

The primary goal of this project is to consolidate all existing color transfer, style transfer, and colorization techniques into a single library with a standardized API. This facilitates both the development and comparison of algorithms within the research community.

Currently, the library supports 11 color transfer, 5 style transfer, and 3 colorization methods across various data types, including images, point clouds, textured meshes, light fields, videos, volumetric videos, and Gaussian splatting. Additionally, it provides 20 evaluation metrics for assessing image-to-image color transfer performance.

A compatibility chart for supported data types and a detailed list of all algorithms can be found below.

API

For seamless integration, adhere to the API specifications of the new color transfer algorithm, depicted in the Figure below.

Each class demands three inputs: Source, Reference, and Options. The Source and Reference should be of the Image, Video, VolumetricVideo, LightField, GaussianSplatting or Mesh class type, with the latter encompassing 3D point clouds and textured triangle meshes. The Options input consists of dictionaries, stored as a JSON file in the Options folder. For a sample option, see Listings 1. Every option details an adjustable parameter for the algorithm.

Save each new color transfer class in the ColorTransferLib Repository under the Algorithms folder. The class should have the apply(...) function, which ingests the inputs and embodies the core logic for color transfer.

The output should resemble a dictionary format, as outlined in Listing 2. A status code of 0 signifies a valid algorithm output, while -1 indicates invalidity. The process time denotes the algorithm's execution duration, useful for subsequent evaluations. The 'object' key in the dictionary holds the result, which should match the class type of the Source input.

Installation

Requirements

(1) Install the following packages:

# for running matlab code
sudo apt-get install octave-dev
# allows writing of mp4 with h246 codec
sudo apt-get install ffmpeg

(2) Install the following octave package:

# activate octave environment
octave
# install packages
octave:1> pkg install -forge image
octave:2> pkg install -forge statistics

(3) Run the gbvs_install.m to make the evaluation metric VSI runnable:

user@pc:~/<Project Path>/ColorTransferLib/Evaluation/VIS/gbvs$ ocatve
octave:1> gbvs_install.m

Install via PyPI

pip install colortransferlib
pip install git+https://github.com/facebookresearch/detectron2.git@main

Install from source

pip install -r requirements/requirements.txt
python setup.py bdist_wheel
pip install ../ColorTransferLib/dist/ColorTransferLib-2.0.3-py3-none-any.whl 
pip install git+https://github.com/facebookresearch/detectron2.git@main

Usage

Color Transfer

from ColorTransferLib.ColorTransfer import ColorTransfer
from ColorTransferLib.DataTypes.Image import Image

src = Image(file_path='/media/source.png')
ref = Image(file_path='/media/reference.png') 

algo = "GLO"
ct = ColorTransfer(src, ref, algo)
out = ct.apply()

# No output file extension has to be given
if out["status_code"] == 0:
    out["object"].write("/media/out")
else:
    print("Error: " + out["response"])

Evaluation

from ColorTransferLib.ColorTransfer import ColorTransferEvaluation
from ColorTransferLib.DataTypes.Image import Image

src = Image(file_path='/media/source.png')
ref = Image(file_path='/media/reference.png') 
out = Image(file_path='/media/output.png') 

cte = ColorTransferEvaluation(src, ref, out)
eva = cte.apply(method)
print(eva)

Test

# Test all Color Transfer algorithms with all data type combinations
python main.py --test all_CT --out_path "/media/out"

# Test all Style Transfer algorithms with all data type combinations
python main.py --test all_ST --out_path "/media/out"

# Test all Colorization algorithms with all data type combinations
python main.py --test all_CT --out_path "/media/out"

# Test all evaluation metric on src, ref and out images
python main.py --test all_EVAL

Available Methods:

The following color transfer, style transfer and colorization methods are integrated in the library. Some of them are reimplemented based on the algorithm's description in the the published papers and others are adopted from existing repositories and adpated to fit the API. The original implementation of the latter methods can be found next to the publication's name. Highlighted icon within the Support Column indicates the supported data types (From left to right: (1) Gaussian Splatting, (2) Light Field, (3) Volumetric Video, (4) Video, (5) Point Cloud, (6) Mesh, (7) Image)

Color Transfer

Year	ID	Support	Publication
2001	$GLO$		Color Transfer between Images
2003	$BCC$		A Framework for Transfer Colors Based on the Basic Color Categories
2005	$PDF$		N-dimensional probability density function transfer and its application to color transfer
2006	$CCS$		Color transfer in correlated color space
2007	$MKL$		The Linear Monge-Kantorovitch Linear Colour Mapping for Example-Based Colour Transfer
2009	$GPC$		Color Transfer between Images
2010	$FUZ$		An efficient fuzzy clustering-based color transfer method
2019	$TPS$		L2 Divergence for robust colour transfer - Original Implementation
2020	$HIS$		Deep Color Transfer using Histogram Analogy - Original Implementation
2021	$RHG$		HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms
2021	$EB3$		Example-Based Colour Transfer for 3D Point Clouds

Style Transfer

Year	ID	Support	Publication
2015	$NST$		A Neural Algorithm of Artistic Style - Original Implementation
2017	$DPT$		Deep Photo Style Transfer - Original Implementation
2020	$PSN$		PSNet: A Style Transfer Network for Point Cloud Stylization on Geometry and Color - Original Implementation
2021	$CAM$		CAMS: Color-Aware Multi-Style Transfer - Original Implementation
2022	$ST2$		Stytr2: Image style transfer with transformers - Original Implementation

Colorization

Year	ID	Support	Publication
2020	$IIC$		Instance-aware image colorization - Original Implementation
2022	$CFM$		Colorformer: Image colorization via color memory assisted hybrid-attention transformer - Original Implementation
2023	$DDC$		DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders - Original Implementation

Available Objective Evaluation Metrics

Three classes of evaluation metrics are considered here. Metrics that evaluate the color consistency with the reference image (indicated with $^r$), metrics that evaluate the structural similarity with the source image (indicated with $^s$) and metrics that evaluates the overall quality of the output (indicated with $^o$).

Year	ID	Name	Publication
/	$PSNR^s_{rgb}$	Peak Signal-to-Noise Ratio	/
/	$HI^r_{rgb}$	Histogram Intersection	/
/	$Corr^r_{rgb}$	Correlation	/
/	$BA^r_{rgb}$	Bhattacharyya Distance	/
/	$MSE^s_{rgb}$	Mean-Squared Error	/
/	$RMSE^s_{rgb}$	Root-Mean-Squared Error	/
2003	$CF^o_{rgyb}$	Colorfulness	Measuring Colourfulness in Natural Images
2003	$MSSSIM^s_{rgb}$	Multi-Scale Structural Similarity Index	Multiscale structural similarity for image quality assessment
2004	$SSIM^s_{rgb}$	Structural Similarity Index	Image quality assessment: from error visibility to structural similarity
2006	$GSSIM^s_{rgb}$	Gradient-based Structural Similarity Index	Gradient-Based Structural Similarity for Image Quality Assessment
2010	$IVSSIM^s_{rgb}$	4-component Structural Similarity Index	Content-partitioned structural similarity index for image quality assessment
2011	$IVEGSSIM^s_{rgb}$	4-component enhanced Gradient-based Structural Similarity Index	An image similarity measure using enhanced human visual system characteristics
2011	$FSIM^s_{c,yiq}$	Feature Similarity Index	FSIM: A Feature Similarity Index for Image Quality Assessment
2012	$BRISQUE^o_{rgb}$	Blind/Referenceless Image Spatial Quality Evaluator	No-Reference Image Quality Assessment in the Spatial Domain
2013	$NIQE^o_{rgb}$	Naturalness Image Quality Evaluator	Making a “Completely Blind” Image Quality Analyzer
2014	$VSI^s_{rgb}$	Visual Saliency-based Index	VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment
2016	$CTQM^{sro}_{lab}$	Color Transfer Quality Metric	Novel multi-color transfer algorithms and quality measure
2018	$LPIPS^s_{rgb}$	Learned Perceptual Image Patch Similarity	The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
2018	$NIMA^o_{rgb}$	Neural Image Assessment	NIMA: Neural Image Assessment
2019	$CSS^{sr}_{rgb}$	Color and Structure Similarity	Selective color transfer with multi-source images

Issues

• PSN crashes if the point clouds are too large

Citation

If you utilize this code in your research, kindly provide a citation:

@inproceeding{potechius2023,
  title={A software test bed for sharing and evaluating color transfer algorithms for images and 3D objects},
  author={Herbert Potechius, Thomas Sikora, Gunasekaran Raja, Sebastian Knorr},
  year={2023},
  booktitle={European Conference on Visual Media Production (CVMP)},
  doi={10.1145/3626495.3626509}
}