Harnessing LLMs for Document-Guided Fuzzing of OpenCV Library (VistaFuzz)

Documentation-guided fuzzing for OpenCV-Python. Reproducible artifact with Docker, scripts, and standardized API.

---

Table of Contents

• Overview
• Repository Layout
• Quickstart (Docker)
• Generate Coverage (gcovr)
• Expected Outputs
• Cite This Artifact
• License

---

Overview

VistaFuzz is a documentation-guided fuzzer for OpenCV-Python. It parses API documentation, standardizes API constraints, and generates valid/diverse inputs to exercise OpenCV operations at scale.

This repository provides:

• Runnable fuzzing entry: main.py
• Docker setup for a consistent environment and code coverage
• Standardized API metadata: API_info.py used by the fuzzer

---

Repository Layout

VistaFuzz/
├─ OpenCV-Testing/
│ ├─ API/
│ │ └─ OpenCV_API_filtered_subset.json
│ ├─ main/
│ │ └─ main.py
│ ├─ tool/
│ │ ├─ API_info.py
│ │ ├─ load_json_file.py
│ │ ├─ mutation.py
│ │ ├─ mutation_1.py
│ │ ├─ mutation_rules.py
│ │ ├─ opencv_args_seed_generator.py
│ │ ├─ oracle.py
│ │ ├─ parser_from_str2funcandinfo.py
│ │ ├─ temporary.py
│ │ └─ test.py
│ ├─ API_info.py
│ ├─ BugLinks.csv
│ ├─ Dockerfile
│ └─ main.py
└─ README.md

---

Quickstart (Docker)

Recommended for reproducibility.

1. Build the Docker image

docker build -t opencv-coverage .

2. Run the container (mount this repo)

• Linux/macOS

docker run -it --rm -v "$PWD/OpenCV-Testing:/app" --name opencv_coverage_container_1 opencv-coverage

• Windows PowerShell

docker run -it --rm -v "${PWD}\OpenCV-Testing:/app" --name opencv_coverage_container_1 opencv-coverage

3. Inside the container: run the fuzzer

cd /app
python3 main.py

Tip: Use ls -l to confirm the volume is mounted; logs are written to the current working directory.

---

Generate Coverage (gcovr)

1. Enter the OpenCV build directory (inside the container)

cd /usr/local/src/opencv/build/

2. Install gcovr

pip install gcovr

3. Generate an HTML coverage report

gcovr -r /usr/local/src/opencv --html --html-details -o coverage_report.html

4. Copy the report back to the host

• Find the container ID:

docker ps

• Copy the report:

docker cp <container_id>:/usr/local/src/opencv/build/coverage_report.html .

• Open the report:

macOS

open ./coverage_report.html

Windows (PowerShell)

start ./coverage_report.html

Linux

xdg-open ./coverage_report.html

---

Standardized API

VistaFuzz consumes standardized API to guide input generation:

OpenCV-Testing/API_info.py

This file describes each API's name, parameters, types/constraints, etc., which the fuzzer uses to synthesize valid and diverse inputs.

---

Optional: Run without Docker

Not recommended — native builds can diverge from the container. If you still choose a local run, mirror the container toolchain and follow this checklist:

• Expect small variations in coverage or numerics across machines due to BLAS/hardware differences.

Run a Small Subset of APIs

If you only want to run a subset of the API dataset, you can limit how many APIs main.py processes:

• Preferred (if supported by your main.py): pass a limit flag, e.g. --max-apis N.

  cd /app
  python3 main.py --max-apis 50

• Otherwise (quick code tweak): at the top of OpenCV-Testing/main/main.py, add a limit and slice the loaded list. For example:

  import os
  MAX_APIS = int(os.getenv("VISTAFUZZ_MAX_APIS", "0"))  # 0 = no limit
  
  apis = load_json_file('API/OpenCV_API_filtered_subset.json')
  if MAX_APIS > 0:
      apis = apis[:MAX_APIS]

  Then run with an environment variable:

  VISTAFUZZ_MAX_APIS=50 python3 main.py

---

Paper ↔ Artifact Mapping

A lightweight map from paper items to where they live in this repo.

Paper item	Where in repo / quick check
Method pipeline	OpenCV-Testing/main.py, OpenCV-Testing/tool/ — running main.py exercises the full pipeline.
Standardized API dataset	OpenCV-Testing/API/OpenCV_API_filtered_subset.json, OpenCV-Testing/tool/API_info.py
Input generation & mutations	OpenCV-Testing/tool/opencv_args_seed_generator.py, OpenCV-Testing/tool/mutation*.py
Oracles (Crash/NaN/Exception)	OpenCV-Testing/tool/oracle.py, OpenCV-Testing/tool/test.py
Scale & bug list	OpenCV-Testing/API/OpenCV_API_filtered_subset.json (API count), OpenCV-Testing/BugLinks.csv
Coverage reproduction	Dockerfile and README section Generate Coverage (gcovr)

Expected Outputs

• Running python3 main.py outputs which APIs/test cases were executed and runtime logs.

• Running gcovr produces an HTML coverage report at:

  /usr/local/src/opencv/build/coverage_report.html
  

  Copy it to your host and open it in a browser. The report includes line-by-line coverage with source highlighting.

Note: Coverage values depend on runtime and the number of generated test cases; the primary verification goal is successful report generation and navigable source coverage.

---

Cite This Paper/Artifact

@INPROCEEDINGS{VistaFuzz,
  author={Duan, Bin and Mahmud, Tarek and Che, Meiru and Yan, Yan and Dong, Naipeng and Kim, Dan Dongseong and Yang, Guowei},
  booktitle={2025 IEEE International Conference on Software Maintenance and Evolution (ICSME)}, 
  title={Harnessing LLMs for Document-Guided Fuzzing of OpenCV Library}, 
  year={2025},
  pages={73-84},
  keywords={Computer vision;Software maintenance;Large language models;Fuzzing;Testing;OpenCV Libraries},
  doi={10.1109/ICSME64153.2025.00017}}

---

License

This project is released under the MIT License.