AI-Enhanced Cameras: Dedicated Hardware for On-Device Vision Processing

Welcome to a dedicated space for exploring and cataloging cameras equipped with hardware-based neural networks. These devices integrate convolutional neural networks (CNNs) directly onto hardware, paired with a camera sensor, facilitating high-speed image processing with minimal power consumption and no need for extensive external data transmission.

Device List Overview

• IDS NXT - Ambarella
• Firefly-DL
• Luxonis OAK-1
• SCAMP5
• Sony IMX500
• Sony IMX501

No Camera sensor / Only AI Chip

• Nvidia Jetson Nano
• BrainChip - Akida
• Qualcomm QCS8250
• Google EdgeTPU
• Intel Neural Stick2 - Movidius Myriad X VPU

Why Hardware-Based AI Cameras?

Integrating AI directly into camera hardware allows for:

• High Frame Rates & Low Bandwidth: Streamline data processing directly on the device, reducing the need for data transfer.
• Low Power Consumption: Ideal for battery-operated or remote devices.
• No Communication Overhead: Operates independently of network constraints.
• Simplified Deployment: No advanced programming required, making it accessible for various applications.

Devices with Camera Sensor

IDS NXT family with built in FPGA / Ambarella

• TODO: Research specs
• Website

Firefly-DL

• TODO: Research specs
• Website

Luxonis OAK-1

• TODO: Research specs
• Website

SCAMP5

• Capabilities:
  • Digit recognition, hand gesture recognition, lip reading
  • 10,000 FPS
  • Analogue signal processing (before DAC)
  • Does not support MMADDs
  • Instead uses additions, subtractions, binary operations (needs specialized CNN architecture)
• Literature:
  • "A Camera that CNNs" (2019) Paper, Video
  • PixelRNN (2024) Paper

Sony IMX500

• Capabilities:
  • Human pose estimation, semantic segmentation
• Literature:
  • Website
  • Video - Human Pose Estimation

No Camera sensor / Only AI Chip

Nvidia Jetson Nano

• [Details TBD]

Brainchip - Akida

• Website, Video

Qualcomm QCS8250

• [Details TBD]

Google EdgeTPU

• [Details TBD]

Intel Neural Stick2 - Movidius Myriad X VPU

• [Details TBD]

Further Reading and Resources

• Edge AI Vision Alliance is a website and community trying to bring together companies, customers, and everyone interested in AI on edge devices.

Contributing

We encourage the community to contribute by adding new devices or updating existing entries. Please provide detailed information including device capabilities, specifications, use cases, and official links.

Updates and Maintenance

This list is maintained by community contributions and curated by Martin Garbade. Updates are made regularly to ensure accuracy and relevance.

Join us in documenting the evolution of AI cameras and pushing the boundaries of what's possible with on-device processing!

Tags

ai, machine-learning, embedded-systems, hardware-acceleration, computer-vision