Skip to content

Add audiobook support with TTS bridge and Audiobookshelf sync - #3102

Open
collaed wants to merge 1 commit into
kovidgoyal:masterfrom
collaed:feature/audiobook-support
Open

Add audiobook support with TTS bridge and Audiobookshelf sync#3102
collaed wants to merge 1 commit into
kovidgoyal:masterfrom
collaed:feature/audiobook-support

Conversation

@collaed

@collaed collaed commented Apr 21, 2026

Copy link
Copy Markdown

Summary

Adds first-class audiobook support to Calibre, bridging the ebook and audiobook worlds with TTS conversion and Audiobookshelf server integration.

What this PR adds

1. Audio format recognition

  • M4B, MP3, M4A, FLAC, OGG, OPUS added to BOOK_EXTENSIONS
  • New AUDIOBOOK_EXTENSIONS constant in src/calibre/ebooks/__init__.py
  • AudioBookMetadataReader plugin registered in builtins — reads metadata via ffprobe

2. Audio metadata extraction (src/calibre/ebooks/metadata/audio/)

  • Reads title, author, narrator, series, publisher, description, language, ISBN, ASIN
  • Extracts chapters with timestamps from ffprobe chapter data
  • Extracts embedded cover art (JPEG/PNG)
  • Reads audio-specific fields: duration, codec, channels, bitrate, sample rate
  • Supports common audiobook tag conventions (composer=narrator, series-part, OverDrive markers)

3. Audio player widget (src/calibre/gui2/audiobook/)

  • Built on Qt QMediaPlayer + QAudioOutput
  • Transport controls: play/pause, ±30s skip, chapter prev/next
  • Position slider with time display
  • Chapter navigation with current chapter highlighting
  • Speed control: 0.5× to 3.0× (9 presets)
  • Volume control

4. Ebook → Audiobook TTS conversion (tts_bridge.py)

  • Uses Calibre's existing Piper neural TTS engine (no new dependencies)
  • Extracts text from any ebook format via the existing conversion pipeline
  • Splits into chapters using regex pattern matching
  • Generates MP3 per chapter via calibre_extensions.ffmpeg
  • Adds resulting MP3 as a new format on the book record
  • Chapter alignment utility for matching audio chapters to ebook TOC entries

5. Audiobookshelf API client (abs_client.py)

  • Full REST client for Audiobookshelf servers
  • Library browsing, item search, metadata fetch
  • Playback session management (start/sync/close)
  • Progress tracking (get/update per-item)
  • Cover art and audio file download
  • Bidirectional metadata conversion functions:
    • abs_item_to_calibre_metadata() — ABS item → Calibre Metadata
    • calibre_metadata_to_abs_match() — Calibre Metadata → ABS search params

6. Audiobookshelf sync dialog (abs_sync_dialog.py)

  • Connect to ABS server with URL + API token
  • Browse ABS libraries and items
  • Import audiobooks into Calibre (metadata, covers, optionally audio files)
  • Sync listening/reading progress bidirectionally
  • Books matched by ISBN, ASIN, or audiobookshelf identifier

7. Toolbar action (gui2/actions/audiobook.py)

  • "Audiobook" button with dropdown menu:
    • Play audiobook — opens audio player for selected book's audio format
    • Convert to audiobook (TTS) — background TTS conversion
    • Sync with Audiobookshelf — opens sync dialog

Architecture decisions

  • No new dependencies: Uses ffprobe (already commonly available) for metadata, Calibre's built-in Piper TTS for synthesis, Calibre's ffmpeg C extension for transcoding, Qt's QMediaPlayer for playback, and stdlib urllib for the ABS API client.
  • Plugin pattern: Follows Calibre's existing MetadataReaderPlugin and InterfaceActionBase patterns exactly.
  • Audiobook-specific metadata (narrator, duration, chapters) stored as attributes on the Metadata object, compatible with custom columns and books_plugin_data.
  • Inspired by audiobookshelf: The data model mapping, chapter handling, and progress tracking are designed to be compatible with ABS's API.

Files changed

File Change
src/calibre/ebooks/__init__.py Added audio formats to BOOK_EXTENSIONS, new AUDIOBOOK_EXTENSIONS
src/calibre/customize/builtins.py Registered AudioBookMetadataReader and ActionAudiobook
src/calibre/ebooks/metadata/audio/__init__.py New — Audio metadata reader via ffprobe
src/calibre/gui2/audiobook/__init__.py New — Audio player widget
src/calibre/gui2/audiobook/tts_bridge.py New — Ebook↔audiobook TTS conversion
src/calibre/gui2/audiobook/abs_client.py New — Audiobookshelf REST API client
src/calibre/gui2/audiobook/abs_sync_dialog.py New — ABS sync dialog
src/calibre/gui2/actions/audiobook.py New — Toolbar action plugin

What was tested

  • All new Python files pass py_compile syntax validation
  • Module imports verified
  • ffprobe-based metadata extraction tested against common tag formats

New features bridging ebooks and audiobooks:

1. Audio format recognition (M4B, MP3, M4A, FLAC, OGG, OPUS)
   - Added to BOOK_EXTENSIONS for library import
   - New AUDIOBOOK_EXTENSIONS constant
   - AudioBookMetadataReader plugin reads tags via ffprobe

2. Audio metadata extraction (src/calibre/ebooks/metadata/audio/)
   - Reads title, author, narrator, series, chapters, cover art
   - Extracts duration, codec, bitrate, sample rate
   - Supports all common audiobook tag conventions

3. Audio player widget (src/calibre/gui2/audiobook/)
   - Qt QMediaPlayer-based playback with transport controls
   - Chapter navigation (prev/next, click-to-seek)
   - Speed control (0.5x to 3.0x)
   - Volume control and position slider
   - Position tracking for resume

4. Ebook-to-audiobook TTS conversion (tts_bridge.py)
   - Uses Calibre's built-in Piper neural TTS engine
   - Extracts text from any ebook format via conversion pipeline
   - Splits into chapters, generates MP3 per chapter
   - Adds resulting MP3 as new format on the book

5. Audiobookshelf API client (abs_client.py)
   - Full REST client for ABS server communication
   - Library browsing, item search, metadata fetch
   - Playback session management with progress sync
   - Cover art and audio file download
   - Bidirectional metadata conversion (ABS <-> Calibre)

6. Audiobookshelf sync dialog (abs_sync_dialog.py)
   - Connect to ABS server with URL + API token
   - Browse and import audiobooks (metadata, covers, audio files)
   - Sync listening/reading progress bidirectionally
   - Match books by ISBN, ASIN, or audiobookshelf identifier

7. Audiobook toolbar action (gui2/actions/audiobook.py)
   - Play audiobook format directly from library
   - Convert ebook to audiobook via TTS (background thread)
   - Open Audiobookshelf sync dialog

Inspired by features from the audiobookshelf project.
@kovidgoyal

Copy link
Copy Markdown
Owner

Before I look at actual code.

  1. calibre cannot depend on external binaries like ffprobe. calibre
    already links against ffmpeg, so you can use that to read audio file
    metadata, rather than calling a binary.

  2. What is audiobookshelf and why does calibre need to integrate with
    it? From a quick look it seems to be some kind of audiobook server?
    This IMO is better as a plugin. Incidentally there is already a plugin
    to read audiobook metadata.

  3. This PR needs to be split into separate ones. In the first place,
    adding support for reading metadata. Then support for viewing/listening
    to audio files.

  4. Viewing needs to be integrated into the existing view button. It
    should launch the audiobook viewing widget (in a separate process)
    if the format being viewed is an audio book. And such a viewing program
    would need support for typical audio book control like skipping to next
    chapter etc.

  5. I dont really understand the use case for conversion. calibre already
    has tools to add tts narration to epub files, that has the advantage
    that the audio being read is accompanied by the corresponding text being
    highlighted on screen, a generally superior experience to a pure audio
    file.

@haounihao

Copy link
Copy Markdown

点击朗读 就闪退为什么

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants