Skip to content

Latest commit

 

History

History
113 lines (84 loc) · 8.82 KB

README.md

File metadata and controls

113 lines (84 loc) · 8.82 KB

Pipecat Client React Native Transports

Docs Discord

A mono-repo to house the various supported Transport options to be used with React Native. Currently, there are two transports: small-webrtc-transport and daily-transport.

Documentation

Please refer to the full Pipecat client documentation here.

Current Transports

SmallWebRTCTransport

Docs README Demo NPM Version

This Transport creates a peer-to-peer WebRTC connection between the client and the bot process. This Transport is the client-side counterpart to the Pipecat SmallWebRTCTransport component.

This is the simplest low-latency audio/video transport for Pipecat. This transport is recommended for local development and demos. Things to be aware of:

  • This transport is a direct connection between the client and the bot process. If you need multiple clients to connect to the same bot, you will need to use a different transport.
  • For production usage at scale, a distributed WebRTC network that can do edge/mesh routing, has session-level observability and metrics, and can offload recording and other auxiliary services is often useful.

Typical media flow using a SmallWebRTCTransport:

                                            ┌──────────────────────────────────────────────────┐
                                            │                                                  │
 ┌─────────────────────────┐                │                       Server       ┌─────────┐   │
 │                         │                │                                    │Pipecat  │   │
 │            Client       │  RTVI Messages │                                    │Pipeline │   │
 │                         │       &        │                                              │   │
 │ ┌────────────────────┐  │  WebRTC Media  │  ┌────────────────────┐    media   │ ┌─────┐ │   │
 │ │SmallWebRTCTransport│◄─┼────────────────┼─►│SmallWebRTCTransport┼────────────┼─► STT │ │   │
 │ └────────────────────┘  │                │  └───────▲────────────┘     in     │ └──┬──┘ │   │
 │                         │                │          │                         │    │    │   │
 └─────────────────────────┘                │          │                         │ ┌──▼──┐ │   │
                                            │          │                         │ │ LLM │ │   │
                                            │          │                         │ └──┬──┘ │   │
                                            │          │                         │    │    │   │
                                            │          │                         │ ┌──▼──┐ │   │
                                            │          │           media         │ │ TTS │ │   │
                                            │          └─────────────────────────┼─┴─────┘ │   │
                                            │                       out          └─────────┘   │
                                            │                                                  │
                                            └──────────────────────────────────────────────────┘

DailyTransport

Docs README Demo NPM Version

This Transport uses the Daily audio and video calling service to connect to a bot and stream media over a WebRTC connection. This Transport is the client-side counterpart to the Pipecat DailyTransport component.

Typical media flow using a DailyTransport:


                                       ┌────────────────────────────────────────────┐
                                       │                                            │
  ┌───────────────────┐                │                 Server       ┌─────────┐   │
  │                   │                │                              │Pipecat  │   │
  │      Client       │  RTVI Messages │                              │Pipeline │   │
  │                   │       &        │                              │         │   │
  │ ┌──────────────┐  │  WebRTC Media  │  ┌──────────────┐    media   │ ┌─────┐ │   │
  │ │DailyTransport│◄─┼────────────────┼─►│DailyTransport┼────────────┼─► STT │ │   │
  │ └──────────────┘  │                │  └───────▲──────┘     in     │ └──┬──┘ │   │
  │                   │                │          │                   │    │    │   │
  └───────────────────┘                │          │                   │ ┌──▼──┐ │   │
                                       │          │                   │ │ LLM │ │   │
                                       │          │                   │ └──┬──┘ │   │
                                       │          │                   │    │    │   │
                                       │          │                   │ ┌──▼──┐ │   │
                                       │          │     media         │ │ TTS │ │   │
                                       │          └───────────────────┼─┴─────┘ │   │
                                       │                 out          └─────────┘   │
                                       │                                            │
                                       └────────────────────────────────────────────┘

Local Development

Build the transport libraries

$ yarn install
$ yarn build

License

BSD-2 Clause

Contributing

We welcome contributions from the community! Whether you're fixing bugs, improving documentation, or adding new features, here's how you can help:

  • Found a bug? Open an issue
  • Have a feature idea? Start a discussion
  • Want to contribute code? Check our CONTRIBUTING.md guide
  • Documentation improvements? Docs PRs are always welcome

Before submitting a pull request, please check existing issues and PRs to avoid duplicates.

We aim to review all contributions promptly and provide constructive feedback to help get your changes merged.