Support for Direct S3 Upload of HLS Segments

[rashied-r]

Summary:
I am looking to see if ffmpeg-go supports directly uploading HLS segments (e.g., .ts files) to an S3-compatible storage like MinIO or AWS S3 as they are generated. Currently, I am handling this by first saving segments locally and then manually uploading them to S3. I would like to know if there is a built-in way for ffmpeg-go to automatically handle the upload process.

Use Case:

• I am generating HLS streams from a video input using ffmpeg-go.
• My goal is to have each HLS segment (.ts files) directly uploaded to S3 or MinIO during the generation process without having to first write them to local storage and then upload them manually.
• This would be particularly useful for large files or live streams where writing segments locally might be inefficient or consume too much disk space.

What I Have Tried:

• I currently use ffmpeg-go with an output like this:

  ffmpeg_go.Input(input).
      Output("s3://my-bucket/my-stream/playlist.m3u8",
          ffmpeg_go.KwArgs{
              "c:v":                  "libx264",
              "c:a":                  "aac",
              "f":                    "hls",
              "hls_time":             10,
              "hls_list_size":        5,
              "hls_segment_filename": "s3://my-bucket/my-stream/segment_%03d.ts",
              "aws_config":           myAWSConfig,
          }).
      Run()

• However, this does not seem to directly upload segments to S3. Instead, I have implemented a custom solution where I:
  • Write the segments to a local directory.
  • Use a file watcher to detect new segments and upload them to S3 using aws-sdk-go.

Questions:

1. Does ffmpeg-go have built-in support for directly uploading HLS segments to S3 or MinIO without first writing them to local storage?
2. If not, is there any recommended way to integrate such functionality (e.g., hooks, custom storage handlers)?
3. Are there any plans to support this feature in future releases?

Additional Context:

• I am using aws-sdk-go for the manual upload process.
• My AWS and MinIO configuration is handled through standard AWS credentials.

Thank you for your time, and I appreciate any guidance or recommendations you can provide!

[TomSoyerNG]

I am having the same problem, did you find any solution?

[ShubhamTiwary914]

Did the same thing, copy to local directory on tmpfs mount and some worker go-routines to offload em' to GCS (it was GCS and not S3 in my case).
Please do tell if anyone got any other solution for this (even if for S3)