A video transcoder is a software application that converts video files from one format to another. In this proof-of-concept (POC) project, I developed a transcoder that supports MP4 videos in 720p, 1080p, and 1440p resolutions. Additionally, it allows up to five concurrent transcoding jobs, all running locally.
Since this project is primarily a demonstration of running a transcoding service locally and deepening my understanding of enterprise architecture—particularly the Claim Check pattern—my main focus was on data flow and the transcoder implementation. Also, I am going to work on 4k downscaled to 720p, 1080p, and 1440p resolutions.
As you'll see in my code, error handling is not robust, nor does it cover all edge cases. Additionally, certain coding implementations are not ideal. However, I believe this project is a good starting point for further development.
- The user submits a video, which is uploaded to Minio and subsequently sent to a message queue (e.g., RabbitMQ) for processing.
- A container is launched with transcoding workers. Each worker handles a specific resolution (720p, 1080p, 1440p). The system is designed to support up to five concurrent transcoding jobs, but in my implementation, it handled a maximum of three thanks to my machine's limitations.
- Workers retrieve videos from the queue, fetch them from Minio, and process the transcoding.
- Once transcoding is complete, the processed videos are stored back in Minio. After all videos are processed, they are
bundled into a
.zipfile and made available for the user to download. Following this, cleanup is performed on both the local file system and Minio.
- Build transcode service
docker build -t transcoder:latest .- Run Service with docker-compose
docker-compose up- Building project
mvn package- Run Spring Boot Application
Just use the run command in your IDE.
Initially, I implemented a simple scaling mechanism that spun up consumers based on the number of jobs in the queue and shut them down when the queue was empty. However, this approach was unreliable.
I later refined it by tracking active consumers and trying to scale dynamically, but this proved challenging. Ideally, scaling should be based on system resource usage (CPU and memory), but I ran into limitations with the Docker Java API. Another approach I considered was scaling based on the average processing time of videos, but that would require significant effort and would still be highly dependent on my machine's current state.
Before this project, I had no experience with Minio. However, I found its API intuitive and well-documented. I would definitely use it again in future projects and recommend it as a free and effective storage solution.
I initially assumed that sending messages through a queue would be straightforward, but I quickly realized there was more complexity than expected. Working with RabbitMQ taught me a lot about message queues and event-driven systems.
Overall, this project was a great learning experience. While there are many aspects I would like to improve, it's time to move on to the next challenge.