In a previous blog post we provided a walk-through on how to setup video file transcoding using Open Source Cloud based on SVT Encore and supporting backend services. In this blog post we are extending the setup by adding the creation of video-on-demand streaming files to the pipeline.
In this solution we will add another open source project made available as a service. The Encore Packager is a backend service that creates the VOD file package. It consumes jobs from a Redis queue and creates the VOD file package and uploads the package to an S3 bucket. For the creation of the VOD file package the open source packager Shaka Packager is used. The red box in the diagram below shows what we will add to our solution.
Step 1: Create another Valkey queue
Valkey provides a Redis compatible key / value store and we will create another queue for the packaging jobs. Navigate to the Valkey service in Open Source Cloud and press “Create valkey”. Give the instance a name and press Create.
Note down the IP and port to the Valkey instance card and this is what will be the Redis URL that we will refer to later in this blog. In this example it would be redis://172.232.131.169:10511.
Step 2: Launch another Encore Callback Listener
We will now create a separate service that can be used to monitor a transcoding job in SVT Encore so we know when the file is ready to be packaged. Navigate to the Encore Callback Listener in the web user interface. Click on button “Create callback” and enter the name of the instance, Redis URL (above), URL to the SVT Encore instance that we created last time and the name of the queue. We will call this queue for “package” now.
Important the URL to the SVT Encore instance is without the trailing slash.
Step 3: Create Encore Packager service
We can now move on with creating the Encore Packager service. Enter the name of the instance, Redis URL, name of queue in Redis (Valkey), output S3 URL, OSC token and the AWS credentials for the output S3 bucket. In this example we will have the following values:
Then press Create and wait for the instance to be ready.
Step 4: Submit a job
Now we are ready to try transcoding and creating a VOD package that we have available on an S3 compatible storage. We will create signed URL to the video file we want to transcode. For example:
https://lab-testcontent-input.s3.eu-north-1.amazonaws.com/NO_TIME_TO_DIE_short_Trailer_2021.mp4?SIGNURLSTUFF
Navigate back to the SVT Encore service and press the menu item to open API docs again. Click on the POST /encoreJobs bar and button “Try it out” and enter the following JSON. Here we have changed the progressCallbackUri to point to our Encore Callback Listener for VOD packaging.
{
"externalId": "blog",
"profile": "program",
"outputFolder": "/usercontent/",
"baseName": "blog",
"progressCallbackUri": "https://demo-vod.eyevinn-encore-callback-listener.auto.prod.osaas.io/encoreCallback",
"inputs": [
{
"uri": "https://lab-testcontent-input.s3.eu-north-1.amazonaws.com/NO_TIME_TO_DIE_short_Trailer_2021.mp4?SIGNURL",
"seekTo": 0,
"copyTs": true,
"type": "AudioVideo"
}
]
}
And then press button Execute. Now a job is submitted and if you want to see the progress you can go to the Encore Callback Listener service and open the instance logs to check that it is receiving the callbacks.
When the transcoding process is completed it will place a job on the packaging queue that will be picked up by the Encore Packager service. And when the packaging job is completed you will in this example find a VOD package ready for streaming: https://lab.cdn.eyevinn.technology/osc/NO_TIME_TO_DIE_short_Trailer_2021/bb347d8e-c095-43dc-ba5f-914c7e74f13d/index.m3u8.
Conclusion
You now have a fully fledged video transcoding and packaging pipeline for preparing video files for streaming using ´SVT Encore with some supporting services. All based on open source and you don’t have to setup your own infrastructure for this to get started. If you later choose to do so you are free to do it as the code and everything demonstrated here is available as open source.
