[FEATURE] PutUploader about egress HOT 4 OPEN

Instead of cloud S3 storage on premise solutions also can be used any s3 compatible storage like minio. Other solution I see it's just mount to egress volume like NAS so recordings can be stored on prem on remote server with no egress code changes. Or event to have sidecar container which will be watching folder changes and upload to remote server

from egress.

@serhiynovos, thank you for your suggestions. While integrating an S3-compatible storage solution like MinIO or utilizing NAS for on-prem storage could work, these approaches would introduce significant changes to our existing infrastructure and operational workflows. Our current ecosystem uses an ASP.NET backend for handling various types of recordings, and we aim to maintain consistency in how video recordings are managed and stored.

Implementing a sidecar container to monitor and upload files would require the development of a new service for handling webhook messages, which increases configuration complexity, CI, maintaning work etc.

The idea of deploying a script that continuously monitors a directory for new files and uploads them might seem like a simpler workaround. However, considering the sophistication of LiveKit's existing upload mechanisms, this solution feels more like a temporary patch rather than a robust, long-term strategy.

Incorporating HTTP PUT (or POST) support directly within LiveKit as an uploader option would offer a consistent and maintenance-friendly solution. It aligns with many existing backend services and simplifies the configuration process, eliminating the need for additional components or significant infrastructural adjustments. This feature would not only benefit our organization but also support other users in the community with similar requirements for privacy, compliance, and on-premises data handling.

As a first proof of concept my colleague has written a standalone uploader that works well, even for bigger file-sizes (if the backend can handle it): goPutUploader

Would integrating something like this be an option?

from egress.

@Nitschi Yes, it could be a cool feature to have file uploader. Here I see one complexity it's how to deal with upload API as it can have different auth mechanisms and egress should support all of them to cover all organisations needs.

But in any case let's wait for Livekit team answer, they have to know much better 😊

from egress.

My guess would be, that 90% of users would be happy with a simple token/secret-based approach.
For our case we could use any of the many possibilites that libraries like ASP.NET supports, so less flexibility is needed on livekits side.

from egress.