cyanreg / avtransport Goto Github PK

libavtransport current status:

• I/O
  • FILE *
  • fd
  • UNIX socket
  • Network socket
  • mmap
  • liburing
  • Callback-based
• Protocols
  • Datagram stream
  • Stream (file)
    • Seeking
  • Quic (via OpenSSL)
  • Packet reordering for UDP and Quic
  • Callback-based
• Output
  • Packet scheduling
  • Compression
  • Session start
  • Stream registration
  • Stream data
  • Font data
  • Cube LUT
  • ICC profile
  • Video orientation
  • Video information
  • Metadata
  • LDPC encode
  • FEC parity
  • FEC grouping
• Input
  • Decompression
  • Session start
  • Stream registration
  • Stream data
  • Font data
  • Cube LUT
  • ICC profile
  • Video orientation
  • Video information
  • Metadata
  • LDPC encode
  • FEC parity
  • FEC grouping
• Robustness
  • Input reordering
  • Packet lifetime
  • Global sequence overflow considerations
  • Fuzzing
• Reliability
  • Retransmission
  • LDPC smart bit weighting (more confidence in bits likely to be correct)
• Optimizations
  • LDPC
    • SSE
    • AVX2
    • AArch64
    • RISC-V
  • FEC (RaptorQ)
    • SSE
    • AVX2
    • AArch64
    • RISC-V
  • Faster packet delimiting for corrupt files

Section 3.1.9. H264 encapsulation says

If they are present, they must contain an `AVCDecoderConfigurationRecord` structure, as defined in IEC 14496-15.

However, it's a common practice to not mix use Annex-B and avcc format. As you already know that in FFmpeg, we use extradata to determine the format of packets, assuming that these two has the same format.

The issue can be can solved by transform AVCDecoderConfigurationRecord into annexb extradata on receiver side. But I think it worth some highlight to avoid unexpected results.

The definition on related_stream_id and derived_stream_id is simple:

1. related_stream_id: Indicates the stream ID for which this stream is related to.
2. derived_stream_id: Indicates the stream ID for which this stream is derived from.

But it isn't clear what kind of cases belonging to 'related', and what's 'derived'. Some examples will be helpful to understand the definitions and which one to use.

brotli_dep = dependency('libzstd', required: false)

Oops?

@JeromeMartinez I've been working on this for a while now. @robUx4 helped out a bit as well, and broadcast folks also gave opinions. The plan is to eventually become part of the FFmpeg project, and get it submitted for an IETF standard once it's proven.
Could you or someone else review it, and give some feedback on how usable it would be for archival purposes? The bytestream is still very much not frozen.

The current specifications can be found here: https://cyanreg.github.io/avtransport/

It's got some features which may be of interest:

• 64 bit presentation and duration timestamps
• Actual timebase
• Only container to include mandatory error correction data on the headers themselves
• Large FEC range, using RaptorQ
• Extra resilient headers (each FEC segment carries a piece of the main header)
• Flexible keyframe indexing
• Lossless cutting of compressed video (any frame dependencies can be made invisible)
• Stream packet data compression
• A reasonable uncompressed video and audio encapsulation
• For H264 and HEVC, a real 64-bit dts timestamp.
• ICC profile support
• Color lookup table support (currently, only Adobe .cube files, which are pretty much the standard)
• DNG support
• Timestamp clock, allowing for jitter compensation
• Unified, cycle-accurate clock for exact synchronization between multiple streams.

Whenever I wondered what I should implement, I just took a look at Mastroska spec issues you've opened :)

It's definitely not as lightweight as Matroska is, each stream packet header is 36 bytes. Also, unlike Matroska, there is no clear file structure - AVTransport is a stream of packets, nothing more. Reordering, deduplication, caching, and so on are handled by the demuxer.

Also, there is no per-packet CRC, or a CRC at all - every header field is protected by error correction codes (which can be used to check integrity as well). The codes are standard irregular LDPCs with a coding ratio of 0.78, able to correct a lot more than a few bit flips.
As for the stream data, the optional FEC is RaptorQ, which is absurdly efficient at both finding faults and fixing them.
FEC may be applied to both individual packets, and groups of packets (which may include multiple streams, this is the traditional way FEC is done).

It was designed to be robust enough to be saved on tape, or sent over pure serial/RF/satellite/optical links, although it's mainly going to be used for point-to-point/point-to-many internet transmissions, and files.

Section multiplexing is an important task/function for media server. It isn't clear how to identify different sections from the spec. I have taken a glance on libavtransport/connection.c, looks like it use some special designed url. It doesn't included or referenced by the spec.

RTMP has tcUrl (rtmp://host:port/app/instance) in stream, SRT has streamid
https://haivision.github.io/srt-rfc/draft-sharabayko-srt.html#section-3.2.1.3

If avtransport protocol doesn't intend to include in-band section identifier, some suggestions on out-of-band identifier will be helpful for interoperation.