ReviewOS

also looking at this

home-lang/home

feat(video): add native AVC/HEVC and synchronized AAC MP4 streaming

#913
Merged chris wants to merge feat/avc-hevc-mp4 into main
14 files +1119 -0
8 days ago

Home capture clients need real encoded AVC/HEVC packets and bounded MP4 recording with synchronized AAC audio. Add header-checked opaque x264/x265 adapters, checked strided RGB-to-I420 conversion, and a streaming fragmented MP4 writer. The writer supports video-only output or an optional AAC-LC track from an explicit encoder configuration; no FFmpeg/Libav or external encoder process is used.

The codec adapter validates sizes/configurations, supports ABR/CRF and GOP settings, exposes real parameter sets, requests actual IDRs for splits and drains delayed packets. The muxer writes complete avc1/hvc1/mp4a sample descriptions, avcC/hvcC/esds and exact fragment sizes/data offsets. Audio uses its own sample-rate clock. Seekable A/V sinks finalize bounded fixed duration/edit fields, actual codec priming, initial audio offset, explicit roll sample groups and a short last AAC duration that excludes padding. One audio and one caller-staged video packet bound latency/memory. The optional AAC encoder is contributed separately in #918; the muxer has no AAC library dependency.

Validation: targeted conversion/muxer tests pass on Zig 0.15.2 and the Home development toolchain, including invalid input, 64-bit time, sink poisoning, final edit/duration patches and bounded destinations. Native codec tests verify actual forced IDR NALs and delayed drain. Camera integration independently decodes AVC/HEVC with OpenH264/libde265 and AAC with FAAD2, checking frame order, waveforms, clocks, sample counts, splits and rotation. AVFoundation playback also validates AAC priming/tail/offset and short segments. Targeted pickier passes. Physical camera/audio timing, sustained Pi throughput, hardware encoding, HE-AAC and crash recovery remain unvalidated or unsupported. See packages/video/NATIVE_MEDIA.md.

14 changed files on the files tab, with 0 review threads.

Sign in to comment on this pull request.