ReviewOS
Reload

also looking at this

home-lang/home

feat(video): add native AVC/HEVC and synchronized AAC MP4 streaming

#913
Merged chris wants to merge feat/avc-hevc-mp4 into main
14 files +1119 -0

Review threads live on the whole diff, not on one commit, so none are shown here - a thread's line means something in the branch's final form, and painting it into an intermediate step would put it on code it is not about.

.github/workflows/native-media.ymladded+34-0
@@ -0,0 +1,34 @@
1name: Native AVC HEVC and MP4
2on:
3 push:
4 paths:
5 - 'packages/video/src/codecs/video/native_encoder.*'
6 - 'packages/video/src/codecs/video/rgb_i420.zig'
7 - 'packages/video/src/containers/mp4_stream.zig'
8 - 'packages/video/src/native_media_tests.zig'
9 - 'packages/video/test-native-media.sh'
10 - '.github/workflows/native-media.yml'
11 pull_request:
12 paths:
13 - 'packages/video/src/codecs/video/native_encoder.*'
14 - 'packages/video/src/codecs/video/rgb_i420.zig'
15 - 'packages/video/src/containers/mp4_stream.zig'
16 - 'packages/video/src/native_media_tests.zig'
17 - 'packages/video/test-native-media.sh'
18 - '.github/workflows/native-media.yml'
19 workflow_dispatch:
20permissions:
21 contents: read
22jobs:
23 native-media:
24 strategy:
25 matrix:
26 os: [ubuntu-24.04, ubuntu-24.04-arm]
27 runs-on: ${{ matrix.os }}
28 steps:
29 - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
30 - uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29
31 with:
32 version: 0.15.2
33 - run: sudo apt-get update && sudo apt-get install -y libx264-dev libx265-dev pkg-config
34 - run: packages/video/test-native-media.sh
packages/video/NATIVE_MEDIA.mdadded+59-0
@@ -0,0 +1,59 @@
1# Native AVC/HEVC and streamed MP4
2
3`src/codecs/video/native_encoder.zig` is the supported standalone Home integration
4for libx264/libx265. Compile `native_encoder.c` against the installed vendor
5headers, then link x264/x265 directly. Only Home-owned fixed scalar structs cross
6the C/Zig boundary; the old handwritten vendor structures in x264.zig/x265.zig
7are not reused. There is no FFmpeg/Libav or encoder subprocess dependency.
8The Zig wrapper works with Zig 0.15.2 and the Home 0.17 development toolchain.
9Vendor libraries keep their GPL/commercial licenses; Home modules remain MIT.
10
11The wrapper accepts checked I420 planes, returns borrowed length-prefixed NAL
12packets, and preserves microsecond PTS. Serialize operations per encoder; packets
13expire on the next call. Encode may return null while AVC VFR buffers one input.
14Always drain `flush()` until null before deinit. Encoding after drain or error is
15rejected. The integration fixes presets to veryfast, one frame thread, no B frames
16or lookahead, closed GOPs, AVC High / HEVC Main, 8-bit 4:2:0. Quality 1..100 maps
17linearly to CRF 51..0 (AVC minimum 1 to retain High-profile compatibility); bitrate >0 selects ABR in bits/sec, not strict CBR. Zero GOP
18means 60. HEVC uses CTU16 across instances to avoid global x265 CTU mismatches.
19
20`rgb_i420.zig` is a bounded, stride-checked conversion from RGB24. It improves
21Home's existing RGB conversion approach with actual 2x2 chroma averaging. Output
22is limited-range BT.601 with sRGB transfer / BT.709 primaries. This is SDR.
23
24`src/containers/mp4_stream.zig` provides a sink-based single-video-track fragmented
25MP4 writer, adapted from Home's existing MP4 box code. It writes complete AVC/HEVC
26sample descriptions, avcC/hvcC from encoder headers, mvex/trex, accurate moof/traf/
27tfhd/trun offsets, 64-bit tfdt, and one mdat per sample. Payload writes go straight
28to the sink; there is no whole-recording buffer or growing sample table.
29
30The writer accepts length-prefixed samples with PTS=DTS and explicit microsecond
31durations. A recorder should retain one encoded packet to compute duration from
32the next timestamp, preserving capture gaps; use nominal duration for the last.
33The writer rejects malformed NAL lengths and non-increasing decode times. Partial
34sink failure poisons the writer. It has a fixed 16KiB header bound, no seek index,
35no audio, and supports the fixed 8-bit profiles emitted by this adapter.
36
37Home's older pure-Zig H.264/HEVC slice encoders and fragmented-muxer implementation
38remain legacy code; their feature lists are not conformance evidence for these
39new modules. The new modules are available through video.native_encoder,
40video.rgb_i420 and video.mp4_stream without pulling in the old codec graph.
41
42From the repository root on Linux:
43
44```sh
45sudo apt install libx264-dev libx265-dev pkg-config
46packages/video/test-native-media.sh
47```
48
49Focused tests validate geometry, original timestamps, all frames through drain,
50invalid lifecycle, averaged chroma, fragment sizes/data offsets/64-bit time and
51sink failure. They pass in Debug/ReleaseSafe with Zig 0.15.2 and Home's installed
520.17 development toolchain. zig-utils/zig-camera integration additionally checks
53MP4 regular/gapped capture timelines and GOPs, and decodes AVC/HEVC using independent
54Apple AVFoundation and Linux OpenH264/libde265. These are synthetic correctness
55tests, not Pi hardware FPS, long-run reliability or compression-quality benchmarks.
56
57References: [x265 API](https://x265.readthedocs.io/en/master/api.html),
58[x264 public header](https://code.videolan.org/videolan/x264/-/blob/master/x264.h),
59[ISO BMFF streaming requirements](https://www.w3.org/TR/mse-byte-stream-format-isobmff/).
packages/video/src/codecs/video/native_encoder.cadded+151-0
@@ -0,0 +1,151 @@
1/* Home's header-checked x264/x265 adapter. No hand-declared vendor structures. */
2#include "native_encoder.h"
3#include <stdlib.h>
4#include <string.h>
5#include <limits.h>
6#include <x264.h>
7#include <x265.h>
8struct home_encoder {
9 uint32_t codec;
10 x264_t *avc;
11 x265_encoder *hevc;
12 x265_param *param;
13 x265_picture *in, *out;
14 uint8_t *buffer;
15 size_t capacity;
16 uint32_t width, height;
17};
18void home_encoder_close(home_encoder *e) {
19 if (!e) return;
20 if (e->avc) x264_encoder_close(e->avc);
21 if (e->hevc) x265_encoder_close(e->hevc);
22 if (e->in) x265_picture_free(e->in);
23 if (e->out) x265_picture_free(e->out);
24 if (e->param) x265_param_free(e->param);
25 free(e->buffer); free(e);
26}
27int home_encoder_open(const home_encoder_config *c, home_encoder **out) {
28 if (!out) return -1;
29 *out=NULL;
30 if (!c || (c->codec!=1 && c->codec!=2) || c->width<16 || c->height<16 || c->width>8192 || c->height>8192 || (c->width|c->height)&1 || !c->fps_num || !c->fps_den || c->fps_num>1000000 || c->fps_den>1000000 || c->quality<1 || c->quality>100 || c->gop>INT_MAX || (c->bitrate && c->bitrate<1000)) return -1;
31 home_encoder *e=calloc(1,sizeof(*e));
32 if (!e) return -2;
33 e->codec=c->codec; e->width=c->width; e->height=c->height;
34 /* Quality 1..100 -> CRF 51..0; AVC clamps at 1 to keep High-profile
35 * compatibility (CRF0 requires High444 lossless). bitrate>0 selects ABR. */
36 const double crf=51.0*(100-c->quality)/99.0;
37 if (c->codec==1) {
38 x264_param_t p;
39 if (x264_param_default_preset(&p,"veryfast","zerolatency")<0) goto fail;
40 p.i_width=(int)c->width; p.i_height=(int)c->height; p.i_csp=X264_CSP_I420;
41 p.i_fps_num=c->fps_num; p.i_fps_den=c->fps_den;
42 p.i_timebase_num=1; p.i_timebase_den=1000000; p.b_vfr_input=1;
43 p.i_threads=1; p.i_bframe=0; p.rc.i_lookahead=0;
44 p.i_keyint_max=c->gop ? (int)c->gop : 60; p.i_keyint_min=1; p.i_scenecut_threshold=0;
45 p.b_open_gop=0; p.b_annexb=0; p.b_repeat_headers=0; p.i_log_level=X264_LOG_ERROR;
46 p.vui.b_fullrange=0; p.vui.i_colorprim=1; p.vui.i_transfer=13; p.vui.i_colmatrix=6;
47 p.rc.i_rc_method=c->bitrate ? X264_RC_ABR : X264_RC_CRF;
48 p.rc.i_bitrate=(int)(c->bitrate/1000); p.rc.f_rf_constant=(float)(crf<1.0 ? 1.0 : crf);
49 if (x264_param_apply_profile(&p,"high")<0) { x264_param_cleanup(&p); goto fail; }
50 e->avc=x264_encoder_open(&p);
51 x264_param_cleanup(&p);
52 if (!e->avc) goto fail;
53 } else {
54 e->param=x265_param_alloc(); e->in=x265_picture_alloc(); e->out=x265_picture_alloc();
55 if (!e->param || !e->in || !e->out) goto fail;
56 x265_param *p=e->param;
57 if (x265_param_default_preset(p,"veryfast","zerolatency")<0) goto fail;
58 p->sourceWidth=(int)c->width; p->sourceHeight=(int)c->height;
59 p->fpsNum=c->fps_num; p->fpsDenom=c->fps_den; p->internalCsp=X265_CSP_I420; p->internalBitDepth=8;
60 p->bframes=0; p->lookaheadDepth=0; p->frameNumThreads=1;
61 p->numaPools="none"; p->bEnableWavefront=0;
62 p->keyframeMax=c->gop ? (int)c->gop : 60; p->keyframeMin=1; p->scenecutThreshold=0; p->bOpenGOP=0;
63 p->bAnnexB=0; p->bRepeatHeaders=0; p->bEmitInfoSEI=0; p->logLevel=X265_LOG_ERROR;
64 /* Fixed CTU sizes across instances avoid x265's process-global CTU conflict. */
65 p->maxCUSize=16; p->minCUSize=8;
66 p->vui.bEnableVideoSignalTypePresentFlag=1; p->vui.bEnableVideoFullRangeFlag=0;
67 p->vui.bEnableColorDescriptionPresentFlag=1; p->vui.colorPrimaries=1;
68 p->vui.transferCharacteristics=13; p->vui.matrixCoeffs=6;
69 p->rc.rateControlMode=c->bitrate ? X265_RC_ABR : X265_RC_CRF;
70 p->rc.bitrate=(int)(c->bitrate/1000); p->rc.rfConstant=crf;
71 if (x265_param_apply_profile(p,"main")<0) goto fail;
72 e->hevc=x265_encoder_open(p);
73 if (!e->hevc) goto fail;
74 x265_picture_init(p,e->in); x265_picture_init(p,e->out);
75 e->in->bitDepth=8;
76 }
77 *out=e; return 0;
78fail:
79 home_encoder_close(e); return -3;
80}
81static int collect(home_encoder *e, x264_nal_t *a, x265_nal *h, uint32_t count, home_packet *out) {
82 size_t total=0;
83 for (uint32_t i=0;i<count;i++) {
84 size_t n=a ? (size_t)a[i].i_payload : h[i].sizeBytes;
85 if (n>SIZE_MAX-total) return -2;
86 total+=n;
87 }
88 if (total>e->capacity) {
89 uint8_t *p=realloc(e->buffer,total);
90 if (!p) return -2;
91 e->buffer=p; e->capacity=total;
92 }
93 size_t offset=0;
94 for (uint32_t i=0;i<count;i++) {
95 size_t n=a ? (size_t)a[i].i_payload : h[i].sizeBytes;
96 memcpy(e->buffer+offset,a ? a[i].p_payload : h[i].payload,n); offset+=n;
97 }
98 out->data=e->buffer; out->size=total; return 0;
99}
100int home_encoder_headers(home_encoder *e, home_packet *out) {
101 memset(out,0,sizeof(*out));
102 if (e->avc) {
103 x264_nal_t *nal=NULL; int n=0;
104 if (x264_encoder_headers(e->avc,&nal,&n)<0 || n<=0) return -3;
105 return collect(e,nal,NULL,(uint32_t)n,out);
106 }
107 x265_nal *nal=NULL; uint32_t n=0;
108 if (x265_encoder_headers(e->hevc,&nal,&n)<0 || !n) return -3;
109 return collect(e,NULL,nal,n,out);
110}
111int home_encoder_encode(home_encoder *e, const uint8_t *const planes[3], const int strides[3], int64_t pts, home_packet *out) {
112 memset(out,0,sizeof(*out));
113 if (!planes[0] || !planes[1] || !planes[2] || strides[0]<(int)e->width || strides[1]<(int)e->width/2 || strides[2]<(int)e->width/2) return -1;
114 if (e->avc) {
115 x264_picture_t in, decoded; x264_picture_init(&in); x264_picture_init(&decoded);
116 in.img.i_csp=X264_CSP_I420; in.img.i_plane=3; in.i_pts=pts;
117 for (int i=0;i<3;i++) { in.img.plane[i]=(uint8_t *)planes[i]; in.img.i_stride[i]=strides[i]; }
118 x264_nal_t *nal=NULL; int n=0;
119 int ret=x264_encoder_encode(e->avc,&nal,&n,&in,&decoded);
120 if(ret<0) return -3;
121 if(ret==0) return 0;
122 out->pts=decoded.i_pts; out->key=decoded.b_keyframe;
123 return collect(e,nal,NULL,(uint32_t)n,out);
124 }
125 for (int i=0;i<3;i++) { e->in->planes[i]=(void *)planes[i]; e->in->stride[i]=strides[i]; }
126 e->in->pts=pts;
127 x265_nal *nal=NULL; uint32_t n=0;
128 if (x265_encoder_encode(e->hevc,&nal,&n,e->in,e->out)<=0 || !n) return -3;
129 out->pts=e->out->pts; out->key=IS_X265_TYPE_I(e->out->sliceType);
130 return collect(e,NULL,nal,n,out);
131}
132
133int home_encoder_flush(home_encoder *e, home_packet *out) {
134 memset(out,0,sizeof(*out));
135 if(e->avc) {
136 if(!x264_encoder_delayed_frames(e->avc)) return 0;
137 x264_picture_t picture; x264_picture_init(&picture);
138 x264_nal_t *nal=NULL; int n=0;
139 int ret=x264_encoder_encode(e->avc,&nal,&n,NULL,&picture);
140 if(ret<0) return -3;
141 if(ret==0) return 0;
142 out->pts=picture.i_pts;out->key=picture.b_keyframe;
143 return collect(e,nal,NULL,(uint32_t)n,out);
144 }
145 x265_nal *nal=NULL;uint32_t n=0;
146 int ret=x265_encoder_encode(e->hevc,&nal,&n,NULL,e->out);
147 if(ret<0) return -3;
148 if(ret==0) return 0;
149 out->pts=e->out->pts;out->key=IS_X265_TYPE_I(e->out->sliceType);
150 return collect(e,NULL,nal,n,out);
151}
packages/video/src/codecs/video/native_encoder.hadded+17-0
@@ -0,0 +1,17 @@
1/* Home codec bridge, MIT. Compile against installed x264.h/x265.h. */
2#ifndef HOME_NATIVE_ENCODER_H
3#define HOME_NATIVE_ENCODER_H
4#include <stddef.h>
5#include <stdint.h>
6typedef struct home_encoder home_encoder;
7typedef struct { uint32_t codec, width, height, fps_num, fps_den, bitrate, quality, gop; } home_encoder_config;
8/* Length-prefixed NAL units; borrowed until the next call. pts uses microseconds.
9 * Low latency: no B frames/lookahead; AVC VFR buffers one input for duration.
10 * Empty output means buffered. Drain flush() before close; serialize calls. */
11typedef struct { const uint8_t *data; size_t size; int64_t pts; int key; } home_packet;
12int home_encoder_open(const home_encoder_config *, home_encoder **);
13int home_encoder_headers(home_encoder *, home_packet *);
14int home_encoder_encode(home_encoder *, const uint8_t *const planes[3], const int strides[3], int64_t pts, home_packet *);
15int home_encoder_flush(home_encoder *, home_packet *);
16void home_encoder_close(home_encoder *);
17#endif
packages/video/src/codecs/video/native_encoder.zigadded+75-0
@@ -0,0 +1,75 @@
1//! Safe optional Home AVC/HEVC integration. Compile native_encoder.c and link
2//! x264/x265. Only Home's fixed scalar ABI crosses the boundary, never vendor
3//! structures; this module works before/after Zig's removal of @cImport.
4const std = ;
5const Handle = opaque {};
6pub const Config = extern struct { codec: u32, width: u32, height: u32, fps_num: u32, fps_den: u32, bitrate: u32, quality: u32, gop: u32 };
7const RawPacket = extern struct { data: [*]const u8, size: usize, pts: i64, key: c_int };
8extern fn home_encoder_open(*const Config, *?*Handle) c_int;
9extern fn home_encoder_headers(*Handle, *RawPacket) c_int;
10extern fn home_encoder_encode(*Handle, *const [3][*]const u8, *const [3]c_int, i64, *RawPacket) c_int;
11extern fn home_encoder_flush(*Handle, *RawPacket) c_int;
12extern fn home_encoder_close(*Handle) void;
13pub const Packet = struct { data: []const u8, pts: i64, key: bool };
14pub const Encoder = struct {
15 handle: *Handle,
16 width: u32,
17 height: u32,
18 last_pts: ?i64 = null,
19 draining: bool = false,
20 failed: bool = false,
21 pub fn init(config: Config) !Encoder {
22 var out: ?*Handle = null;
23 try check(home_encoder_open(&config, &out));
24 return .{ .handle = out.?, .width = config.width, .height = config.height };
25 }
26 pub fn deinit(self: *Encoder) void {
27 home_encoder_close(self.handle);
28 }
29 pub fn headers(self: *Encoder) ![]const u8 {
30 var packet: RawPacket = undefined;
31 try check(home_encoder_headers(self.handle, &packet));
32 return packet.data[0..packet.size];
33 }
34 pub fn flush(self: *Encoder) !?Packet {
35 if (self.failed) return error.EncoderFailed;
36 self.draining = true;
37 var packet: RawPacket = undefined;
38 errdefer self.failed = true;
39 try check(home_encoder_flush(self.handle, &packet));
40 return view(packet);
41 }
42 /// Borrowed output lasts until the next call; input may be released on return.
43 /// Null means buffered (AVC VFR retains one input); always drain flush().
44 pub fn encode(self: *Encoder, planes: [3][]const u8, strides: [3]u32, width: u32, height: u32, pts: i64) !?Packet {
45 if (self.failed or self.draining) return error.EncoderFailed;
46 if (pts < 0 or (self.last_pts != null and pts <= self.last_pts.?)) return error.InvalidTimestamp;
47 if (width != self.width or height != self.height) return error.InvalidFrame;
48 for (0..3) |i| {
49 const w = if (i == 0) width else width / 2;
50 const h = if (i == 0) height else height / 2;
51 if (strides[i] < w or strides[i] > std.math.maxInt(c_int) or planes[i].len < @as(u64, strides[i]) * (h - 1) + w) return error.InvalidFrame;
52 }
53 const pointers = [3][*]const u8{ planes[0].ptr, planes[1].ptr, planes[2].ptr };
54 const pitch = [3]c_int{ @intCast(strides[0]), @intCast(strides[1]), @intCast(strides[2]) };
55 var packet: RawPacket = undefined;
56 check(home_encoder_encode(self.handle, &pointers, &pitch, pts, &packet)) catch |err| {
57 self.failed = true;
58 return err;
59 };
60 self.last_pts = pts;
61 return view(packet);
62 }
63};
64fn check(status: c_int) !void {
65 return switch (status) {
66 0 => {},
67 -1 => error.InvalidConfiguration,
68 -2 => error.OutOfMemory,
69 else => error.EncodeFailed,
70 };
71}
72fn view(packet: RawPacket) ?Packet {
73 if (packet.size == 0) return null;
74 return .{ .data = packet.data[0..packet.size], .pts = packet.pts, .key = packet.key != 0 };
75}
packages/video/src/codecs/video/rgb_i420.zigadded+37-0
@@ -0,0 +1,37 @@
1//! Home RGB -> limited-range BT.601 I420. Adapted from perf/simd.zig, with
2//! checked strides and actual 2x2 chroma averaging rather than corner sampling.
3const std = ;
4pub fn convert(rgb: []const u8, width: u32, height: u32, stride: u32, out: []u8) !void {
5 if (width == 0 or height == 0 or (width | height) & 1 != 0 or stride < @as(u64, width) * 3) return error.InvalidGeometry;
6 const pixels = std.math.mul(usize, width, height) catch return error.InvalidGeometry;
7 if (rgb.len < @as(u64, stride) * (height - 1) + @as(u64, width) * 3 or out.len < pixels + pixels / 2) return error.InvalidGeometry;
8 for (0..height / 2) |cy| {
9 for (0..width / 2) |cx| {
10 var r: i32 = 0;
11 var g: i32 = 0;
12 var b: i32 = 0;
13 for (0..2) |dy| {
14 for (0..2) |dx| {
15 const x = cx * 2 + dx;
16 const y = cy * 2 + dy;
17 const p = rgb[y * stride + x * 3 ..][0..3];
18 out[y * width + x] = @intCast(((66 * @as(i32, p[0]) + 129 * @as(i32, p[1]) + 25 * @as(i32, p[2]) + 128) >> 8) + 16);
19 r += p[0];
20 g += p[1];
21 b += p[2];
22 }
23 }
24 const uv = cy * (width / 2) + cx;
25 out[pixels + uv] = @intCast(((-38 * r - 74 * g + 112 * b + 512) >> 10) + 128);
26 out[pixels + pixels / 4 + uv] = @intCast(((112 * r - 94 * g - 18 * b + 512) >> 10) + 128);
27 }
28 }
29}
30test "strided RGB conversion averages chroma and rejects truncation" {
31 var out: [6]u8 = undefined;
32 const rgb = [_]u8{ 255, 0, 0, 0, 255, 0, 99, 99, 0, 0, 255, 255, 255, 255 };
33 try convert(&rgb, 2, 2, 8, &out);
34 try std.testing.expectEqualSlices(u8, &.{ 82, 144, 41, 235, 128, 128 }, &out);
35 try std.testing.expectError(error.InvalidGeometry, convert(rgb[0..13], 2, 2, 8, &out));
36 try std.testing.expectError(error.InvalidGeometry, convert(&rgb, 1, 2, 8, &out));
37}
packages/video/src/codecs/video/x264.zigmodified+2-0
@@ -1,3 +1,5 @@
1// Legacy handwritten vendor ABI. New integrations should use native_encoder.zig,
2// whose C boundary is compiled against installed headers. See NATIVE_MEDIA.md.
13// Home Video Library - x264 FFI Integration
24// Production H.264 encoding via libx264
35// https://www.videolan.org/developers/x264.html
packages/video/src/codecs/video/x265.zigmodified+2-0
@@ -1,3 +1,5 @@
1// Legacy handwritten vendor ABI. New integrations should use native_encoder.zig,
2// whose C boundary is compiled against installed headers. See NATIVE_MEDIA.md.
13// Home Video Library - x265 FFI Integration
24// Production H.265/HEVC encoding via libx265
35// https://x265.readthedocs.io/
packages/video/src/containers/mp4_stream.zigadded+302-0
@@ -0,0 +1,302 @@
1//! Home single-track streaming fragmented MP4, adapted from mp4_muxer.zig.
2//! Complete sample description + mvex; exact sizes/offsets; no file-sized buffer.
3//! AVC/HEVC are length-prefixed, with out-of-band parameter sets (avc1/hvc1).
4const std = ;
5pub const Codec = enum { h264, hevc };
6pub const Sink = struct { context: *anyopaque, write: *const fn (*anyopaque, []const u8) anyerror!void };
7const Buffer = struct {
8 data: [16384]u8 = undefined,
9 len: usize = 0,
10 fn bytes(s: *Buffer, b: []const u8) !void {
11 if (b.len > s.data.len - s.len) return error.HeaderTooLarge;
12 @memcpy(s.data[s.len..][0..b.len], b);
13 s.len += b.len;
14 }
15 fn zero(s: *Buffer, n: usize) !void {
16 if (n > s.data.len - s.len) return error.HeaderTooLarge;
17 @memset(s.data[s.len..][0..n], 0);
18 s.len += n;
19 }
20 fn int16(s: *Buffer, n: u16) !void {
21 var b: [2]u8 = undefined;
22 std.mem.writeInt(u16, &b, n, .big);
23 try s.bytes(&b);
24 }
25 fn int32(s: *Buffer, n: u32) !void {
26 var b: [4]u8 = undefined;
27 std.mem.writeInt(u32, &b, n, .big);
28 try s.bytes(&b);
29 }
30 fn int64(s: *Buffer, n: u64) !void {
31 var b: [8]u8 = undefined;
32 std.mem.writeInt(u64, &b, n, .big);
33 try s.bytes(&b);
34 }
35 fn box(s: *Buffer, name: *const [4]u8) !usize {
36 const at = s.len;
37 try s.int32(0);
38 try s.bytes(name);
39 return at;
40 }
41 fn end(s: *Buffer, at: usize) void {
42 std.mem.writeInt(u32, s.data[at..][0..4], @intCast(s.len - at), .big);
43 }
44 fn full(s: *Buffer, name: *const [4]u8, flags: u32) !usize {
45 const at = try s.box(name);
46 try s.int32(flags);
47 return at;
48 }
49 fn slice(s: *const Buffer) []const u8 {
50 return s.data[0..s.len];
51 }
52 fn matrix(s: *Buffer) !void {
53 for ([_]u32{ 0x10000, 0, 0, 0, 0x10000, 0, 0, 0, 0x40000000 }) |v| try s.int32(v);
54 }
55};
56pub const Nals = struct {
57 data: []const u8,
58 at: usize = 0,
59 pub fn next(s: *Nals) !?[]const u8 {
60 if (s.at == s.data.len) return null;
61 if (s.data.len - s.at < 4) return error.InvalidNal;
62 const n = std.mem.readInt(u32, s.data[s.at..][0..4], .big);
63 s.at += 4;
64 if (n == 0 or n > s.data.len - s.at) return error.InvalidNal;
65 const nal = s.data[s.at..][0..n];
66 s.at += n;
67 return nal;
68 }
69};
70fn config(b: *Buffer, codec: Codec, headers: []const u8) !void {
71 var nals: Nals = .{ .data = headers };
72 var sps: []const u8 = &.{};
73 var pps: []const u8 = &.{};
74 var vps: []const u8 = &.{};
75 while (try nals.next()) |nal| {
76 if (nal.len > 2048) return error.HeaderTooLarge;
77 const kind = if (codec == .h264) nal[0] & 31 else (nal[0] >> 1) & 63;
78 if (kind == (if (codec == .h264) @as(u8, 7) else 33)) sps = nal;
79 if (kind == (if (codec == .h264) @as(u8, 8) else 34)) pps = nal;
80 if (codec == .hevc and kind == 32) vps = nal;
81 }
82 if (sps.len < 4 or pps.len < 2) return error.MissingParameterSets;
83 const at = try b.box(if (codec == .h264) "avcC" else "hvcC");
84 if (codec == .h264) {
85 // The adapter fixes AVC to 8-bit 4:2:0 High profile.
86 if (sps[1] != 100) return error.UnsupportedProfile;
87 try b.bytes(&.{ 1, sps[1], sps[2], sps[3], 255, 225 });
88 try b.int16(@intCast(sps.len));
89 try b.bytes(sps);
90 try b.bytes(&.{1});
91 try b.int16(@intCast(pps.len));
92 try b.bytes(pps);
93 try b.bytes(&.{ 253, 248, 248, 0 }); // high profile: 4:2:0, 8-bit, no SPS extensions
94 } else {
95 if (vps.len < 2 or sps.len < 16) return error.MissingParameterSets;
96 // Remove emulation prevention before reading SPS profile_tier_level.
97 var rbsp: [2048]u8 = undefined;
98 var len: usize = 0;
99 var zeros: usize = 0;
100 for (sps[2..]) |byte| {
101 if (zeros >= 2 and byte == 3) {
102 zeros = 0;
103 continue;
104 }
105 rbsp[len] = byte;
106 len += 1;
107 zeros = if (byte == 0) zeros + 1 else 0;
108 }
109 if (len < 13 or (rbsp[0] & 14) != 0 or (rbsp[1] & 31) != 1) return error.UnsupportedProfile;
110 try b.bytes(&.{1});
111 try b.bytes(rbsp[1..13]);
112 try b.bytes(&.{ 240, 0, 252, 253, 248, 248, 0, 0, 15, 3 });
113 for ([_]u8{ 32, 33, 34 }, [_][]const u8{ vps, sps, pps }) |kind, nal| {
114 try b.bytes(&.{128 | kind});
115 try b.int16(1);
116 try b.int16(@intCast(nal.len));
117 try b.bytes(nal);
118 }
119 }
120 b.end(at);
121}
122pub const Writer = struct {
123 sink: Sink,
124 sequence: u32 = 1,
125 last_dts: u64 = 0,
126 frames: u64 = 0,
127 failed: bool = false,
128 pub fn init(sink: Sink, width: u32, height: u32, codec: Codec, headers: []const u8) !Writer {
129 if (width == 0 or height == 0 or width > 8192 or height > 8192) return error.InvalidConfiguration;
130 var b: Buffer = .{};
131 const ftyp = try b.box("ftyp");
132 try b.bytes("iso6");
133 try b.int32(512);
134 try b.bytes("iso6isommp41");
135 b.end(ftyp);
136 const moov = try b.box("moov");
137 const mvhd = try b.full("mvhd", 0);
138 try b.zero(8);
139 try b.int32(1000000);
140 try b.int32(0);
141 try b.int32(0x10000);
142 try b.int16(0x100);
143 try b.zero(10);
144 try b.matrix();
145 try b.zero(24);
146 try b.int32(2);
147 b.end(mvhd);
148 const trak = try b.box("trak");
149 const tkhd = try b.full("tkhd", 7);
150 try b.zero(8);
151 try b.int32(1);
152 try b.zero(16);
153 try b.zero(8);
154 try b.matrix();
155 try b.int32(width << 16);
156 try b.int32(height << 16);
157 b.end(tkhd);
158 const mdia = try b.box("mdia");
159 const mdhd = try b.full("mdhd", 0);
160 try b.zero(8);
161 try b.int32(1000000);
162 try b.int32(0);
163 try b.int16(0x55c4);
164 try b.int16(0);
165 b.end(mdhd);
166 const hdlr = try b.full("hdlr", 0);
167 try b.int32(0);
168 try b.bytes("vide");
169 try b.zero(12);
170 try b.bytes("Home Video\x00");
171 b.end(hdlr);
172 const minf = try b.box("minf");
173 const vmhd = try b.full("vmhd", 1);
174 try b.zero(8);
175 b.end(vmhd);
176 const dinf = try b.box("dinf");
177 const dref = try b.full("dref", 0);
178 try b.int32(1);
179 const url = try b.full("url ", 1);
180 b.end(url);
181 b.end(dref);
182 b.end(dinf);
183 const stbl = try b.box("stbl");
184 const stsd = try b.full("stsd", 0);
185 try b.int32(1);
186 const entry = try b.box(if (codec == .h264) "avc1" else "hvc1");
187 try b.zero(6);
188 try b.int16(1);
189 try b.zero(16);
190 try b.int16(@intCast(width));
191 try b.int16(@intCast(height));
192 try b.int32(0x480000);
193 try b.int32(0x480000);
194 try b.int32(0);
195 try b.int16(1);
196 try b.zero(32);
197 try b.int16(24);
198 try b.int16(65535);
199 try config(&b, codec, headers);
200 const colr = try b.box("colr");
201 try b.bytes("nclx");
202 try b.int16(1);
203 try b.int16(13);
204 try b.int16(6);
205 try b.bytes(&.{0});
206 b.end(colr);
207 b.end(entry);
208 b.end(stsd);
209 for ([_]*const [4]u8{ "stts", "stsc", "stco" }) |name| {
210 const at = try b.full(name, 0);
211 try b.int32(0);
212 b.end(at);
213 }
214 const stsz = try b.full("stsz", 0);
215 try b.zero(8);
216 b.end(stsz);
217 b.end(stbl);
218 b.end(minf);
219 b.end(mdia);
220 b.end(trak);
221 const mvex = try b.box("mvex");
222 const trex = try b.full("trex", 0);
223 try b.int32(1);
224 try b.int32(1);
225 try b.zero(12);
226 b.end(trex);
227 b.end(mvex);
228 b.end(moov);
229 try sink.write(sink.context, b.slice());
230 return .{ .sink = sink };
231 }
232 /// One complete sample. Caller supplies its actual duration, using the next
233 /// capture time; the last sample uses nominal frame duration. PTS == DTS.
234 pub fn frame(s: *Writer, sample: []const u8, dts_us: u64, duration_us: u32, key: bool) !void {
235 if (s.failed) return error.WriterFailed;
236 if (sample.len == 0 or sample.len > std.math.maxInt(u32) - 8 or duration_us == 0 or (s.frames != 0 and dts_us <= s.last_dts) or s.sequence == std.math.maxInt(u32)) return error.InvalidSample;
237 var nals: Nals = .{ .data = sample };
238 while (try nals.next()) |_| {}
239 var b: Buffer = .{};
240 const moof = try b.box("moof");
241 const mfhd = try b.full("mfhd", 0);
242 try b.int32(s.sequence);
243 b.end(mfhd);
244 const traf = try b.box("traf");
245 const tfhd = try b.full("tfhd", 0x20000);
246 try b.int32(1);
247 b.end(tfhd);
248 const tfdt = try b.full("tfdt", 0x1000000);
249 try b.int64(dts_us);
250 b.end(tfdt);
251 const trun = try b.full("trun", 0x701);
252 try b.int32(1);
253 const offset = b.len;
254 try b.int32(0);
255 try b.int32(duration_us);
256 try b.int32(@intCast(sample.len));
257 try b.int32(if (key) 0x02000000 else 0x01010000);
258 b.end(trun);
259 b.end(traf);
260 b.end(moof);
261 std.mem.writeInt(u32, b.data[offset..][0..4], @intCast(b.len + 8), .big);
262 try b.int32(@intCast(sample.len + 8));
263 try b.bytes("mdat");
264 errdefer s.failed = true;
265 try s.sink.write(s.sink.context, b.slice());
266 try s.sink.write(s.sink.context, sample);
267 s.sequence += 1;
268 s.frames += 1;
269 s.last_dts = dts_us;
270 }
271};
272test "NAL lengths reject truncation and empty units" {
273 var n: Nals = .{ .data = &.{ 0, 0, 0, 2, 0x65 } };
274 try std.testing.expectError(error.InvalidNal, n.next());
275 n = .{ .data = &.{ 0, 0, 0, 0 } };
276 try std.testing.expectError(error.InvalidNal, n.next());
277}
278test "fragment sizes, offsets and 64-bit time are exact; sink failure poisons writer" {
279 const Out = struct {
280 data: std.ArrayList(u8) = .empty,
281 fail: bool = false,
282 fn write(ctx: *anyopaque, bytes: []const u8) anyerror!void {
283 const s: *@This() = @ptrCast(@alignCast(ctx));
284 if (s.fail) return error.DiskFull;
285 try s.data.appendSlice(std.testing.allocator, bytes);
286 }
287 };
288 var out: Out = .{};
289 defer out.data.deinit(std.testing.allocator);
290 var writer: Writer = .{ .sink = .{ .context = &out, .write = Out.write } };
291 try writer.frame(&.{ 0, 0, 0, 2, 0x65, 0x80 }, 0x100000000, 41667, true);
292 const bytes = out.data.items;
293 const size = std.mem.readInt(u32, bytes[0..4], .big);
294 try std.testing.expectEqualStrings("moof", bytes[4..8]);
295 try std.testing.expectEqual(@as(u32, 114), std.mem.readInt(u32, bytes[size..][0..4], .big) + size);
296 try std.testing.expectEqual(@as(u64, 0x100000000), std.mem.readInt(u64, bytes[60..68], .big));
297 try std.testing.expectEqual(size + 8, std.mem.readInt(u32, bytes[84..88], .big));
298 try std.testing.expectError(error.InvalidSample, writer.frame(&.{ 0, 0, 0, 1, 1 }, 0x100000000, 1, false));
299 out.fail = true;
300 try std.testing.expectError(error.DiskFull, writer.frame(&.{ 0, 0, 0, 1, 1 }, 0x100000001, 1, false));
301 try std.testing.expectError(error.WriterFailed, writer.frame(&.{ 0, 0, 0, 1, 1 }, 0x100000002, 1, false));
302}
packages/video/src/native_media_tests.zigadded+49-0
@@ -0,0 +1,49 @@
1//! Focused optional codec suite, independent of Home's broad video import graph.
2const std = ;
3const native = ;
4test {
5 _ = ;
6 _ = ;
7}
8test "AVC/HEVC drain every frame with original timestamps and reject invalid lifecycle" {
9 var y: [1024]u8 = undefined;
10 @memset(&y, 90);
11 var uv: [256]u8 = undefined;
12 @memset(&uv, 128);
13 for ([_]u32{ 1, 2 }) |codec| {
14 var encoder = try native.Encoder.init(.{ .codec = codec, .width = 32, .height = 32, .fps_num = 24, .fps_den = 1, .quality = 90, .bitrate = 0, .gop = 4 });
15 defer encoder.deinit();
16 try std.testing.expect((try encoder.headers()).len > 8);
17 var count: usize = 0;
18 for (0..6) |i| {
19 const pts: i64 = @intCast(i * 41667);
20 if (try encoder.encode(.{ &y, &uv, &uv }, .{ 32, 16, 16 }, 32, 32, pts)) |packet| {
21 try std.testing.expectEqual(@as(i64, @intCast(count * 41667)), packet.pts);
22 count += 1;
23 }
24 try std.testing.expectError(error.InvalidTimestamp, encoder.encode(.{ &y, &uv, &uv }, .{ 32, 16, 16 }, 32, 32, pts));
25 }
26 while (try encoder.flush()) |packet| {
27 try std.testing.expectEqual(@as(i64, @intCast(count * 41667)), packet.pts);
28 count += 1;
29 }
30 try std.testing.expectEqual(@as(usize, 6), count);
31 try std.testing.expectError(error.EncoderFailed, encoder.encode(.{ &y, &uv, &uv }, .{ 32, 16, 16 }, 32, 32, 999999));
32 }
33}
34test "codec configuration and plane bounds fail before vendor calls" {
35 try std.testing.expectError(error.InvalidConfiguration, native.Encoder.init(.{ .codec = 1, .width = 17, .height = 16, .fps_num = 30, .fps_den = 1, .quality = 90, .bitrate = 0, .gop = 0 }));
36 var encoder = try native.Encoder.init(.{ .codec = 1, .width = 32, .height = 32, .fps_num = 30, .fps_den = 1, .quality = 90, .bitrate = 0, .gop = 0 });
37 defer encoder.deinit();
38 try std.testing.expectError(error.InvalidFrame, encoder.encode(.{ &.{}, &.{}, &.{} }, .{ 32, 16, 16 }, 32, 32, 0));
39}
40test "quality endpoints preserve supported profiles" {
41 for ([_]u32{ 1, 2 }) |codec| {
42 for ([_]u32{ 1, 100 }) |quality| {
43 var encoder = try native.Encoder.init(.{ .codec = codec, .width = 32, .height = 32, .fps_num = 30, .fps_den = 1, .quality = quality, .bitrate = 0, .gop = 0 });
44 defer encoder.deinit();
45 try std.testing.expect((try encoder.headers()).len > 8);
46 try std.testing.expectEqual(@as(?native.Packet, null), try encoder.flush());
47 }
48 }
49}
packages/video/src/video.zigmodified+6-0
@@ -97,6 +97,12 @@ pub const TrackInfo = mp4.TrackInfo;
9797pub const TrackType = mp4.TrackType;
9898pub const SampleTable = mp4.SampleTable;
9999
100// Optional header-checked AVC/HEVC integration: compile native_encoder.c and
101// link x264/x265 only when this module is used. See NATIVE_MEDIA.md.
102pub const native_encoder = ;
103pub const rgb_i420 = ;
104pub const mp4_stream = ;
105
100106pub const mp4_muxer = ;
101107pub const Mp4Muxer = mp4_muxer.Mp4Muxer;
102108pub const VideoTrackConfig = mp4_muxer.VideoTrackConfig;
packages/video/test-native-media.shadded+11-0
@@ -0,0 +1,11 @@
1#!/bin/sh
2# Focused suite for the optional Home AVC/HEVC + MP4 modules. No FFmpeg.
3set -eu
4cd "$(dirname "$0")"
5build_dir=$(mktemp -d "${TMPDIR:-/tmp}/home-native-media.XXXXXX")
6trap 'rm -rf "$build_dir"' EXIT HUP INT TERM
7cc -std=c11 -Wall -Wextra -Werror $(pkg-config --cflags x264 x265) -c src/codecs/video/native_encoder.c -o "$build_dir/encoder.o"
8${ZIG:-zig} fmt --check src/codecs/video/native_encoder.zig src/codecs/video/rgb_i420.zig src/containers/mp4_stream.zig src/native_media_tests.zig
9for mode in Debug ReleaseSafe; do
10 ${ZIG:-zig} test src/native_media_tests.zig -O "$mode" -lc "$build_dir/encoder.o" $(pkg-config --libs x264 x265) ${HOME_MEDIA_LDFLAGS:-}
11done