#17 · stacks/drone

What to build

Turn the current 2D selected-person observation into a calibrated local-frame measurement only when range/depth and vehicle pose are available.

Acceptance criteria

Calibrate intrinsics/extrinsics, lens/ROI mapping, synchronization and position uncertainty against independent ground truth. Software import/validation and independent tabulated synthetic ground truth are implemented; measured physical calibration and trajectories remain required.
Pair the visual lock with a chosen GNSS/target source explicitly; never infer metres from an uncalibrated box. Implemented in the owned simulator and declared-bench data contracts.
Test crossings, occlusion, reacquisition, identity drift and camera loss; add appearance tracking only with measured privacy and accuracy behavior. Synthetic failure/reselection campaigns are implemented; no appearance or biometric re-identification has been added.

Implemented software

6660d1b adds:

Owned fixed Brown5/pinhole lens, crop/output mode and camera-to-body mount calibration import, checked against independent held-out 3D correspondences. This evaluates a supplied calibration; it does not fit checkerboard images or authenticate provenance.
Exact current selected candidate/frame/capture association, registered optical Z at the selected point, capture-time owned estimator pose, explicit heading alignment and bounded synchronization/uncertainty. Generic closest-obstacle range and box-size distance estimates cannot qualify.
Explicit camera/lock/calibration/depth and GNSS source/session/target/reference pairing, visible-point offset uncertainty, conservative covariance intersection and innovation/motion gates.
Qualified expiry derived from the oldest contributing capture, configured age limit and clock uncertainty, preserved through retained fusion packets and navigation. Loss, replay, reselection, changed scope, ambiguity and source disagreement latch release; good data cannot resume control automatically.
Actual raw IMU/GNSS -> owned estimator -> qualified capture pose -> calibrated vision/fusion -> follow navigation -> authority/controller -> independent modeled board -> rigid-body simulation. Rendering uses plant truth separately; localization and controls use the estimator.
Native Stacks vision:validate and vision:simulate, compiled standalone CLI support and stx /vision recording/playback. Playback withholds future pairing information.

Read the vision contract, ADR 17, calibration example and synthetic scenario evidence.

Verification

Exact pushed-head CI passed 393 Bun tests, 24,031 assertions and 13 Python bridge tests, with zero failures. Final affected local vision/navigation/camera/CLI suite: 76 tests, 698 assertions, zero failures.
Lint, application/runtime/view typechecks, roadmap/parity integrity, 13 Python bridge tests and standalone CLI build passed. The audit integrity gate still explicitly reports full parity incomplete.
Compiled CLI executed outside the checkout: calibration validation and nominal/identity-loss scenarios pass; a 10-second run ending before delayed fault observation exits unsuccessfully.
Browser: nominal completion, crossing/camera loss, clear/scenario changes, recorded-time scrubbing and no future pairing at the initial epoch; empty error console and no horizontal overflow at 320/375 px. Desktop and mobile screenshots inspected.
Nominal synthetic run: 5.78 m follow travel, 3.22 m minimum true separation, 0.0389 m maximum visual-point error and 0.0304 m maximum fused error. These are fixture measurements, not hardware accuracy claims.
Exact-head CI passed, including Linux standalone commands and both neutral-only Pico 2 cross-builds.

Remaining physical and broader acceptance gates

Independently measured Pi lens/mount/ROI calibration and a qualified registered external depth adapter. The current Pi AI Camera worker remains image-only.
Surveyed heading/body-to-NED reference and measured camera/depth/pose synchronization. The current estimator observes relative yaw; the fixture explicitly assumes initial north alignment and budgets drift.
Separate calibration-fit/validation datasets, held-out physical trajectories and uncertainty coverage across range, lighting, motion, rolling shutter and temperature.
On-Pi restart/loss/reselection, hardware/HIL authority and vehicle campaigns. Imported declared-bench data never reports physical calibration verified.
Broader nearby-person identity ambiguity remains unresolved by geometric association. Appearance tracking requires a separately measured privacy/accuracy case.

Blocked by

#4 Estimation: publish owned attitude and local-position estimates
#11 Navigation: avoid an obstacle and land at a selected site
#8 Hardware adapters: command and verify the first camera or payload

Scope and evidence

Owned physics simulation and declared bench-data evaluation only. This is not external firmware SITL, native C++ flight execution, HIL or physical flight. Physical outputs remain inhibited. The owned implementation remains independent of ArduPilot/PX4. Keep this issue open for measured physical acceptance and the remaining integrations.