Eve library

phase-153.md

Tracker

Revision 1

On this page

Historical task source. The Eve task board holds current task state.

Phase 153: Sovereign VFX Match-Move & 3D Camera Tracking Suite — Science-D-Visions 3DEqualizer / Foundry PFTrack / Andersson SynthEyes / Boujou / Blender Motion Tracking / Blackmagic Fusion Tracker Replacement#

Closes the match-move sovereignty gap. Phase 135 (VFX Compositor & Post) flags "Nuke-class... does not cover 3DEqualizer, PFTrack, SynthEyes" and Phase 135.4 mentions matchmove only as a category. Phase 138 covers optical mocap ingest but not 2D→3D camera solving. This phase delivers a full sovereign match-move workstation for feature-film, episodic, VFX, virtual-production, and AR placement pipelines.

Sovereign replacement for Science-D-Visions 3DEqualizer 4/R8/R9, Foundry PFTrack, Andersson Technologies SynthEyes, 2d3 Boujou, Blender Motion Tracking (camera / object), Blackmagic Fusion Tracker, After Effects Camera Tracker, Mocha Pro Insert, Cinema 4D Motion Tracker.


153.1 @neith/matchmove-core — Project & Pipeline#

  • 153.1.1 Project entity with shot / sequence / reel structure
  • 153.1.2 Per-shot metadata (camera, lens, aperture, FPS, gate)
  • 153.1.3 Slate / burn-in data import (ALE / CDL / XMP)
  • 153.1.4 Color management (OCIO 2.x, ACES working space)
  • 153.1.5 Multi-take shot versioning
  • 153.1.6 Cross-shot survey-data reuse
  • 153.1.7 Imperial / metric unit switching
  • 153.1.8 Geospatial georeferencing option
  • 153.1.9 USD / Alembic / FBX scene roundtrip
  • 153.1.10 Job queue with farm submission (Phase 138 farm)

153.2 @neith/matchmove-ingest — Media & Camera Metadata#

  • 153.2.1 Video ingest (ProRes / DNx / Red / ARRI / BRAW / MP4 / EXR seq)
  • 153.2.2 Proxy generation with timestamp-exact sync
  • 153.2.3 Pulldown / field-handling (3:2, 2:3:2:3)
  • 153.2.4 Variable frame-rate stretch-to-constant
  • 153.2.5 Shutter angle / exposure metadata capture
  • 153.2.6 Timecode + reel name preservation
  • 153.2.7 Camera package database (sensor size, crop factor)
  • 153.2.8 Lens package database (prime, zoom, anamorphic squeeze)
  • 153.2.9 Cooke /i, Zeiss XD, ARRI LDS metadata parse
  • 153.2.10 Gyroscope / accelerometer IMU ingest (mobile / drone)
  • 153.2.11 Depth sensor (LiDAR / ToF) ingest
  • 153.2.12 Set LiDAR scan (E57 / PLY / LAS) ingest
  • 153.2.13 HDRI environment probe import (32-bit linear)
  • 153.2.14 Survey point cloud alignment
  • 153.2.15 Witness-camera / reference-camera multi-cam rig

153.3 @neith/matchmove-features — 2D Feature Tracking#

153.3.1 Trackers#

  • 153.3.1.1 KLT point tracker with pyramid
  • 153.3.1.2 SIFT / AKAZE / ORB descriptor tracker
  • 153.3.1.3 LightGlue / SuperPoint learned matcher
  • 153.3.1.4 Planar tracker (homography)
  • 153.3.1.5 Spline tracker for curved features
  • 153.3.1.6 Patch tracker with photometric model
  • 153.3.1.7 Dense optical-flow (RAFT-class)
  • 153.3.1.8 Bi-directional tracking with loop-refine
  • 153.3.1.9 Track-through-occlusion (re-identification)
  • 153.3.1.10 User tracker cook-and-freeze

153.3.2 Authoring#

  • 153.3.2.1 Manual track placement on viewer
  • 153.3.2.2 Automatic feature harvesting (corner density targets)
  • 153.3.2.3 Grid / mask track-spawn constraints
  • 153.3.2.4 Track ripple edit (all → selected → one)
  • 153.3.2.5 Hold frame, keyframe, interpolate
  • 153.3.2.6 Confidence + residual score per track
  • 153.3.2.7 Track colouring by quality
  • 153.3.2.8 Garbage-matte / inclusion-mask region
  • 153.3.2.9 Markers-on-set (physical tape / chroma-dot) recognition
  • 153.3.2.10 Autocorrect drift with sub-pixel refine

153.3.3 Constraints#

  • 153.3.3.1 Survey-constraint (real-world XYZ)
  • 153.3.3.2 Distance constraint between two tracks
  • 153.3.3.3 Co-planar constraint
  • 153.3.3.4 Line / vanishing-point constraint
  • 153.3.3.5 Horizon constraint
  • 153.3.3.6 Rigid-body group constraint
  • 153.3.3.7 Moving-object constraint (different solve group)
  • 153.3.3.8 Static-only filter for background solve
  • 153.3.3.9 Weighted track importance
  • 153.3.3.10 Per-frame constraint activation

153.4 @neith/matchmove-lens — Lens Distortion#

  • 153.4.1 Radial 3DE-class distortion model (k1, k2, k3 + anamorphic)
  • 153.4.2 Brown-Conrady radial + tangential
  • 153.4.3 OpenCV 5/8/12-parameter models
  • 153.4.4 Fisheye (equidistant / stereographic / orthographic)
  • 153.4.5 Anamorphic squeeze + cooke-Pin
  • 153.4.6 Per-frame distortion (zoom-breathing)
  • 153.4.7 ST-map / UV map export
  • 153.4.8 Lens-grid calibration tool (chessboard / checkerboard / dot)
  • 153.4.9 Distortion preview + grid overlay
  • 153.4.10 LDPK plug-in compatibility (3DE plug-ins)
  • 153.4.11 Undistort / redistort to / from filmback
  • 153.4.12 Focal-length vs distortion polynomial fit
  • 153.4.13 Principal point offset solve
  • 153.4.14 Entrance-pupil offset modelling
  • 153.4.15 Round-trip distortion diff verification

153.5 @neith/matchmove-solver — Camera Solver#

  • 153.5.1 Structure-from-Motion (SfM) pipeline
  • 153.5.2 Incremental + global SfM modes
  • 153.5.3 Pair / triplet initialisation with F + E matrix RANSAC
  • 153.5.4 5-point / 7-point / 8-point + Nister solver
  • 153.5.5 PnP pose estimation for known-3D points
  • 153.5.6 Bundle adjustment (Ceres / sovereign) with sparse Jacobian
  • 153.5.7 Robust loss (Huber / Cauchy / truncated)
  • 153.5.8 Focal-length auto-solve vs locked-focal
  • 153.5.9 Known-focal / partial-focal hybrid
  • 153.5.10 Variable-focal (zoom) solve
  • 153.5.11 Lens distortion joint solve
  • 153.5.12 Rolling-shutter correction during solve
  • 153.5.13 Gravity vector constraint from IMU
  • 153.5.14 Scale from survey / stereoscopic baseline
  • 153.5.15 Monocular scale from known-object
  • 153.5.16 Nodal-point tripod (panoramic) solver
  • 153.5.17 Stereo-pair rig solve
  • 153.5.18 Multi-camera rig solve
  • 153.5.19 Tracking reliability pruning + re-solve
  • 153.5.20 Solve-error heatmap per frame

153.6 @neith/matchmove-object — Object & Body Tracking#

  • 153.6.1 Rigid-body object solver (car / prop / sign)
  • 153.6.2 Articulated-body solver (skeleton-aware)
  • 153.6.3 Non-rigid / morph (face performance capture 2D → 3D)
  • 153.6.4 Mesh-deform tracker for cloth / skin
  • 153.6.5 Known-geometry (CAD / scan) align + solve
  • 153.6.6 ML object segmentation (SAM-class) for mask
  • 153.6.7 Pose estimation (OpenPose-class) for actor
  • 153.6.8 Hand / facial landmark tracking
  • 153.6.9 Multi-object scene solve
  • 153.6.10 Occlusion handling with z-depth hint

153.7 @neith/matchmove-scan — Set Reconstruction & Dense 3D#

  • 153.7.1 Multi-view stereo (MVS) dense point cloud
  • 153.7.2 Depth-map fusion (volumetric TSDF)
  • 153.7.3 Mesh reconstruction (Poisson / Delaunay / TSDF-mesh)
  • 153.7.4 Texture projection onto mesh
  • 153.7.5 Gaussian splatting ingest (Phase 71)
  • 153.7.6 Neural radiance field (NeRF) solve option
  • 153.7.7 LiDAR scan merge into photogrammetric solve
  • 153.7.8 Scale-from-scan enforcement
  • 153.7.9 Ground plane auto-detect
  • 153.7.10 Occlusion mesh (matte-object) export
  • 153.7.11 Point-cloud decimation / filter
  • 153.7.12 Surface normal estimation
  • 153.7.13 Albedo / lighting inverse-render
  • 153.7.14 Reflection / shadow probe extraction
  • 153.7.15 Hydra-compatible USD scene export

153.8 @neith/matchmove-stabilize — Stabilisation & Retime#

  • 153.8.1 2D stabilise (translation / rotation / scale / perspective)
  • 153.8.2 3D stabilise (lock to 3D point / plane)
  • 153.8.3 Smooth-cam with path tension slider
  • 153.8.4 Rolling-shutter correction filter
  • 153.8.5 Motion-blur removal
  • 153.8.6 Motion-blur synthesize (for match)
  • 153.8.7 Retime with optical-flow warp
  • 153.8.8 Gyro-flow stabilisation from IMU
  • 153.8.9 Horizon auto-level
  • 153.8.10 Counter-animated lock to moving object

153.9 @neith/matchmove-export — Exporters & Roundtrip#

  • 153.9.1 Nuke-class compositor (Phase 135) native
  • 153.9.2 Maya / Blender / sovereign DCC (Phase 138) camera + locators
  • 153.9.3 Neith Engine (Phase 42) virtual camera
  • 153.9.4 USD camera primitive + lens metadata
  • 153.9.5 Alembic camera + tracker points
  • 153.9.6 FBX camera + animation
  • 153.9.7 3DE lens distortion STMap for re-render
  • 153.9.8 Ground-plane geo + matte object
  • 153.9.9 Shotgun / ftrack / sovereign ShotOps (Phase 135) publish
  • 153.9.10 Virtual-production (Phase 163) LED-volume camera feed
  • 153.9.11 Real-time nDisplay camera over OSC
  • 153.9.12 Lens-file export (3DE / Nuke / PFTrack LD / Blender)
  • 153.9.13 Survey data export (CSV / E57 / USD)
  • 153.9.14 Error-report PDF attachment to shot pub
  • 153.9.15 Preview render with render-preview pass

153.10 @neith/matchmove-viz — Viewer & Interaction#

  • 153.10.1 Dual-pane (2D track viewer + 3D scene viewer)
  • 153.10.2 Per-frame onion-skin track trail
  • 153.10.3 Residual overlay per track
  • 153.10.4 Reprojection error graph
  • 153.10.5 3D viewport with camera path
  • 153.10.6 VR headset preview of solved scene
  • 153.10.7 Interactive solve-and-preview
  • 153.10.8 Keyframe / graph editor for animated quantities
  • 153.10.9 Layer / group / colour per track
  • 153.10.10 Side-by-side before / after lens undistort

153.11 @neith/matchmove-ai — ML-Assisted Matchmove#

  • 153.11.1 Auto-mask ("paint the moving car") (Phase 94)
  • 153.11.2 Deep-matcher feature correspondence
  • 153.11.3 Monocular depth prior
  • 153.11.4 Learned lens-distortion inference from single frame
  • 153.11.5 Auto-orient (up-vector) from semantic cues
  • 153.11.6 Scene understanding (ground / wall / sky) label
  • 153.11.7 LLM assistant for solve-failure diagnosis (Phase 47)
  • 153.11.8 Shot-graph classifier (handheld / steadicam / drone)
  • 153.11.9 Suggested-track-density heatmap
  • 153.11.10 Auto-QA before publish

153.12 @neith/matchmove-script — Python / Rust SDK & Scripting#

  • 153.12.1 Python 3 bindings mirroring 3DE API style
  • 153.12.2 Rust core API
  • 153.12.3 Headless CLI batch solve
  • 153.12.4 Farm-distribute solve jobs
  • 153.12.5 Jupyter notebook tutorials
  • 153.12.6 PyPI mock-compat package for 3DE-plugins
  • 153.12.7 Custom solver extension API
  • 153.12.8 Event hooks (on-solve / on-export)
  • 153.12.9 Integration test harness
  • 153.12.10 Preset library shareable across studios

153.13 Testing & Validation#

  • 153.13.1 Public benchmark datasets (ETH3D, Tanks & Temples, KITTI)
  • 153.13.2 Residual ≤ 0.3 px on clean 4K plates
  • 153.13.3 Anamorphic T1.3 calibration accuracy test
  • 153.13.4 Rolling-shutter camera benchmark
  • 153.13.5 Nodal-pan stitch regression
  • 153.13.6 Object-solve accuracy vs mocap ground truth
  • 153.13.7 3DE / PFTrack / SynthEyes export compat conformance
  • 153.13.8 1000-shot feature-film VFX pipeline dogfood
  • 153.13.9 Interactive solve on 4K plate < 3 min
  • 153.13.10 Licence cost benchmark (zero vs proprietary)

Implementation and verification evidence: phase-153-evidence.md.