GaussCtrl edits a 3DGS scene by rendering depth from each view, conditioning a depth ControlNet on it, and fitting the edited views back into the scene. This directory is the measured CPU attempt at its first two steps. There is no GPU on the dev server.
make_probe_scene.pywrites a small 3DGS scene in the reference trainer's PLY format: 3,552 Gaussians forming a sphere on a floor.render-depth.tsreads that PLY with the library'sparseGaussianPlyand renders depth withrenderGaussianDepth(libs/isis/3dgs-diffusion-editing/src/gaussian-depth-renderer.ts, EWA splatting and front-to-back compositing). It writes a PFM.controlnet_depth_cpu.pyturns that depth into the inverse-depth image the depth ControlNet was trained on, and runs Stable Diffusion 1.5 withlllyasviel/control_v11f1p_sd15_depthon CPU. It writes the image, the conditioning image and a JSON record of the run.
python3 -m venv --system-site-packages .venv-isis-controlnet # system torch 2.11.0
.venv-isis-controlnet/bin/pip install "diffusers==0.35.2" "transformers==4.57.1" "accelerate==1.10.1"
cd tools/isis-gaussctrl-cpu
../../.venv-isis-controlnet/bin/python make_probe_scene.py /tmp/gc/scene.ply
npx tsx --tsconfig ../../tsconfig.json render-depth.ts /tmp/gc/scene.ply /tmp/gc/depth.pfm
../../.venv-isis-controlnet/bin/python controlnet_depth_cpu.py /tmp/gc/depth.pfm /tmp/gc/out --steps 20 --seed 7
Measured on the dev server (4 cores, AMD EPYC-Milan 2.0 GHz), 2026-09-21#
Depth. 3,552 Gaussians, of which 3,136 were splatted, rendered at 512×512 in
159 ms. Depth ran from 1.56 m to 8.71 m over 146,351 covered pixels, and every
uncovered pixel is NaN. evidence/control-depth.png is the conditioning image:
near is bright, far is dark, no surface is black.
Depth ControlNet. Stable Diffusion 1.5 with
lllyasviel/control_v11f1p_sd15_depth, fp32, 20 steps at 512×512, 4 threads,
torch 2.11.0, diffusers 0.35.2 and transformers 4.57.1. diffusers 0.35.2's
safety checker does not load under transformers 5.x.
| seed | generate | per step | result |
|---|---|---|---|
| 7 | 311.2 s | 15.56 s | the safety checker flagged it and returned a black image |
| 11 | 314.0 s | 15.70 s | evidence/controlnet-seed11.png: the sphere and floor follow the rendered depth |
The records are evidence/run-seed7.json and evidence/run-seed11.json. Model
load from cache took 1 s; the first download was about 5 GB.
So the depth-conditioned ControlNet GaussCtrl needs runs on this CPU at about 5 minutes per view, before any multi-view attention or the refit of the splats. An edit over 40 views would spend over three hours in ControlNet alone. It works; it is not interactive.