Fix or replace part of an existing video in one pass: SAM 3.1 on ComfyUI's core nodes tracks the thing you name, the mask is grown to cover its edge, and VACE regenerates inside it from a prompt. The mask never leaves the graph, so there is no intermediate clip to write, upload and read back.
19Graph nodes
17Node classes
19Inputs
4Models
Tags: motionsam3sam3.1vaceinpaintcompositionrunpod-serverlessvolume-backed
Inputs (19)#
The typed parameter surface callers bind when they request this workflow. Enum options and numeric bounds are the values the workflow document declares.
source_videovideorequiredThe clip to edit.
track_promptstringrequireddefault personWhat SAM 3.1 should track and hand to VACE as the region to replace; its text encoder reads at most 32 tokens per prompt. Comma-separated for several categories.
promptstringrequireddefault the same scene with the tracked subject replaced, cinematic lightingWhat should appear inside the tracked region.
negative_promptstringdefault 色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走Wan's own published negative prompt.
reference_imageimageOptional still VACE conditions on — what the replacement should look like.
mask_growintegerdefault 12min -64max 64Pixels to grow the tracked mask by before VACE sees it. A tight track leaves the subject's own edge inside the KEPT region, and the replacement then has to blend against it; growing the mask hands that edge to the generator. Negative shrinks.
track_thresholdfloatdefault 0.5min 0.0max 1.0SAM 3.1 detection score floor for the track prompt (SAM3_VideoTrack detection_threshold; 0.5 is the node's default).
track_max_objectsintegerdefault 4min 0max 64Most objects tracked at once (SAM3_VideoTrack max_objects; 0 means the node's internal cap of 64). Every tracked object joins the region VACE replaces.
track_detect_intervalintegerdefault 1min 1max 240Run text detection every N frames to pick up objects entering the shot; 1 is every frame.
strengthfloatdefault 1.0min 0.0max 2.0How hard VACE binds to the control track.
resolutionenumdefault 480p_landscape_832x480Frame size. It should match the source; VACE does not letterbox either.
options:
480p_landscape_832x480480p_portrait_480x832720p_landscape_1280x720720p_portrait_720x1280lengthintegerdefault 81min 5max 241Frames to produce. Wan wants 4n+1.
fpsfloatdefault 16.0min 1.0max 30.0Frame rate written into the output.
stepsintegerdefault 20min 1max 60Sampling steps.
cfgfloatdefault 5.0min 1.0max 20.0Classifier-free guidance.
shiftfloatdefault 8.0min 0.0max 20.0ModelSamplingSD3 shift.
seedintegerdefault 0min 0Sampler seed.
samplerenumdefault uni_pcSampler.
options:
uni_pceulerdpmpp_2mschedulerenumdefault simpleSchedule.
options:
simplenormalbetakarrasComfyUI node graph (19)#
The executable ComfyUI prompt graph: 19 nodes across 17 distinct node classes, wired by 25 data dependencies. Nodes tinted green come from a custom node pack this workflow declares; the rest are ComfyUI core / baked-community classes.
flowchart TD
n0["1: LoadVideo"]
n1["2: GetVideoComponents"]
n2["3: CheckpointLoaderSimple"]
n3["4: CLIPTextEncode"]
n4["5: SAM3_VideoTrack"]
n5["6: SAM3_TrackToMask"]
n6["7: GrowMask"]
n7["8: UNETLoader"]
n8["9: CLIPLoader"]
n9["10: VAELoader"]
n10["11: ModelSamplingSD3"]
n11["12: CLIPTextEncode"]
n12["13: CLIPTextEncode"]
n13["14: WanVaceToVideo"]
n14["15: KSampler"]
n15["16: TrimVideoLatent"]
n16["17: VAEDecode"]
n17["18: CreateVideo"]
n18["19: SaveVideo"]
n0 -->|video| n1
n2 -->|clip| n3
n1 -->|images| n4
n2 -->|model| n4
n3 -->|conditioning| n4
n4 -->|track_data| n5
n5 -->|mask| n6
n7 -->|model| n10
n8 -->|clip| n11
n8 -->|clip| n12
n11 -->|positive| n13
n12 -->|negative| n13
n9 -->|vae| n13
n1 -->|control_video| n13
n6 -->|control_masks| n13
n10 -->|model| n14
n13 -->|positive| n14
n13 -->|negative| n14
n13 -->|latent_image| n14
n14 -->|samples| n15
n13 -->|trim_amount| n15
n15 -->|samples| n16
n9 -->|vae| n16
n16 -->|images| n17
n17 -->|video| n18
Nodes (19)#
1LoadVideocorefile = {{source_video}} tmploutputs:
VIDEO2GetVideoComponentscorevideo = ◂ node 1 · out[0]outputs:
IMAGEAUDIOFLOATCOMBOCOMBO3CheckpointLoaderSimplecoreckpt_name = sam3.1_multiplex_fp16.safetensorsoutputs:
MODELCLIPVAE4CLIPTextEncodecoreclip = ◂ node 3 · out[1]text = {{track_prompt}} tmploutputs:
CONDITIONING5SAM3_VideoTrackcoreimages = ◂ node 2 · out[0]model = ◂ node 3 · out[0]conditioning = ◂ node 4 · out[0]detection_threshold = {{track_threshold}} tmplmax_objects = {{track_max_objects}} tmpldetect_interval = {{track_detect_interval}} tmploutputs:
SAM3_TRACK_DATA6SAM3_TrackToMaskcoretrack_data = ◂ node 5 · out[0]object_indices = outputs:
MASK7GrowMaskcoremask = ◂ node 6 · out[0]expand = {{mask_grow}} tmpltapered_corners = trueoutputs:
MASK8UNETLoadercoreunet_name = wan2.1_vace_14B_fp16.safetensorsweight_dtype = defaultoutputs:
MODEL9CLIPLoadercoreclip_name = umt5_xxl_fp16.safetensorstype = wandevice = defaultoutputs:
CLIP10VAELoadercorevae_name = wan_2.1_vae.safetensorsoutputs:
VAE11ModelSamplingSD3coremodel = ◂ node 8 · out[0]shift = {{shift}} tmploutputs:
MODEL12CLIPTextEncodecoreclip = ◂ node 9 · out[0]text = {{prompt}} tmploutputs:
CONDITIONING13CLIPTextEncodecoreclip = ◂ node 9 · out[0]text = {{negative_prompt}} tmploutputs:
CONDITIONING14WanVaceToVideocorepositive = ◂ node 12 · out[0]negative = ◂ node 13 · out[0]vae = ◂ node 10 · out[0]width = {{resolution_map[resolution].width}} tmplheight = {{resolution_map[resolution].height}} tmpllength = {{length}} tmplbatch_size = 1strength = {{strength}} tmplcontrol_video = ◂ node 2 · out[0]control_masks = ◂ node 7 · out[0]outputs:
CONDITIONINGCONDITIONINGLATENTINT15KSamplercoremodel = ◂ node 11 · out[0]positive = ◂ node 14 · out[0]negative = ◂ node 14 · out[1]latent_image = ◂ node 14 · out[2]seed = {{seed}} tmplsteps = {{steps}} tmplcfg = {{cfg}} tmplsampler_name = {{sampler}} tmplscheduler = {{scheduler}} tmpldenoise = 1.0outputs:
LATENT16TrimVideoLatentcoresamples = ◂ node 15 · out[0]trim_amount = ◂ node 14 · out[3]outputs:
LATENT17VAEDecodecoresamples = ◂ node 16 · out[0]vae = ◂ node 10 · out[0]outputs:
IMAGE18CreateVideocoreimages = ◂ node 17 · out[0]fps = {{fps}} tmploutputs:
VIDEO19SaveVideocorevideo = ◂ node 18 · out[0]filename_prefix = sam3-vaceformat = mp4Parameter banks (3)#
The prompt / configuration lookup tables this workflow keys into from its inputs — the vocabulary that turns a style / palette / preset selection into graph parameters.
resolution_map (4)#
480p_landscape_832x480{"width": 832, "height": 480}480p_portrait_480x832{"width": 480, "height": 832}720p_landscape_1280x720{"width": 1280, "height": 720}720p_portrait_720x1280{"width": 720, "height": 1280}post_render (1)#
optional_image_input{"input": "reference_image", "target_class": "WanVaceToVideo", "target_input": "reference_image"}requires_families (3)#
sam31wan21-vacewan-sharedModels & dependencies#
Models required (4)#
sam3.1_multiplex_fp16.safetensorswan2.1_vace_14B_fp16.safetensorsumt5_xxl_fp16.safetensorswan_2.1_vae.safetensorsOutput contract#
What a successful run of this workflow returns.
primary{"type": "video", "format": "mp4", "codec": "h264", "fps_source": "declared", "audio": false, "alpha": false, "description": "The clip with the tracked region regenerated."}Taxonomy & routing#
How the control plane classifies this workflow — from the committed workflow-taxonomy-registry.json. It drives the consistency / control surface the agentic director can exercise over the workflow.
assetFamilyregion-edited-videooutputPackageProfilevideo-master-profilecontrolModalitiesmodel-lockprompt-template-locksampler-scheduler-lockseed-locktemporal-lockcontrolnet-segmentationcontrolnet-inpaintreference-ensembleconsistencyDimensionsidentitymotionlightingenvironmentnotesThe composition of sam3-video-track and wan21-vace-edit's inpaint mode in ONE graph: the MASK goes straight from SAM3_TrackToMask through GrowMask into WanVaceToVideo, so there is no intermediate mask clip to write, upload and read back.