Generated reference · ComfyUI workflow · motion

SAM 3 Video Track

Track objects through a clip with SAM 3.1 on ComfyUI's core nodes and save the result as a grayscale mask video plus a preview overlay. The mask producer for the inpaint, replacement and matting graphs.

motion16nodesGPU H100out unspecified~15sv1.0.0
workflow.json
On this page
16Graph nodes
14Node classes
7Inputs
1Models

Tags: motionsam3sam3.1segmentationtrackingrunpod-serverlessvolume-backed

Inputs (7)#

The typed parameter surface callers bind when they request this workflow. Enum options and numeric bounds are the values the workflow document declares.

source_videovideorequired
The clip to track through.
text_promptstringrequireddefault person
What to track, described in words; SAM 3.1's text encoder reads at most 32 tokens per prompt. Comma-separated for several categories ("person, dog, car"): each is encoded separately and detected on its own. Text is the only prompt this graph takes; SAM3_VideoTrack's other seed is a first-frame mask, which would need its own graph.
thresholdfloatdefault 0.5min 0.0max 1.0
Detection score floor for the text prompt (SAM3_VideoTrack detection_threshold; 0.5 is the node's default). Lower catches more objects and admits more false positives.
max_objectsintegerdefault 4min 0max 64
Most objects tracked at once (SAM3_VideoTrack max_objects; 0 means the node's internal cap of 64). A crowd prompted as "person" is cut at this many.
detect_intervalintegerdefault 1min 1max 240
Run text detection every N frames to pick up objects entering the shot; 1 is every frame. Higher values save compute and pick up newcomers later.
obj_idintegerdefault -1min -1max 63
Which tracked object the mask video contains, by track index (0 is the first object found). -1 combines every object into one mask, which is what the downstream inpaint and matte graphs want.
output_fpsfloatdefault 16.0min 1.0max 60.0
Frame rate written into both outputs. It should match the source; the mask video is a frame-for-frame companion to it, so a mismatch silently desynchronises every downstream graph.

ComfyUI node graph (16)#

The executable ComfyUI prompt graph: 16 nodes across 14 distinct node classes, wired by 21 data dependencies. Nodes tinted green come from a custom node pack this workflow declares; the rest are ComfyUI core / baked-community classes.

flowchart TD n0["1: LoadVideo"] n1["2: GetVideoComponents"] n2["3: CheckpointLoaderSimple"] n3["4: CLIPTextEncode"] n4["5: SAM3_VideoTrack"] n5["6: SAM3_TrackToMask"] n6["7: MaskToImage"] n7["8: CreateVideo"] n8["9: SaveVideo"] n9["10: GetImageSize"] n10["11: SolidMask"] n11["12: MaskComposite"] n12["13: EmptyImage"] n13["14: ImageCompositeMasked"] n14["15: CreateVideo"] n15["16: SaveVideo"] n0 -->|video| n1 n2 -->|clip| n3 n1 -->|images| n4 n2 -->|model| n4 n3 -->|conditioning| n4 n4 -->|track_data| n5 n5 -->|mask| n6 n6 -->|images| n7 n7 -->|video| n8 n1 -->|image| n9 n9 -->|width| n10 n9 -->|height| n10 n5 -->|destination| n11 n10 -->|source| n11 n9 -->|width| n12 n9 -->|height| n12 n1 -->|destination| n13 n12 -->|source| n13 n11 -->|mask| n13 n13 -->|images| n14 n14 -->|video| n15
Data-flow DAG — scroll to zoom, drag to pan.

Nodes (16)#

1LoadVideocore
file = {{source_video}} tmpl
outputs: VIDEO
2GetVideoComponentscore
video = ◂ node 1 · out[0]
outputs: IMAGEAUDIOFLOATCOMBOCOMBO
3CheckpointLoaderSimplecore
ckpt_name = sam3.1_multiplex_fp16.safetensors
outputs: MODELCLIPVAE
4CLIPTextEncodecore
clip = ◂ node 3 · out[1]text = {{text_prompt}} tmpl
outputs: CONDITIONING
5SAM3_VideoTrackcore
images = ◂ node 2 · out[0]model = ◂ node 3 · out[0]conditioning = ◂ node 4 · out[0]detection_threshold = {{threshold}} tmplmax_objects = {{max_objects}} tmpldetect_interval = {{detect_interval}} tmpl
outputs: SAM3_TRACK_DATA
6SAM3_TrackToMaskcore
track_data = ◂ node 5 · out[0]object_indices = {{'' if obj_id < 0 else str(int(obj_id))}} tmpl
outputs: MASK
7MaskToImagecore
mask = ◂ node 6 · out[0]
outputs: IMAGE
8CreateVideocore
images = ◂ node 7 · out[0]fps = {{output_fps}} tmpl
outputs: VIDEO
9SaveVideocore
video = ◂ node 8 · out[0]filename_prefix = sam3-maskformat = mp4
10GetImageSizecore
image = ◂ node 2 · out[0]
outputs: INTINTINT
11SolidMaskcore
value = 0.5width = ◂ node 10 · out[0]height = ◂ node 10 · out[1]
outputs: MASK
12MaskCompositecore
destination = ◂ node 6 · out[0]source = ◂ node 11 · out[0]x = 0y = 0operation = multiply
outputs: MASK
13EmptyImagecore
width = ◂ node 10 · out[0]height = ◂ node 10 · out[1]batch_size = 1color = 65280
outputs: IMAGE
14ImageCompositeMaskedcore
destination = ◂ node 2 · out[0]source = ◂ node 13 · out[0]x = 0y = 0resize_source = falsemask = ◂ node 12 · out[0]
outputs: IMAGE
15CreateVideocore
images = ◂ node 14 · out[0]fps = {{output_fps}} tmpl
outputs: VIDEO
16SaveVideocore
video = ◂ node 15 · out[0]filename_prefix = sam3-overlayformat = mp4

Parameter banks (1)#

The prompt / configuration lookup tables this workflow keys into from its inputs — the vocabulary that turns a style / palette / preset selection into graph parameters.

requires_families (1)#

sam31

Models & dependencies#

Models required (1)#

sam3.1_multiplex_fp16.safetensors

Output contract#

What a successful run of this workflow returns.

primary
{"type": "video", "format": "mp4", "codec": "h264", "fps_source": "declared", "audio": false, "alpha": false, "description": "Grayscale mask video, one frame per source frame."}
secondary
{"type": "video", "format": "mp4", "codec": "h264", "fps_source": "declared", "audio": false, "alpha": false, "description": "Preview overlay: the tracked region tinted green at 50 % over the source frames."}

Taxonomy & routing#

How the control plane classifies this workflow — from the committed workflow-taxonomy-registry.json. It drives the consistency / control surface the agentic director can exercise over the workflow.

assetFamily
region-edited-video
outputPackageProfile
video-master-profile
controlModalities
model-lockprompt-template-locktemporal-lockcontrolnet-segmentation
consistencyDimensions
identitymotion
notes
SAM 3.1 text-prompted video tracking on the core SAM3_VideoTrack node. It edits no pixels of the source -- it produces the MASK the region-editing graphs then use -- so it is filed under region-edited-video as the producer half of that family. identity is a real dimension here: the tracker's job is to keep the same object selected across frames.