SAM 2 added video tracking to Meta's Segment Anything Model research in 2024. SAM 3.1, detailed in a Meta AI blog post this week, makes that tracking substantially faster - and able to follow many objects at once without the latency penalty that previously made multi-object tracking impractical.
SAM is a computer vision model that draws precise pixel-level outlines around objects in images and video. Point it at a frame, tell it which object to follow, and it keeps tracking that object as the scene changes. The two new capabilities in 3.1 are multiplexing and global reasoning.
Multiplexing means the model processes multiple tracking targets in parallel rather than sequentially. In earlier versions, tracking five objects was roughly five times slower than tracking one - each additional target added processing time. SAM 3.1 handles them concurrently, keeping latency close to constant regardless of how many objects you're following. For video editors working with complex scenes, production pipelines doing automated object isolation, or any application tracking multiple subjects at once, this removes a meaningful bottleneck.
Global reasoning addresses how these models handle occlusions - moments when a tracked object gets partially or fully blocked. Patch-based vision models (those that analyze a video in sections rather than looking at the whole frame) lose track of objects during occlusions because they lack context about what's happening outside their current section. SAM 3.1 maintains awareness of the full frame, keeping tracking stable when a person steps behind a pillar or a product slides off-screen and back.
What Changes for Developers and Creative Tools
Meta is shipping SAM 3.1 with reduced memory requirements and more efficient inference - the actual compute work of running the model. This brings it within reach of consumer-grade GPUs, rather than requiring the research hardware that made earlier versions impractical outside of labs.
SAM has already been integrated into creative tools for background removal, motion tracking, and rotoscoping (the frame-by-frame process of tracing moving objects, previously done by hand). The model weights and code are released through Meta's open research channels. Developers building on SAM 2 should be able to migrate to 3.1 without significant pipeline changes and will get the multiplexing and global reasoning improvements automatically.