TrackEverything: Breaking the Horizon Barrier in Dense 3D Point Tracking
TrackEverything：透過去重 3D 表徵實現破紀錄的長序列全員稠密追蹤
Existing 3D point trackers trade off density for duration. TrackEverything overcomes this constraint by modeling videos as persistent 3D scene tracks in world coordinates. By utilizing voxel-based de-duplication to merge redundant co-located tracks, decomposing tracking into destination prediction and dynamic-only trajectory refinement, and employing 3D WAFT for efficient feature sampling, it enables dense tracking over 1,000 frames within 40 GB GPU memory. It outperforms state-of-the-art open-source dense trackers by over 20% APD on TAPVid-3D.
Key points
Voxel-based De-duplication
Merges co-located tracks at sliding-window boundaries to prevent redundant memory accumulation from repeating surface observations.
Two-stage Decomposition
Predicts destinations and static-versus-dynamic states first, decoding dense trajectories exclusively for moving physical points.
Efficient 3D WAFT
Replaces memory-intensive 4D correlation volumes with extremely efficient feature sampling directly within the 3D scene cloud.
Long Horizon Scalability
The first 3D tracking model capable of tracking all visible points in videos exceeding 1,000 frames on a single 40 GB GPU.
How it works
Why it matters
This research breaks the fundamental tradeoff in point tracking. By showing that model complexity can scale with physical 3D scene geometry rather than video duration, it introduces a highly scalable framework. This is vital for downstream tasks requiring long-term, high-fidelity spatial understanding, such as dynamic 3D reconstruction, non-rigid robot manipulation, and autonomous navigation.
Who it affects
- AI Researcher
- AI Developer
- Product Manager
How to use it
- 1Long-horizon dynamic 3D scene reconstruction and visual editing
- 2Robotic manipulation of non-rigid objects requiring precise physical interaction tracking
- 3Advanced trajectory and motion analysis of dynamic obstacles in autonomous driving
Limitations & caveats
- Highly dependent on the accuracy of initial 3D scene representation and depth estimation
- Robustness may be challenged in scenes with extreme occlusions or drastic lighting variations
Related
Overcoming Incomplete Data in MSA: SemMSA Harnesses LLM Latent Semantics and Spectral Alignment
突破多模態情緒分析瓶頸：SemMSA 藉由 LLM 潛在語意與無錨點頻譜對齊解決資料缺失問題
The proposed SemMSA framework leverages frozen LLMs to extract latent semantics and applies anchor-free spectral alignment to address incomplete modality data in sentiment analysis.
PoEM: Predicting RL Outcomes Without Re-training Existing Policies
不需重新訓練！PoEM 演算法直接預測新獎勵函數的強化學習結果
The PoEM framework enables researchers to accurately predict and synthesize new RL policies without any actual RL training, by leveraging linear combinations of existing post-trained models in log-space.
NVIDIA Introduces NV-Reason-CT: Open 3D CT VLM with Radiologist Chain-of-Thought Reasoning
NVIDIA 推出首款開源 3D CT 醫療視覺語言模型 NV-Reason-CT，導入放射科醫師思考鏈推論
NVIDIA introduces NV-Reason-CT, an open 3D CT vision-language model that combines a native 3D ViT with a Qwen LLM to emulate radiologist chain-of-thought reasoning and generate structured reports.