NVIDIA Introduces NV-Reason-CT: Open 3D CT VLM with Radiologist Chain-of-Thought Reasoning
NVIDIA 推出首款開源 3D CT 醫療視覺語言模型 NV-Reason-CT，導入放射科醫師思考鏈推論
NV-Reason-CT is an open 3D CT vision-language model designed to overcome the limitations of 2D slice-based analysis. By combining a native 3D Vision Transformer (ViT) and 3D MRoPE with a Qwen3.5-4B LLM, it processes entire 3D volumes holistically. Trained via a two-stage pipeline consisting of supervised fine-tuning and GRPO reinforcement learning, the model emulates radiologist reasoning, generates structured reports covering chest and abdominal findings, and supports interactive multi-turn clinical dialogue.
Key points
Native 3D Spatial Perception
Rejects 2D slice-by-slice processing, utilizing a native 3D ViT encoder and 3D MRoPE to maintain anatomical continuity across spatial dimensions.
Radiologist-Emulating Reasoning
Generates step-by-step internal reasoning, systematically reviewing anatomical regions, weighing differentials, and expressing calibrated uncertainty instead of raw labels.
Two-Stage Training Pipeline
Trained on ~550,000 QA examples via supervised fine-tuning, followed by Group Relative Policy Optimization (GRPO) reinforcement learning to enhance diagnostic accuracy.
Interactive Multi-Turn Dialogue
Supports multi-turn dialogue, enabling clinicians and researchers to ask follow-up questions, request clarifications, or probe model reasoning interactively.
How it works
| Type | Macro-F1 | Macro-AUROC | |
|---|---|---|---|
| NV-Reason-CT | Native 3D generative VLM | 0.614 | 0.871 |
| VoxelFM | 3D image-only pretraining | 0.581 | 0.870 |
| Pillar-0 | 3D contrastive | 0.544 | 0.861 |
| ClinFusion-8B | Fused 2D/3D generative MLLM | 0.442 | n/r |
| MedGemma 1.5 | Up to 85 axial slices | 0.303 | n/r |
Why it matters
Traditional medical AI operates as a black box, outputting labels without clinical justification. NV-Reason-CT changes this by outputting auditable, step-by-step clinical reasoning. By providing a transparent, open-source foundation, it reduces cognitive load for routine reporting, saves time for radiologists, and offers a verifiable reasoning trace that builds clinical trust and accelerates medical research.
Who it affects
- AI Developer
- AI Researcher
- Enterprise Leader
How to use it
- 1Automated structured CT report generation covering 60+ chest and abdominal abnormalities.
- 2Interactive multi-turn clinical QA to assist researchers in analyzing specific slices or findings.
- 3Fine-tuning foundation model for medical institutions or workflow vendors using proprietary CT datasets.
Limitations & caveats
- Designed as an open research foundation; not an autonomous diagnostic system or a cleared clinical product.
- Input CT volumes must be resampled to 192³ voxels (2 mm isotropic resolution), which may limit detection of extremely fine structures.
Related
PoEM: Predicting RL Outcomes Without Re-training Existing Policies
不需重新訓練！PoEM 演算法直接預測新獎勵函數的強化學習結果
The PoEM framework enables researchers to accurately predict and synthesize new RL policies without any actual RL training, by leveraging linear combinations of existing post-trained models in log-space.
Who Said What to Whom? SpeakerMem-R1 Introduces Dual-Track Memory for Multi-Party Dialogues
「誰對誰說了什麼？」SpeakerMem-R1 打造多方對話專屬的雙軌記憶機制
SpeakerMem-R1 tackles multi-party dialogue memory challenges using a speaker-labeled dual-track system and GRPO training to improve message attribution and social relation tracking.
FleXray: Universal Generalist Model for Full-Body X-Ray Segmentation
FleXray：首款通用型全身體檢 X 光影像分割模型
FleXray is a generalist model that segments 60 anatomical structures across clinical X-rays, trained using a physics-based generative engine that simulates fully-annotated 2D X-rays from 3D CT datasets.