Subscribe us: https://groups.google.com/u/2/g/bodymaps

[!NOTE] We have publicly released our dataset!
Please check the download instructions below.

[!NOTE] We have publicly released AtlasNet (download here): organ and tumor segmentation models trained on AbdomenAtlas 3.0!

AbdomenAtlas 3.0 is the first public dataset with high quality abdominal CTs and paired radiology reports. The database includes more than 9,000 CT scans with radiology reports and per-voxel annotations of liver, kidney and pancreatic tumors.

AbdomenAtlas 3.0 is the first public, high-quality abdominal CT dataset with detailed, expert-reviewed radiology reports. The database includes more than 9,000 CT scans with radiology reports and per-voxel annotations of liver, kidney and pancreatic tumors.

Overall, AbdomenAtlas 3.0 presents per-voxel segmentation masks for:

• tumors in the kidney, pancreas, and liver
• blood vessels, including the aorta, IVC, SMA, CHA, CA, SA, portal vein, and splenic vein
• liver sub-segments
• pancreas sub-segments
• 18 organs, including the gallbladder, left kidney, right kidney, spleen, stomach, left adrenal gland, right adrenal gland, bladder, colon, duodenum, esophagus, left femur, right femur, intestine, left lung, right lung, prostate, and rectum

Moreover, we present RadGPT, a segmentation-based report generation model which significantly surpasses the current state of the art in report generation for abdominal CTs.

Our “superhuman” reports are more accurate, detailed, standardized, and generated faster than traditional human-made reports.

Paper

RadGPT: Constructing 3D Image-Text Tumor Datasets Pedro R. A. S. Bassi, Mehmet Yavuz, Kang Wang, Sezgin Er, Ibrahim E. Hamamci, Wenxuan Li, Xiaoxi Chen, Sergio Decherchi, Andrea Cavalli, Yang Yang, Alan Yuille, Zongwei Zhou* Johns Hopkins University ICCV, 2025

Download the AbdomenAtlas 3.0 Dataset

You can directly download AbdomenAtlas 3.0 from HuggingFace using your preferred HuggingFace dowload method (e.g., huggingface-cli), or you can just use the simple download code below:

git clone https://github.com/MrGiovanni/RadGPT.git
cd RadGPT
bash download_atlas_3.sh

Train/Test Splits

You can find csv files with the training and testing IDs inside the downloaded dataset (folder TrainTestIDS).

• IID train/test split: the dataset was randomly partitioned into traing (90%) and testing (10%). These train/test partitions were used in our paper (IID results).
• OOD train/test split: the training and testing patitions come from different hospitals. This is useful for external evaluation, but the training set here is smaller (50% of the whole dataset). These train/test partitions were not used in our paper.

Installation

wget https://repo.anaconda.com/archive/Anaconda3-2024.06-1-Linux-x86_64.sh
bash Anaconda3-2024.06-1-Linux-x86_64.sh -b -p ./anaconda3
./anaconda3/bin/conda init
source ~/.bashrc

git clone https://github.com/MrGiovanni/RadGPT.git
cd RadGPT
conda create -n vllm python=3.12 -y
conda activate vllm
conda install -y ipykernel
conda install -y pip
pip install vllm==0.6.1.post2
pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830
pip install -r requirements.txt
mkdir HFCache

AbdomenAtlasNet: Trained Organ and Tumor Segmentation Models

You can download AbdomenAtlasNet, an organ and tumor segmentation model trained on AbdomenAtlas 3.0, here: HuggingFace

Generate Structured, Narrative and Enhanced Human Reports

Use RadGPT to generate reports from organ and tumor per-voxel segmentations and to enhance human-made reports.

generate_reports/README.md

Evaluate the Diagnoses in the Reports with LLM

LLM (labeler) extracts binary labels indicating if reports indicate the presence or absence of liver, kidney and pancreatic cancers (or any cancer). These labels can be used to compare AI-made reports to human-made reports (ground-truth) and evaluate cancer detection specificity and sensitivity.

evaluate_reports/README.md

Citation

@inproceedings{bassi2025radgpt,
  title={Radgpt: Constructing 3d image-text tumor datasets},
  author={Bassi, Pedro RAS and Yavuz, Mehmet Can and Hamamci, Ibrahim Ethem and Er, Sezgin and Chen, Xiaoxi and Li, Wenxuan and Menze, Bjoern and Decherchi, Sergio and Cavalli, Andrea and Wang, Kang and others},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={23720--23730},
  year={2025},
  title={RadGPT: Constructing 3D Image-Text Tumor Datasets},
  author={Bassi, Pedro RAS and Yavuz, Mehmet Can and Wang, Kang and Chen, Xiaoxi and Li, Wenxuan and Decherchi, Sergio and Cavalli, Andrea and Yang, Yang and Yuille, Alan and Zhou, Zongwei},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={23720--23730},
  year={2025},
  url={https://github.com/MrGiovanni/RadGPT}
}

Acknowledgement

This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research, the Patrick J. McGovern Foundation Award, and the National Institutes of Health (NIH) under Award Number R01EB037669. We would like to thank the Johns Hopkins Research IT team in IT@JH for their support and infrastructure resources where some of these analyses were conducted; especially DISCOVERY HPC. Paper content is covered by patents pending.