Spatial Retrieval Augmented Autonomous Driving

🗺️ Task: Online HD Map Construction

📖 Introduction

This repository contains the official implementation of the Online Mapping task from our paper: "Spatial Retrieval Augmented Autonomous Driving".

We introduce a novel Spatial Retrieval Paradigm that retrieves offline geographic images (Satellite/Streetview) based on GPS coordinates to enhance autonomous driving tasks. For Online Mapping, we design a plug-and-play Spatial Retrieval Adapter and a Reliability Estimation Gate to robustly fuse this external knowledge into BEV representations.

This main branch provides the implementation based on MapTRv2.

Note: If you are looking for the implementation based on MapTR (v1), please switch to the legacy/maptr branch.

🚀 News

• [2025-12-09] Code and checkpoints for Online Mapping (MapTRv2 & MapTR) are released!

📊 Model Zoo & Main Results

MapTRv2 (ResNet50) - main branch

Method	Modality	Epoch	APped	APdiv	APbound	mAP	Config	Download
MapTRv2	C	24	59.8	62.4	62.4	61.5	-	-
MapTRv2 + Geo	C + Geo	24	74.8	75.0	71.2	73.7	config	model
MapTRv2	C	110	68.1	68.3	69.7	68.7	-	-
MapTRv2 + Geo	C + Geo	110	79.8	77.4	77.3	78.2	config	model

C: Camera, Geo: Geographic Images.

MapTR (ResNet50) - legacy/maptr branch

For comparison with the v1 baseline:

Method	Modality	Epoch	APped	APdiv	APbound	mAP	Config	Download
MapTR	C	24	46.3	51.5	53.1	50.3	-	-
MapTR + Geo	C + Geo	24	56.5	67.3	59.8	61.2	-	-
MapTR	C	110	55.9	60.9	61.1	59.3	-	-
MapTR + Geo	C + Geo	110	72.5	75.0	70.5	72.7	-	-

C: Camera, Geo: Geographic Images.

🔄 Versions & Branches

The repository is organized as follows:

• main (Default): Implementation based on MapTRv2, using LSS as BEV encoder.
• legacy/maptr: Implementation based on the original MapTR, using BEVFormer as BEV encoder.

To switch to the MapTR v1 version:

git checkout legacy/maptr

📦 Installation

Prerequisites

• CUDA 11.3
• Python 3.8
• GCC 9.4.0

Step 1: Create and Activate Environment

conda create -n newmaptr python=3.8
conda activate newmaptr

Step 2: Set Environment Variables

# to install successfully on RTX 4090
export TORCH_CUDA_ARCH_LIST="8.6"
export CUDA_HOME=/usr/local/cuda-11.3

Step 3: Install PyTorch and Dependencies

# Install PyTorch with CUDA 11.3
pip install torch==1.10.0+cu113 torchvision==0.11.0+cu113 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html

# Install MMDetection ecosystem
pip install mmcv-full==1.4.0 mmdet==2.14.0 mmsegmentation==0.14.1 timm==0.9.5

Step 4: Install MMDetection3D

cd mmdetection3d
conda install -c conda-forge gcc=9.4.0 gxx_linux-64=9.4.0
pip install -v -e .

Step 5: Install MapTR Plugins

# Install geometric kernel attention
cd projects/mmdet3d_plugin/maptr/modules/ops/geometric_kernel_attn
python setup.py build install

# Install project requirements
cd path/to/Online_Mapping
pip install -r requirement.txt

# Install additional dependencies
pip install numpy==1.22.2 yapf==0.40.1 setuptools==59.5.0

📂 Data Preparation

Step 1: Prepare Base Dataset (Following MapTR)

NuScenes Dataset

Download nuScenes V1.0 full dataset and CAN bus expansion data from HERE.

Download CAN bus expansion:

# Download 'can_bus.zip'
unzip can_bus.zip 
# Move can_bus to data directory

Prepare nuScenes data:

python tools/create_data.py nuscenes --root-path ./data/nuscenes --out-dir ./data/infos --extra-tag nuscenes --version v1.0 --canbus ./data

This will generate nuscenes_infos_temporal_{train,val}.pkl.

You can also download prepared pkl files from MapTRv2.

Step 2: Generate Geographic Data (nuScenes-Geography)

Following the instructions in the SpatialRetrievalAD-Dataset-Devkit project, prepare both the nuScenes-Geography dataset and its devkit.

After preparation, place the dataset directory nuScenes-Geography under data/

Step 3: Add Retrieved Geographic Data

After preparing the base dataset, merge geographic information:

python tools/merge_data.py

Final Folder Structure

Online_Mapping/
├── mmdetection3d/
├── projects/
├── tools/
├── ckpts/
├── data/
│   ├── infos/
│   │   ├── nuscenes_map_infos_temporal_train.pkl
│   │   ├── nuscenes_map_infos_temporal_val.pkl
│   ├── can_bus/
│   ├── nuscenes/
│   │   ├── maps/
│   │   ├── samples/
│   │   ├── sweeps/
│   │   ├── v1.0-test/
│   │   ├── v1.0-trainval/
│   ├── nuScenes-Geography-Data/

🚄 Training & Evaluation

Train MapTRv2_Geo with 8 GPUs

bash ./tools/dist_train.sh projects/configs/maptrv2_geo/maptrv2_nusc_r50_110ep_geo.py 8

Eval MapTRv2_Geo with 8 GPUs

bash ./tools/dist_test_map.sh projects/configs/maptrv2_geo/maptrv2_nusc_r50_110ep_geo.py ./path/to/ckpts.pth 8

🖊️ Citation

@misc{spad,
      title={Spatial Retrieval Augmented Autonomous Driving}, 
      author={Xiaosong Jia and Chenhe Zhang and Yule Jiang and Songbur Wong and Zhiyuan Zhang and Chen Chen and Shaofeng Zhang and Xuanhe Zhou and Xue Yang and Junchi Yan and Yu-Gang Jiang},
      year={2025},
      eprint={2512.06865},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.06865}, 
}

🙏 Acknowledgements

This work is based on MapTRv2. It is also greatly inspired by the following outstanding contributions to the open-source community: BEVFormer, PETR.