The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes
Contributing person | National Center for Tumor Diseases (NCT) - Experimental OR Team | |
References to related material | Docea, R. et al. The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes. Sci. Data (2025). Manuscript submitted for publication. | |
References to related material | https://github.com/reubendocea/d4d | |
Description of the data | This dataset supports research on non-rigid 3D/4D reconstruction, SLAM and depth estimation in minimally invasive surgery. Use rectified images, masks, and stereo depth in clips/, intrinsics in camera_info/, and curated_camera_pose_{start,end}.txt to align reconstructions to structured-light start/end point clouds. Evaluate geometry by comparing to SLC point clouds and optionally assess photometric quality of view-synthesis with PSNR/SSIM/LPIPS on non-masked pixels. Minor residual pose misalignments may remain despite the pose refinement steps undertaken. Whole sequences assess end-to-end robustness, Incremental sequences probe deformation magnitude and enable a more granular study of tissue deformation, and Moved-camera sequences test handling of out-of-view deformation and camera movement. While the Moved-camera sequences test the reconstruction of out-of-view deformation in a more significant manner, all sequences in fact possess out-of-view deformation whose evaluation this dataset also aims to enable. The ideal ground truth for this assessment would be to know correspondences between the points in the SLC pointclouds and points in the reconstructed pointclouds derived from the clips themselves - this is in practice, however, difficult to achieve. A lower-hanging fruit would be to evaluate simply what lies behind masked out tools, assuming that the respective tools have moved and what lies behind has already been observed during the clip. If the evaluation protocol involves tuning hyperparameters (or training parameters) on the dataset prior to testing, we recommend using a 4:1 train–test split (across the 98 core sequences of Specimens 1 to 5) within a 5-fold Leave-One-Out cross-validation setup. The data are usable with standard open-source Python tools and can be quickly interacted with through the following repository: https://github.com/reubendocea/d4d. | |
Type of the data | Dataset | |
Type of the data | Image | |
Type of the data | Model | |
Total size of the dataset | 480212297387 | |
Author | Docea, Reuben | |
Author | Younis, Rayan | |
Author | Long, Yonghao | |
Author | Fleury, Maxime | |
Author | Xu, Jinjing | |
Author | Li, Chenyang | |
Author | Schulze, André | |
Author | Wierick, Ann | |
Author | Bender, Johannes | |
Author | Pfeiffer, Micha | |
Author | Dou, Qi | |
Author | Wagner, Martin | |
Author | Speidel, Stefanie | |
Upload date | 2025-12-10T19:42:56Z | |
Publication date | 2025-12-10T19:42:56Z | |
Data of data creation | 2025 | |
Publication date | 2025-12-10 | |
Abstract of the dataset | The D4D Dataset provides paired endoscopic video and high-quality structured-light geometry for evaluating 3D reconstruction of deforming abdominal soft tissue in realistic surgical conditions. Data were acquired from six porcine cadaver sessions using a da Vinci Xi stereo endoscope and a Zivid structured-light camera, registered via optical tracking and manually curated iterative alignment methods. Three sequence types - whole deformations, incremental deformations, and moved-camera clips - probe algorithm robustness to non-rigid motion, deformation magnitude, and out-of-view updates. Each clip provides rectified stereo images, per-frame instrument masks, stereo depth, start/end structured-light point clouds, curated camera poses and camera intrinsics. In postprocessing, ICP and semi-automatic registration techniques are used to register data, and instrument masks are created. The dataset enables quantitative geometric evaluation in both visible and occluded regions, alongside photometric view-synthesis baselines. Comprising over 300,000 frames and 369 point clouds across 98 curated recordings, this resource can serve as a comprehensive benchmark for developing and evaluating non-rigid SLAM, 4D reconstruction, and depth estimation methods. | |
Public reference to this page | https://opara.zih.tu-dresden.de/handle/123456789/1882 | |
Public reference to this page | https://doi.org/10.25532/OPARA-1033 | |
Publisher | Technische Universität Dresden | |
Licence | Attribution 4.0 International | en |
URI of the licence text | http://creativecommons.org/licenses/by/4.0/ | |
Specification of the discipline(s) | 4::44::409::409-05 | |
Specification of the discipline(s) | 2::22::205::205-07 | |
Title of the dataset | The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes | |
Research instruments | da Vinci Xi using stereo endoscope with 0° or 30° tip | |
Research instruments | Zivid 2+ M60 Structured Light Camera | |
Research instruments | Polaris Vega Optical Tracking System | |
Research instruments | Polaris Spectra Optical Tracking System | |
Underlying research object | Pig | |
Software | Python, C++, Robot Operating System 2 (ROS2) | |
Funding Acknowledgement | This project has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No 101092646. This work was supported by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany's Excellence Strategy – EXC 2050/1 – Project ID 390696704 – Cluster of Excellence "Centre for Tactile Internet with Human-in-the-Loop" (CeTI) as well as by the German Federal Ministry of Health (BMG) within the SurgOmics project (grant number BMG 2520DAT82). The authors acknowledge the financial support by the Federal Ministry of Research, Technology and Space of Germany in the programme of "Souverän. Digital. Vernetzt.". Joint project 6G-life, project identification number: 16KISK001K. This work is supported by the project "Next Generation AI Computing (gAIn)," funded by the Bavarian Ministry of Science and the Arts and the Saxon Ministry for Science, Culture, and Tourism. The authors acknowledge the financial support by the Federal Ministry of Research, Technology and Space of Germany in the programme of “DigiLeistDAT”. Joint project SurgicalAIHubGermany, project identification number: 02K23A112. |
Files
Original bundle
- Name:
- specimen_0_ambiguous.tar.gz
- Size:
- 1.25 GB
- Format:
- Unknown data format
- Description:
- Name:
- specimen_1_ambiguous.tar.gz
- Size:
- 9.93 GB
- Format:
- Unknown data format
- Description:
- Name:
- specimen_2_ambiguous.tar.gz
- Size:
- 9.69 GB
- Format:
- Unknown data format
- Description:
- Name:
- specimen_3_ambiguous.tar.gz
- Size:
- 2.88 GB
- Format:
- Unknown data format
- Description:
- Name:
- specimen_5_ambiguous.tar.gz
- Size:
- 959.37 MB
- Format:
- Unknown data format
- Description:
License bundle
- Name:
- license.txt
- Size:
- 3.86 KB
- Format:
- Item-specific license agreed to upon submission
- Description:
Collections
