WildDet3D · NeurIPS 2026 Rebuttal Tracker

Meta Review of Submission10926 by Area Chair PPk5
Meta Reviewby Area Chair PPk519 Jul 2026, 11:55 (modified: 23 Jul 2026, 10:42)Senior Area Chairs, Area Chairs, Authors, Reviewers Submitted, Program Chairs, Area Chair PPk5[Revisions](https://openreview.net/revisions?id=N8y5GX0R0O)
Metareview:
This paper proposes an architecture that detects objects in 3D (more exactly predicts a 3D bounding box) from images and a dataset, which is used to train this architecture. The architecture takes various types of prompts (similar to the ones for SAM) and is relatively standard now. The dataset is large and created semi-automatically from existing 2d datasets (COCO, LVIS, Obj365, V3Det).

Strength: The model and the dataset are potentially very valuable.

Weaknesses: The reviewers are not convinced that the dataset annotations are really high quality. Rev bVkw complains that the technical novelty is limited--this is not a problem for the AC (no need to invent a fancy architecture if a simpler one is just fine) however, the contribution is then more on the data creation side, which is not really suitable to NeurIPS.

Because of these weaknesses, the paper is aiming toward rejection at this stage.

Add:
Official Review of Submission10926 by Reviewer Z8Kd
Official Reviewby Reviewer Z8Kd28 Jun 2026, 08:14 (modified: 23 Jul 2026, 07:23)Program Chairs, Senior Area Chairs, Area Chairs, Reviewers Submitted, Authors, Reviewer Z8Kd[Revisions](https://openreview.net/revisions?id=FHNaJxfA81)
Summary:
This paper presents WildDet3D, an open-world monocular 3D object detector with optional depth as input. WildDet3D is designed to support different types of prompts, including text, box, point, and exemplar prompts, in one architecture. The architecture includes a RGB image encoder, a RGB-D encoder, a depth fusion module with zero-conv, followed by 2D/3D detection heads. Crucially, the authors propose a new large-scale dataset, WildDet3D-Data, to train the WildDet3D detector. WildDet3D-Data is built by lifting existing 2D annotations from various datasets (COCO, LVIS, Obj365, V3Det) into 3D using multiple existing 3D detection methods, and then selecting/filtering the candidates through human annotation and VLM scoring. The final dataset covers many more categories than existing 3D detection datasets. The paper reports strong results on WildDet3D-Bench, Omni3D, zero-shot AV2/ScanNet, and two additional benchmarks for stereo and embodied settings.

Contribution Type: General: Most submissions will fall into this type.
Strengths And Weaknesses:
Strengths
The paper addresses an important problem of open-world 3D detection from single images, which is useful for robotics, AR/VR, and embodied AI. Supporting different types of prompts and in one model is a practical direction.
The effort of making the WildDet3D-Data dataset is very impressive. The authors propose a human-in-the-loop pipeline to select the candidate annotations, and the final dataset delivers a large category coverage. The ablation studies show that the dataset can be potentially very useful to the community.
Weaknesses
The promptable claim is broader than the evaluation shows. The paper claims support for text, box, point, and exemplar prompts, but the main quantitative evaluation only covers text and box prompts. I also did not see the qualitative results of point and exemplar prompts in the paper or from the supplement videos. Although the authors may not be able to compare against other methods on point and exemplar prompts, showing quantiative numbers as an ablation study to compare with box and text prompts as well as qualitative results are crucial to support the claim of the paper.
I am not convinced that the WildDet3D-Data is a high quality dataset with accurate 3D bounding boxes. WildDet3D-Data seems to be built from lifted 2D annotations using candidate 3D box generators plus human/VLM selection, rather than from independent metric 3D ground truth. This could be useful, but can also be very noisy, as we can clearly see from the GT row in Figure 5, and potentially biased toward methods similar to the candidate generators. In Table 8, the authors provide a dataset ablation study and show the trend of using WildDet3D-Data for training is useful. But critically it uses WildDet3D-Bench, which is created using the same annotaion pipeline, as the evaluation ground truth, which can bias the evalution toward the WildDet3D-Data training. In the table it also shows that using "Other 3D" did not help the evaluaiton on WildDet3D-Bench, making me more suspicious that this bias is real.
Quality: 3: good
Clarity: 3: good
Significance: 2: not good
Originality: 2: not good
Questions:
Can the authors provide quantiative evaluations on the point and examplar prompts?
How independent is WildDet3D-Bench from the annotation pipeline? Since the 3D labels are selected from candidates produced by existing models and geometry heuristics, could the benchmark favor methods trained on similar generated labels? Can the authors provide a similar ablation as in Table 8, but using other more accurate datasets, such as Omni3D and/or ScanNet, as the evaluation?
Limitations:
yes

Rating: 3: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Ethical Concerns: NO or VERY MINOR ethics concerns only
Paper Formatting Concerns:
none

Code Of Conduct Acknowledgement: Yes
Responsible Reviewing Acknowledgement: Yes
Add:
Official Review of Submission10926 by Reviewer qeMe
Official Reviewby Reviewer qeMe23 Jun 2026, 15:16 (modified: 23 Jul 2026, 07:23)Program Chairs, Senior Area Chairs, Area Chairs, Reviewers Submitted, Authors, Reviewer qeMe[Revisions](https://openreview.net/revisions?id=1SbXkgQ6nJ)
Summary:
WildDet3D targets open-world monocular 3D detection with two contributions: (1) a geometry-aware architecture that takes a single RGB image, accepts text/point/box/exemplar prompts through a shared encoder, and treats depth as an optional inference-time refinement and (2) WildDet3D-Data, a 1M-image open-vocabulary 3D dataset (13.5K categories, 230K human-verified boxes via 1,786 Prolific annotators plus ~900K VLM-scored), built by lifting 2D detection datasets to 3D through five candidate generators, multi-stage filtering, and a two-path human/VLM selection pipeline. They report SOTA on Omni3D (34.2/36.4 text/box, 10× fewer epochs), large zero-shot gains on AV2/ScanNet, and a new in-the-wild benchmark plus stereo and embodied benchmarks.

Contribution Type: General: Most submissions will fall into this type.
Strengths And Weaknesses:
Strengths

High Quality data: 138× Omni3D's category count, with a human-in-the-loop pipeline (gold-task QC, 84–98% pass rates) rather than VLM-only labeling albiet VLM scoring.

+4.2 on Omni3D, +16.5/+17.4 ODS zero-shot, and a near-doubling with depth are well outside noise

Weaknesses

On WildDet3D-Bench the depth signal and the ground-truth 3D boxes are both derived from the same MoGe-2 monocular depth via the same lifting pipeline. So "add depth -> 22.6 -> 41.6" partly measures agreement between the model's depth input and depth-derived labels, not independent geometric gain.

WildDet3D's gains conflate a new architecture, a new training set, and strong pretrained backbones (SAM 3 ViT-H, a metric-depth DINOv2). To know what the architecture contributes, one wants a baseline trained on the same WildDet3D-Data, or WildDet3D trained on only Omni3D vs 3D-MOOD on Omni3D at matched backbone capacity

Quality: 3: good
Clarity: 3: good
Significance: 3: good
Originality: 3: good
Questions:
Overall, the dataset contribution is a significant one. I would have liked more backbone ablations to understand the architecture better.

Limitations:
yes

Rating: 5: Accept: Technically solid paper, with high potential value on at least one sub-area of AI or moderate-to-high impact on more than one area of AI, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Ethical Concerns: NO or VERY MINOR ethics concerns only
Paper Formatting Concerns:
na

Code Of Conduct Acknowledgement: Yes
Responsible Reviewing Acknowledgement: Yes
Add:
Official Review of Submission10926 by Reviewer bYkw
Official Reviewby Reviewer bYkw19 Jun 2026, 04:08 (modified: 23 Jul 2026, 07:23)Program Chairs, Senior Area Chairs, Area Chairs, Reviewers Submitted, Authors, Reviewer bYkw[Revisions](https://openreview.net/revisions?id=bRrwbpFfWz)
Summary:
This paper presents WildDet3D, a promptable open-vocabulary monocular 3D object detection framework that supports multiple prompt modalities, including text, 2D points, and 2D boxes, while optionally leveraging depth cues at inference time. The authors further introduce WildDet3D-Data, a large-scale in-the-wild 3D detection dataset with human-in-the-loop annotation and VLM-assisted scaling. Extensive experiments on the proposed benchmark, Omni3D, zero-shot transfer settings, and robotic/stereo scenarios show that WildDet3D achieves strong performance over prior methods, especially when depth information is available.

Contribution Type: General: Most submissions will fall into this type.
Strengths And Weaknesses:
clear writing and figures.
extensive experiments and achive state-of-the-art performance.
introduce WildDet3D, a geometry-aware architecture that unifies text, point, box, and exemplar prompts.
introduce WildDet3D-Data, the first truly open-vocabulary 3D detection dataset—built by applying multiple complementary methods to generate candidate 3D boxes for 2D annotations from diverse detection datasets.
Quality: 3: good
Clarity: 3: good
Significance: 2: not good
Originality: 2: not good
Questions:
I personally disagree with the claim at Line 75. It is impossible to infer metric knowledge from a single RGB. Only supply depth/3D information can get metric knowledge.
It uses many VLM/3D foundation models to annotate the dataset. However, those large models are not perfect. For example, using MoGe-2 to infer metric depth is not fully accurate while the pipeline are based on the MoGe-2 depth. It is very important to analysis the depth quality and how it affects the downstream annoatation pipeline. Second, it is hard to filter per-category physical dimensions using GPT-4.1-mini (or other VLMs) due to large intra-class variance. And the depth-to-width ratio/ axis proportions are hard to define robustly because they depend on the object pose: after rotation, an object’s intrinsic height, width, or depth may align with different coordinate axes.
The technique novelty is a bit limited. The whole pipeline is built upon existing modules, such as controlnet-style module, SAM 3, DINOv2, LingBot Depth. To me, it more looks like an engineer work. Moreover, the claims at Line 74-79 remain questioned since it is hard to determin whether the performance comes from the using multiple foudation modules or their depth design.
Limitations:
The authors have discussed limitation in the supplmentary.

Rating: 3: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Ethical Concerns: NO or VERY MINOR ethics concerns only
Paper Formatting Concerns:
Not avaliable.

Code Of Conduct Acknowledgement: Yes
Responsible Reviewing Acknowledgement: Yes