Introducing ScienceVision

A scientific question about a three-dimensional object asks three things at once: find the structure the question is about, measure its geometry, and interpret what you measured using what you know about the field. Fixed images, the native data file, and interactive tools each give a different amount of help with the first two.

The questions are concrete. Find the pipe bundle furthest from the origin in a building model, then count how many independent pipes it contains, not how many segments they are drawn in. Locate a target in a CT volume, measure its extent, and report a signed centroid offset under a specified coordinate convention. Neither can be answered from a picture of the object.

ScienceVision asks 250 of them. Fifty each in biology, chemistry, medical imaging, mechanical CAD and architecture, across 22 subdomains. Each task keeps the artifact in its native representation (molecular, volumetric, CAD or building) and pairs it with a question and a task-specific reference answer.

“Expert-level” here means the domain knowledge and conventions needed to interpret these objects. It is not difficulty calibrated against a human baseline, and the benchmark does not claim one.

Three ways to see the same object

The design is a comparison between three kinds of access to the same artifact.

1. Source. A coding agent gets the native file and can execute code against it.
2. Source + MCP. The same, plus an inspection interface exposed through the Model Context Protocol: changing views, isolating structures, navigating slices, measuring geometry.
3. Fixed 6. Six pre-rendered views and the question. The solver cannot choose a new viewpoint. This is the image-based control.

What the tools look like in use

The MCP condition is not one picture. The solver opens the artifact and then works on it, orbiting, zooming, hiding what is in the way, slicing through it and measuring, for dozens of operations before it answers. Below are five of those recorded sessions, one per discipline, from UniPat’s own run.

Inspect one yourself

Watching a recorded session is still watching. These are the same five artifacts, opened in the viewer their discipline uses — the file itself, not a rendering of it. Orbit it, zoom it, turn it over. Nothing is uploaded and nothing is scored; this is the expert-facing viewer, not the evaluation harness.

The point of the split is to keep honest about what a score means. A model with source access can compute directly over the artifact, so a correct answer there is not evidence of successful visual observation. The paper is explicit about that, and so is this post.

Giving a model tools does not reliably help

Source access beats fixed views for every model tested, without exception. Being able to interrogate the artifact is worth more than being handed pictures of it.

Adding the inspection tools is not. Source access with MCP beats source access alone for eight of the twelve models and loses for four, and the margins are small either way, from +2.17 points for GPT-6.1 Sol to −2.40 for Kimi K3. The tools are the part of this benchmark that took the most work to build, and across a twelve-model field they do not reliably pay for themselves.

And the leader changes with the setting. GPT-6 Astra tops the source-access settings at 0.53. Claude Fable 5.1 tops the fixed-view condition at 0.30, ahead of the models that lead on source access. There is no single ordering of these models; there is an ordering per kind of access.

The gains and the losses cancel out

Looking at why the tools wash out is more interesting than the average. The tools help in some domains and cost in others, and the two roughly cancel.

The domains are not equally hard. Biology holds the highest single domain score in the set, 0.65, and no model falls below 0.24 in either biology or chemistry. Mechanical CAD, medical imaging and architecture have the lowest floors, with their weakest models under 0.10. That gap describes performance on these particular questions under these configurations. It does not establish that medical imaging is intrinsically harder, and it does not say whether the errors came from observation, calculation, domain interpretation or answer formatting.

Why this sits alongside DecoyCount

We published DecoyCount the same week, and the two benchmarks arrive at a similar place from opposite directions.

On DecoyCount, handing a model a general code-interpreter harness improved its accuracy and widened the gap between what it could do when told the rule and what it did unaided. The one intervention that narrowed that gap was targeted segmentation — a tool supplying the specific missing visual evidence.

On ScienceVision, a general-purpose inspection interface helps eight models and hurts four, while the per-domain picture shows it helping exactly where the evidence it supplies is the evidence the question needs, and hurting where it is not.

Two benchmarks, different artifacts, different failure modes, same conclusion: giving a model more general capability, or more general tooling, is not the same as giving it the right evidence.

Get the benchmark

Scores are averaged over the available repeats within each task, then across the 250 tasks. All 36 reported configurations carry a score for every task.

Who built it

ScienceVision was built by Elorian AI and UniPat AI together, with expert annotation across all five disciplines coordinated through a single platform.

Core contributors

Tianyi Ma^2,*, Jing Gu^1,*, Haozhe Zhao^2,‡, Liang Chen^2,‡, Yunfan Zhang², Haijun Su¹, Jialong Wu², Le Xue¹, Arnav Das¹, Dustin Tran¹, Yinfei Yang¹, Andrew Dai^1,†, Kuan Li^2,†

Contributors

Liangkun He², Zihao Guo², Xiang Gong², Yutao Zhou², Jiarui Sun², Chengda Wu², Zhao Zhang², Qianhui Yang², Zhenji Kang², Yi Shen², Wenda Wang¹

¹ Elorian AI  ·  ² UniPat AI  ·  ^* Equal contribution  ·  ^‡ Project lead  ·  ^† Corresponding author

Beyond the named contributors, the annotation pool ran to 66 people and 482.98 recorded hours: 74 in biology, 113.33 in chemistry, 103 in medical imaging, 115.65 in mechanical CAD and 77 in architecture, covering question writing, revision, reference-answer annotation and review.