See the question and answer. Hover the small points to see how hard it really is.
A
GPT-6 AstraOpenAI-reported result
64.6%Terminal-Bench Science 0.1
The best result in the published comparison.
Next best shown: Claude Fable 5.1 at 52.6%. GPT-5.6 Sol scored 22.4%.
Maximum score at any reasoning effort, measured in OpenAI’s research/API setup.Source ↗
01 Astronomy
The question, in plain English
Which star is hiding a planet?
Find a small, repeating dip in three stars’ brightness. Classify all three. Make the same program work on six sets you cannot see.
25,635brightness readings
3stars in each set
6hidden tests
What the model sees
Go deeperHover or tap any point
Detail 1 of 5Five signals in every row
The visible set contains 7,221 + 9,192 + 9,222 time-stamped measurements. Each row includes brightness, uncertainty and the star’s detector position in two directions.
Detail 2 of 5A tiny dip inside messy data
The program must separate a shallow, repeating planet-shaped dip from stellar activity, missing observations, telescope motion and eclipse-like false alarms.
Detail 3 of 5Six sets the model never sees
The same Python file is rerun offline on 18 new stars. Names, file order and signal shapes change—and missing even one of the seven planet candidates fails the task.
Detail 4 of 5Nearly perfect is required
It must correctly find all 7 planets, score at least 0.95 across 21 classifications, and recover each planet’s timing and depth above strict per-set thresholds.
Detail 5 of 5No internet. 150 seconds.
Each hidden run gets 3 GB of memory and 150 seconds. The answer must be exact JSON; extra diagnostics are outside the schema and do not count.
AnswerStar 2
It dims by 0.235% for 3.2 hours every 2.7342 days.
Star 1 Two stars eclipse each otherStar 3 Variable star or telescope noise
Follow 40 lakes through one summer. Decide how each drained from five possible explanations.
6,120daily pictures
2.9 GBimage data
5possible answers
8 of 153 days for one lake
Go deeperHover or tap any point
Detail 1 of 5Forty lakes, every day
Each lake has 153 daily frames. Every 512 × 512 image covers the ice at 10 metres per pixel, with colour, cloud, water and lake-boundary data.
Detail 2 of 5Far more than a photo album
The model must open 40 scientific NetCDF files and combine imagery with a daily water-area time series. Looking at a few attractive pictures is not enough.
Detail 3 of 5The decisive day may be hidden
Cloud and snow can obscure the exact moment of drainage. The model has to connect evidence before and after the gap without mistaking cloud cover for vanished water.
Detail 4 of 5Five answers with tricky tie-breaks
Rapid in-basin drainage under four days defaults to hydrofracture. An outlet stream, a visible hole or a crevasse field can change the label—even when the lake looks similar.
Detail 5 of 5Every rare case matters
The 40 expert labels are kept from the model. It must match every label exactly, while the scoring also gives rare drainage types as much importance as common ones.
Answer shownCracked through
Lake 0018: the water rapidly drained down through the ice. Scientists call this hydrofracture.
Read every scan image. Find and measure the main injury, choose its diagnosis code, and cite the exact pictures that prove it.
237scanner pictures
8different scan views
82body structures
All 237 images
Go deeperHover or tap any point
Detail 1 of 5Eight views of one ankle
The scan contains 8 separate image series with 22–39 slices each: top-down, front and side views, using different settings to make anatomy or swelling stand out.
Detail 2 of 5Not ordinary image files
Each DICOM file stores pixels plus the slice’s position, angle and scale in the patient’s body. The model must turn chosen pixels into real 3D coordinates measured in millimetres.
Detail 3 of 5Thousands of possible combinations
The allowed answer system includes 82 body structures, 30 finding types, 34 measurement types and 59 possible diagnosis codes.
Detail 4 of 5The geometry is checked
The grader recalculates each distance from the model’s 3D points. If the stated measurement and coordinates disagree by more than 2 mm, that measurement scores zero.
Detail 5 of 5A radiologist’s answer key
The key is hidden. The main injury needs real citations from two different scan series, while confident but false claims—such as an acute fracture—reduce the score.
AnswerAchilles torn
The Achilles tendon is completely torn. The gap is 20–30 mm, around 70–90 mm above its heel attachment.
