WeirdML Leaderboard

WeirdML (WeirdML (currently WeirdML v2)): Tests an LLM's ability to understand an unusual dataset, design an appropriate ML architecture, and write PyTorch code that trains and predicts, iterating over feedback. It targets creative ML-engineering rather than memorized patterns.

GPT-6 Astra leads WeirdML at 93.57. The full leaderboard above lists every recorded measurement, not just the headline number.

119 models have recorded scores on WeirdML, spanning a score spread of 91.84.

tensor.news grades WeirdML A for integrity (score 86/100), ranking #30 of 65 benchmarks we assess across discrimination, saturation, contamination resistance, harness comparability, and freshness.

No — WeirdML still has headroom and continues to discriminate between models rather than bunching them at the ceiling.

Its test set is semi-private, which lowers — but does not eliminate — the risk that training data leaked into the questions.

Scores are reported under a consistent harness, so comparisons on WeirdML are reasonably apples-to-apples.

Every WeirdML measurement is source-backed and tagged reproduced, self-reported, or unverified; the underlying record cites htihle.github.io/weirdml.html.