Corpus

Every single-element Materials Project entry (num_elements = 1) — 834 structures across 89 elements, with no convex-hull cutoff (the corpus is not filtered to on-hull phases). There are 90 ground-state rows: Br, I and O each have two on-hull ground-state polymorphs, and Hg and Pr have none, so it is not one ground state per element. One row per distinct polymorph per element.

DFT ranking variable. Ordering uses the Materials Project energy_above_hull. Within one element this differs from true energy per atom only by an additive constant, so per-element ranking, ground-state identification and relative-energy MAE are identical to using E/atom. No absolute DFT energy is stored, so these numbers must not be used for cross-element energetics.

Evaluation modes

Mode What Cell
sp (static) single point at the fixed MP geometry —
hydro (hydrostatic) volume-only relaxation; cell shape frozen isotropic volume
full (full relax) BFGS + FrechetCellFilter variable cell (shape + volume)

The leaderboard shows three evaluation modes: static (sp, single-point at the MP geometry), hydrostatic (hydro, volume-only relaxation with the cell shape frozen) and full relax (full, cell shape + volume via FrechetCellFilter). The two relaxed modes come from the dedicated relax-cell campaign (59 potentials × 834 structures × 2 modes); each ordering row uses the final relaxed energy on status = done relaxations (BFGS, fmax = 1e-2 eV/Å).

Metrics

Metric Definition
mean_spearman mean Spearman rank correlation of MLIP vs DFT polymorph ordering across elements (higher is better) — the headline and default sort
gs_hit_rate fraction of elements whose lowest-energy polymorph matches DFT; single-polymorph elements are excluded from the denominator
median_dE_mae median over elements of the per-element MAE of relative polymorph energies (median, not mean — the distribution has heavy tails)

Rows are aggregated per (potential, element, eval_mode); the default weighting counts each element once (micro), with an element-category macro option.

Reference & caveats

The reference is the DFT polymorph ordering itself (no separate experimental comparand). CUDA device-assert recovery (exit + requeue) is implemented only for the single-point worker; the relax worker records an error row and continues.

Provenance & verification (deep, code-cited) → /methodology/deep/ordering

See the leaderboard → · ← Methodology overview

Code

Per element: relax every candidate polymorph and rank by energy per atom; the lowest is the predicted ground state (compared against the DFT ranking).

from ase.filters import FrechetCellFilter
from ase.optimize import FIRE
calc = ...   # any ASE calculator for your MLIP (e.g. mace_mp(model="medium", device="cuda"))

def relaxed_epa(atoms):
    atoms = atoms.copy(); atoms.calc = calc
    FIRE(FrechetCellFilter(atoms)).run(fmax=1e-2, steps=300)   # full cell+ion relax
    return atoms.get_potential_energy() / len(atoms)           # eV/atom

ranking = sorted(polymorphs, key=relaxed_epa)   # ranking[0] = predicted ground state