Schema and units

Each parquet has one row per DFT calculation. Atom-resolved fields (positions, dft_forces, dft_magmoms, numbers) are stored as flat lists and need reshaping to (n_atoms, 3) or (n_atoms,) after loading. cell and the stress tensors are flat length-9 lists representing row-major (3, 3) matrices.

Column	Type	Shape after reshape	Units	Notes
id	int64	scalar	—	Unique row id within the export
atoms_hash	string	scalar	—	SHA-256 of the structure (unique per row — no duplicate structures)
source_files	list<string>	(k,)	—	Native source parquet(s) the row came from (per-row provenance)
dataset	string	scalar	—	FM_MatBench_<Class>_<Composition>
host / element	string	scalar	—	Host element and (for defects) the solute
z	int32	scalar	—	Atomic number of the host element
defect_type	string	scalar	—	Unary / Binary / Ternary
site, gb, state	string/float	scalar	—	Defect site, GB id, state tag (often empty)
job_name	string	scalar	—	Pipeline job identifier
n_atoms	int32	scalar	—	Atom count for this row
formula	string	scalar	—	Hill formula, e.g. Cu4Fe4
convergence	bool	scalar	—	SCF converged
numbers	list<int32>	(n_atoms,)	—	Atomic numbers Z
positions	list<f64>	(n_atoms, 3)	Å	Cartesian coordinates
cell	list<f64>	(3, 3)	Å	Row-major lattice vectors
pbc	list<bool>	(3,)	—	Periodic boundary flags
initial_magmoms	list<f64>	(n_atoms,)	μB/atom	Initial moments (often None)
dft_method	string	scalar	—	e.g. VASP
dft_energy	float64	scalar	eV	Total DFT energy
dft_forces	list<f64>	(n_atoms, 3)	eV/Å	Atomic forces
dft_stress_kbar	list<f64>	(3, 3)	kbar	Raw VASP-convention stress
dft_stress_eV_per_A3	list<f64>	(3, 3)	eV/ų	Same tensor in ASE-friendly units
dft_magmoms	list<f64>	(n_atoms,)	μB/atom	Final magnetic moments
eseg	float64	scalar	eV	Segregation energy (if applicable)
extra	string	scalar	—	JSON blob of INCAR tags + provenance; the per-row ISPIN (1 = non-spin-polarised, 2 = spin-polarised) that drives the magnetic split lives here

Rebuilding ASE Atoms + numpy arrays

The flat list columns round-trip cleanly through pandas. Reshape on the way out and either work with numpy arrays directly or feed them into ASE:

import pandas as pd
import numpy as np
from ase import Atoms
from ase.calculators.singlepoint import SinglePointCalculator

df = pd.read_parquet("FM_MatBench_Binary_Fe_Cu.parquet")  # or pyarrow / duckdb
row = df.iloc[0]

# numpy arrays
numbers   = np.asarray(row["numbers"], dtype=int)              # (n_atoms,)
positions = np.asarray(row["positions"], dtype=float).reshape(-1, 3)   # Å
cell      = np.asarray(row["cell"], dtype=float).reshape(3, 3)         # Å
pbc       = np.asarray(row["pbc"], dtype=bool)                  # (3,)
energy    = float(row["dft_energy"])                            # eV
forces    = np.asarray(row["dft_forces"], dtype=float).reshape(-1, 3)  # eV/Å
stress    = np.asarray(row["dft_stress_eV_per_A3"], dtype=float).reshape(3, 3)  # eV/ų
magmoms   = np.asarray(row["dft_magmoms"], dtype=float)         # μ_B / atom

# ASE Atoms with the DFT result attached
atoms = Atoms(numbers=numbers, positions=positions, cell=cell, pbc=pbc)
atoms.calc = SinglePointCalculator(
    atoms,
    energy=energy,
    forces=forces,
    stress=stress,        # SinglePointCalculator accepts (3, 3) or Voigt (6,)
    magmoms=magmoms,
)

To iterate the whole file as an ASE list[Atoms]:

def row_to_atoms(row):
    atoms = Atoms(
        numbers=np.asarray(row["numbers"]),
        positions=np.asarray(row["positions"]).reshape(-1, 3),
        cell=np.asarray(row["cell"]).reshape(3, 3),
        pbc=np.asarray(row["pbc"]),
    )
    atoms.calc = SinglePointCalculator(
        atoms,
        energy=float(row["dft_energy"]),
        forces=np.asarray(row["dft_forces"]).reshape(-1, 3),
        stress=np.asarray(row["dft_stress_eV_per_A3"]).reshape(3, 3),
        magmoms=np.asarray(row["dft_magmoms"]) if row["dft_magmoms"] is not None else None,
    )
    return atoms

frames = [row_to_atoms(r) for _, r in df.iterrows()]

dft_stress_kbar and dft_stress_eV_per_A3 are the same tensor in two unit systems (1 eV/ų ≈ 1602.176 kbar) — use the eV/ų version when handing the stress to ASE.

Citation

TBD

License

TBD