Low dimensional benchmark by JonathanSchmidt1 · Pull Request #355 · ddmms/ml-peg

JonathanSchmidt1 · 2026-02-09T08:07:02Z

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.

Summary

Adds relaxation of two-dimensional and one-dimensional crystal structures benchmarking energies and areas/lengths.

Linked issue

Resolves #354

Progress

Calculations
Analysis
Application
Documentation

Testing

mace-mpa-0

New decorators/callbacks

No

…l-peg into high-pressure

Initial implementation of benchmark for 2D crystal structures from Alexandria database. Uses cell mask to constrain relaxation to in-plane directions only. Metrics: - Area MAE (Å²/atom): In-plane area per atom vs PBE reference - Energy MAE (eV/atom): Energy per atom vs PBE reference - Convergence (%): Percentage of structures that converged Data source: alexandria_2d_001.json.bz2 from Alexandria database Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add generic download_data() and load_structures() functions for both 2D/1D - Add get_length_per_atom() for 1D chain length calculation - Add relax_low_dimensional() with dimensionality parameter for cell masks - Add 1D metrics: Length MAE, Energy MAE, Convergence - Add 1D parity plots for length and energy - Parametrize test function for both 2D and 1D dimensionalities - Update app with 1D plot configurations Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Document the new low-dimensional relaxation benchmark including: - Summary of 2D and 1D structure relaxation - Metrics: Area/Length MAE, Energy MAE, and Convergence for both 2D and 1D - Description of cell masks for constrained relaxation - Data availability from Alexandria database Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add set_vacuum_padding() function that sets lattice constants to 100 Å in non-periodic directions while preserving vector directions: - 2D: sets c vector to 100 Å - 1D: sets b and c vectors to 100 Å Apply vacuum padding before relaxation in test function. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add symmetrize_structure() using spglib.standardize_cell() - Remove FixSymmetry constraint (doesn't work for low-d relaxations) - Apply symmetrization before vacuum padding and relaxation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update DEFAULT_DATA_FILES to use lists (2 files for 2D, 1 for 1D) - Add download_all_data() to download multiple files - Update load_structures() to pool entries from all files before sampling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Replace 10 near-identical per-dimensionality fixtures with: - DIM_CONFIGS dict mapping dimensionality to column/label/unit - _compute_mae() and _compute_convergence() helpers - _generate_all_plots() looping over DIM_CONFIGS - Single parity_plots fixture and loop-based metrics fixture Reduces 465 lines to 353 lines with no duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace custom symmetrize_structure implementation with ase.spacegroup.symmetrize.refine_symmetry. Drop spglib import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…anSchmidt1/ml-peg into low_dimensional_benchmark

- Remove symmetrize_structure(), call refine_symmetry() directly - Add test_get_area_per_atom and test_get_length_per_atom unit tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

joehart2001 · 2026-04-15T20:02:41Z

+            geom_opt.run()
+            relaxed = geom_opt.struct
+            # assess forces to determine convergence
+            max_force = max(relaxed.get_forces().flatten(), key=abs)


I think key=abs chooses the largest value regardless of the sign (good), but returns the signed version of the value (bad).

joehart2001 · 2026-04-15T20:05:04Z

+                steps=max_steps,
+                # Use cell mask to constrain relaxation to appropriate dimensions
+                filter_kwargs={"mask": CELL_MASKS[dimensionality]},
+                calc_kwargs={"default_dtype": "float64"},


same issue as in pr #340

joehart2001 · 2026-04-15T20:19:26Z

Thanks for the PR @JonathanSchmidt1 ! some comments:

i think a good place for the data download would be in ~/cache/ml_peg/low_dimensional_relaxation, as this is where the s3 download stores the other data
i think we should add structure visualisaitont to the density scatters
can you mark the test with either slow or very slow using e.g. @pytest.mark.slow
rebase

Convergence:

i think i remember discussing this in a drop in on how to best visualise this?
one option could be a bar chart, which is being added in PR Water slab dipoles #349
but it is probably interesting to see which are the structures which fail, so maybe a scatter of energy vs area where the colour is converged or not converged? i did something like this ish for the phonon benchmakr wehre we colour the points based on which quater of the confusion matrix the point is in
or maybe we save the forces and do a violin plot with the final max force, and we have a dashed line indicated the fmax thrreshold. maybe i like this idea the most. we can plot the points so we can still visualise the structures. heres the example from the phonon benchmark BZ metric:

JonathanSchmidt1 and others added 25 commits February 2, 2026 22:25

add high pressure benchmark

1883421

add random subsampling

cb5c052

remove outliers

8ef30cc

add orb-v3-cons-inf-mpa for comparison with original benchmark

5bbeba1

Merge branch 'ddmms:main' into high-pressure

b80781a

add docs for benchmark

1d2fb2e

Merge branch 'high-pressure' of https://github.com/JonathanSchmidt1/m…

de0ff5a

…l-peg into high-pressure

fix good/bad metrics, use internal mae function

3a06816

metrics.yml update

b0c5d56

add two path

4a9d2d1

Merge branch 'ddmms:main' into low_dimensional_benchmark

136f6d9

Use ASE refine_symmetry instead of custom spglib function

be940bb

Replace custom symmetrize_structure implementation with ase.spacegroup.symmetrize.refine_symmetry. Drop spglib import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Merge branch 'low_dimensional_benchmark' of https://github.com/Jonath…

ad072fb

…anSchmidt1/ml-peg into low_dimensional_benchmark

Remove symmetrize_structure wrapper, add geometry unit tests

9b02013

- Remove symmetrize_structure(), call refine_symmetry() directly - Add test_get_area_per_atom and test_get_length_per_atom unit tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Move geometry unit tests to tests/ directory

7e9ef0e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

remove high-pressure part

17b4926

add references remove high-pressure mention

68b131f

remove high pressure test

a17cf3e

ElliottKasoar added the new benchmark Proposals and suggestions for new benchmarks label Feb 9, 2026

JonathanSchmidt1 and others added 2 commits March 4, 2026 18:15

Merge branch 'ddmms:main' into low_dimensional_benchmark

fe4f31c

add xyz storage of relaxed structures and density plots

7af3ea5

joehart2001 reviewed Apr 15, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Low dimensional benchmark#355

Low dimensional benchmark#355
JonathanSchmidt1 wants to merge 27 commits intoddmms:mainfrom
JonathanSchmidt1:low_dimensional_benchmark

JonathanSchmidt1 commented Feb 9, 2026

Uh oh!

joehart2001 Apr 15, 2026

Uh oh!

joehart2001 Apr 15, 2026

Uh oh!

joehart2001 commented Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

JonathanSchmidt1 commented Feb 9, 2026

Pre-review checklist for PR author

Summary

Linked issue

Progress

Testing

New decorators/callbacks

Uh oh!

joehart2001 Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

joehart2001 Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

joehart2001 commented Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants