Skip to content

Add simplified SPINE script for modern spine-prod configs#79

Open
mjkramer wants to merge 4 commits intomainfrom
feature/spine-prod
Open

Add simplified SPINE script for modern spine-prod configs#79
mjkramer wants to merge 4 commits intomainfrom
feature/spine-prod

Conversation

@mjkramer
Copy link
Copy Markdown
Member

@mjkramer mjkramer commented Mar 23, 2026

This adds a new script run_spine.spine-prod.sh that sources spine-prod's configure.sh (so that included yamls are properly found, etc.).

The new script does away with the temp-dir creation / template substitution complexity of the old script, since it is no longer necessary with modern configs.

The interpretation of ND_PRODUCTION_SPINE_CONFIG is simplified: It's just the path relative to run-mlreco; no more magic. So if you want to use a config from spine-prod, it'll look like install/spine-prod/.... If you want to use a config maintained within ND_Production, it'll start with configs/....

Also, this catches up with SPINE's ongoing campaign to replace underscores with hyphens. A symlink from spine-prod to spine_prod is created to ensure backward compatibility.

Addresses #76

TODO:

  • Verify that SPINE's output filename mangling scheme is consistent. I have only tested this with one SPINE config so far (the "ND-LAr truth matching" config used in some of the recent samples for charge-readout studies). Confirmed with infer/nd-lar/full_chain_260310.yaml

@mjkramer mjkramer marked this pull request as ready for review March 27, 2026 01:09
source install/spine-prod/configure.sh

# spine does something interesting with the output filename
actualOutFile=${tmpOutDir}/${inName}.LARCV_$(basename "$outFile" .hdf5).h5
Copy link
Copy Markdown
Member

@alexbooth92 alexbooth92 Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I ran into this running the first NDLAr MiniProdN5 through SPINE, I fixed it at the spine-prod config level. @francois-drielsma and I have been discussing this config-based solution in this PR and it's likely not appropriate. However, before merging this PR, we should wait and see whether this "interesting" output filename behaviour is expected and whether team SPINE wish to fix it DeepLearnPhysics.

Copy link
Copy Markdown
Member

@alexbooth92 alexbooth92 Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now have a new tag of spine which includes work done by Francois to (details in this closed PR) change this behaviour.

Now, when run is provided one input file and an explicitly set -o, the output file name will be exactly what was passed to -o.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes from this new tag should have been addressed in #84. Please check @mjkramer!

Copy link
Copy Markdown
Member

@alexbooth92 alexbooth92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from making a decision on what I've hightlighted in my inline comment, this all looks good to me, thanks Matt!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants