Mag L2 rotation - propogate FILLVALS#2903
Mag L2 rotation - propogate FILLVALS#2903maxinelasp merged 3 commits intoIMAP-Science-Operations-Center:devfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates MAG L2/L1D frame rotation to preserve missing-data sentinels (NaN/FILLVAL) during rotations, preventing “nearly-but-not-quite” fill values from being introduced into rotated science vectors.
Changes:
- Apply a post-rotation mask in MAG L2 and L1D
rotate_frame()to propagate NaN/FILLVAL from the pre-rotation vectors into the rotated output. - Add unit tests for both L2 and L1D to verify FILLVAL/NaN preservation behavior during rotation.
- Adjust the MAG L2 test fixture vector dtype to floating-point to support NaN/FILLVAL handling.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| imap_processing/mag/l2/mag_l2_data.py | Propagates NaN/FILLVAL into rotated vectors after frame_transform() in L2 rotation. |
| imap_processing/mag/l1d/mag_l1d_data.py | Propagates NaN/FILLVAL into rotated MAGO/MAGI vectors after frame_transform() in L1D rotation. |
| imap_processing/tests/mag/test_mag_l2.py | Adds regression test ensuring L2 rotate_frame() preserves NaN/FILLVAL positions. |
| imap_processing/tests/mag/test_mag_l1d.py | Adds regression test ensuring L1D rotate_frame() preserves NaN/FILLVAL positions for both MAGO and MAGI vectors. |
| imap_processing/tests/mag/conftest.py | Updates the L2 norm_dataset vectors to floating dtype (but comment/dtype intent needs alignment). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if np.isnan(self.vectors).any() or (self.vectors == FILLVAL).any(): | ||
| new_vectors = np.where( | ||
| np.isnan(self.vectors) | (self.vectors == FILLVAL), | ||
| FILLVAL, | ||
| new_vectors, | ||
| ) |
There was a problem hiding this comment.
Same as above: this does redundant full-array scans and recomputes the isnan|==FILLVAL mask twice. Computing the mask once and reusing it (and potentially extracting a small helper to apply the mask for both MAGO and MAGI arrays) would reduce work on large datasets and avoid duplicated logic.
subagonsouth
left a comment
There was a problem hiding this comment.
A couple of thoughts to consider.
subagonsouth
left a comment
There was a problem hiding this comment.
Good to go from my perspective.
e2b9a69
into
IMAP-Science-Operations-Center:dev
Change Summary
This moves the change from #2901 into MAG-specific code.