Skip to content

[1893][eval] csvreader cleanup#1906

Open
mtar wants to merge 6 commits intoecmwf:developfrom
mtar:1893-csvreader-cleanup
Open

[1893][eval] csvreader cleanup#1906
mtar wants to merge 6 commits intoecmwf:developfrom
mtar:1893-csvreader-cleanup

Conversation

@mtar
Copy link
Contributor

@mtar mtar commented Feb 23, 2026

Description

This PR updates the docstrings in the CsvReader class. The original logic for handling empty data has been fixed.

Issue Number

Closes #1893

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

Comment on lines -267 to -290
def _rename_channels(data) -> pd.DataFrame:
"""
The scores downloaded from Quaver have a different convention. Need renaming.
Rename channel names to include underscore between letters and digits.
E.g., 'z500' -> 'z_500', 't850' -> 't_850', '2t' -> '2t', '10ff' -> '10ff'

Parameters
----------
name :
Original channel name.

Returns
-------
Dataset with renamed channel names.
"""
for name in list(data.index):
# If it starts with digits (surface vars like 2t, 10ff) → leave unchanged
if re.match(r"^\d", name):
continue

# Otherwise, insert underscore between letters and digits
data = data.rename(index={name: re.sub(r"([a-zA-Z])(\d+)", r"\1_\2", name)})

return data
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function still needed? No reference can be found.

@iluise iluise added eval anything related to the model evaluation pipeline labels Feb 25, 2026
@mtar mtar marked this pull request as ready for review February 25, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval anything related to the model evaluation pipeline

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Clean up CsvReader

2 participants