Add read_files function to SDK

[sminot]

Bioinformaticians frequently embed metadata in file paths. They will likely find it useful to be able to quickly identify files which match a particular naming pattern and return both the file contents and also the file-path-embedded metadata.

Summary

Adds read_files generator methods to DataPortalDataset and DataPortalProject for reading and
parsing dataset files directly into Python objects.

  DataPortalDataset.read_files(glob, pattern, file_format, **kwargs)                              

Iterates over files in the dataset and yields parsed content for each match. Exactly one of glob
or pattern must be provided.

glob mode — standard wildcard matching:

• • matches within a single path segment; ** matches across segments
• Suffix-anchored, so *.csv matches at any depth
• Yields parsed file content per match (e.g., a DataFrame, str, etc.)

pattern mode — like glob but with {name} capture placeholders:

• {name} captures one path segment (no /)
• Yields (content, captures) tuples where captures is a dict of values extracted from the path
• Useful for pulling metadata (sample names, conditions, etc.) directly out of file paths

The file_format argument controls parsing ('csv', 'json', 'h5ad', 'parquet', 'feather',
'pickle', 'excel', 'text'); omitting it infers the format from the file extension. **kwargs are
forwarded to the underlying parsing function (e.g., sep='\t' for TSV files).

  DataPortalProject.read_files(dataset, glob, pattern, file_format, **kwargs)                     

A thin convenience wrapper around DataPortalDataset.read_files. Resolves the dataset argument —
which may be a name string, ID string, or DataPortalDataset object — then delegates to the
dataset method. The glob/pattern behavior and yield types are identical.

Usage examples

  # Read all CSV files from a dataset — yields DataFrames
  for df in dataset.read_files(glob='*.csv'):
      print(df.shape)                                                                             
                                                                                                  
  # Extract sample names from filenames — yields (DataFrame, captures) tuples                     
  for df, captures in dataset.read_files(pattern='{sample}.csv'):                                 
      print(captures['sample'], df.shape)                                                         
                                                            
  # Multi-level capture: condition directory + sample filename                                    
  for df, captures in dataset.read_files(pattern='{condition}/{sample}.csv'):
      print(captures['condition'], captures['sample'], df.shape)                                  
                                                            
  # Read gzip-compressed TSV files at any depth                                                   
  for df in dataset.read_files(glob='**/*.tsv.gz', file_format='csv', sep='\t'):
      print(df.shape)                                                                             
                                                                                                  
  # Same, but resolve the dataset by name from a project                                          
  for df in project.read_files('My Dataset', glob='*.csv'):                                       
      print(df.shape)