Redesign Buscar

[axiomcura]

This PR introduces a redesign of the data processing workflow.

The clustering component of the pipeline has been removed, since EMD inherently accounts for the spread of the distribution, making explicit clustering unnecessary.

As a result, Buscar becomes significantly faster.

The retained modules are: signatures, check, and metric.py, which have also been reorganized accordingly.

Once this PR is merged, all downstream analyses will be re-run and documentation updates will come as separate PRs .

[Copilot]

Pull request overview

This PR implements a major redesign of the Buscar data processing workflow by removing the clustering component and reorganizing the codebase. The change is motivated by the observation that EMD (Earth Mover's Distance) inherently accounts for distribution spread, making explicit clustering unnecessary. This redesign significantly improves performance.

Changes:

• Removed clustering-related modules (refinement.py, heterogeneity.py, identify_hits.py) from utils/
• Reorganized core modules (signatures, metrics, checks) from utils/ to new buscar/ directory
• Updated measure_phenotypic_activity API to work without clustering, now comparing treatments directly to reference/target states

Reviewed changes

Copilot reviewed 13 out of 16 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
utils/refinement.py	Deleted - clustering refinement utilities no longer needed
utils/metrics.py	Deleted - replaced by redesigned buscar/metrics.py
utils/identify_hits.py	Deleted - compound hit identification removed with clustering
utils/heterogeneity.py	Deleted - clustering functionality removed from pipeline
buscar/signatures.py	New file - moved from utils/signatures.py with statistical tests for feature significance
buscar/metrics.py	New file - redesigned phenotypic activity measurement without clustering dependency
buscar/checks.py	New file - moved from utils/checks.py with data validation utilities
tests/test_hit_identifcation.py	Updated imports to reference buscar._identify_hits (non-existent module)
notebooks/4.cpjump1-analysis/nbconverted/4.run_buscar_rankings_base_on_moa.py	Updated imports with incorrect paths
notebooks/4.cpjump1-analysis/nbconverted/3.calculate-on-off-scores.py	Updated imports with incorrect paths
notebooks/1.compound-prioritization/nbconverted/4.measure-phenotypic-activity.py	Updated imports with incorrect paths
notebooks/1.compound-prioritization/nbconverted/1.signatures.py	Correctly updated import to buscar.signatures
notebooks/2.cfret-analysis/nbconverted/1.cfret-pilot-buscar-analysis.py	Updated imports with incorrect paths and obsolete heterogeneity reference
notebooks/4.cpjump1-analysis/nbconverted/1.generate-on-off-signatures.py	Correctly updated import to buscar.signatures
.pre-commit-config.yaml	Minor version update for ruff-pre-commit (v0.15.0 → v0.15.1)
.gitignore	Added .vscode/ directory to ignore list

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.