Missing linter for transformation specification files.

[paulmillar]

I think it would be helpful if there was a linter for the Transformation Specification contained in a yaml file.

Currently, the existing linter (linkml-lint) rejects a valid transformation specification. This makes sense, as a transformation specification isn't a valid linkml description.

[github-actions]

Summary / Triage Notes

Request: Add a dedicated linter/validator for linkml-map transformation specification YAML files (*.transform.yaml), since linkml-lint correctly rejects them (they are not LinkML schema files).

Current state of validation in the codebase

There is currently some implicit validation at load time, but no dedicated lint command:

1. load_specification() (src/linkml_map/utils/loaders.py) — calls _normalize_spec_dict() then Pydantic's TransformationSpecification(**obj). This catches structural errors at runtime but only when you actually try to run a transformation.
2. _normalize_spec_dict() / normalize_transform_spec() (src/linkml_map/transformer/transformer.py) — uses ReferenceValidator from linkml-runtime to expand/normalize the YAML before Pydantic parses it. This is primarily a normalization pass, not a diagnostic pass.
3. No linkml-map-lint CLI command exists. There is a linkml-map CLI (src/linkml_map/cli/cli.py) but no subcommand dedicated to linting/validating a spec file in isolation.

What a linter could check

A useful linter for transformation specs would validate things like:

• Structural validity: correct YAML structure matching the TransformationSpecification Pydantic model
• Reference integrity: populated_from names refer to slots/classes that exist in the source schema (requires schema context)
• Expression syntax: expr: fields contain parseable/evaluable expressions
• Completeness: required fields are present, derivations are non-trivially defined
• Cross-class reference consistency: dot-notation FK paths (e.g. org_id.name) resolve correctly — see Support cross-table slot lookup in class_derivation slot_derivations #134 for related cross-table lookup needs

Related issues

• Is specifying source and destination schemata supported? #138 (same reporter, @paulmillar) — Question about specifying separate source/target schemata in the transformation spec; relates to linting since a linter needs schema context to validate references
• Consolidate the spec normalization pipeline #124 — Proposal to consolidate the multi-layer spec normalization pipeline; a cleaner normalization pass would be a good foundation for a linting pass that produces user-friendly diagnostics rather than Pydantic exceptions
• Support cross-table slot lookup in class_derivation slot_derivations #134 — Cross-table slot lookup; a linter could help catch misconfigured cross-table references early

Suggested implementation approach

A minimal implementation could be a new CLI subcommand (e.g. linkml-map lint <spec.yaml>) that:

1. Loads the spec via load_specification() (catching Pydantic/YAML errors and presenting them as friendly diagnostics)
2. Optionally accepts --source-schema to perform reference validation against the source schema
3. Returns exit code 0 on success, non-zero on errors (useful for CI pipelines)

A more complete implementation could use the existing Transformer infrastructure (with induce_missing_values()) to validate cross-schema references.

@dragon-ai-agent please implement a linkml-map lint CLI subcommand that validates transformation specification YAML files. It should: (1) parse and validate the spec structure using the existing load_specification() / TransformationSpecification Pydantic model, reporting friendly errors rather than stack traces; (2) optionally accept a --source-schema flag and validate populated_from references against it; (3) exit with code 0 on success and non-zero on failure; (4) include doctests and pytest tests following the project conventions in CLAUDE.md.

[matentzn]

I know your issue is about schema instances, not the transformation schema itself, but I still think losely tying this issue to this discussion makes some sense: https://github.com/orgs/linkml/discussions/3256

But other than that, I 100% agree with your request! @amc-corey-cox let me know if I can be of assistance!

[amc-corey-cox]

Investigation Update: Validation is achievable with the existing schema

After hands-on investigation, the path to validating trans-specs is much shorter than expected. Here's what we found:

The schema already works

transformer_model.yaml is a valid LinkML schema that correctly describes trans-specs. The JSON schema generated from it validates real data — tested against all 7 dm-bip toy_data specs with zero errors when data uses the canonical format.

What validates today (no changes needed)

Data format	Validates?	Notes
Explicit list (class_derivations: [{name: Foo, ...}])	✅	Canonical LinkML serialization. Works now.
Dict-keyed (class_derivations: {Foo: {...}})	❌	Syntactic sugar. Schema expects array (inlined_as_list).
Compact-key list (- Foo: {populated_from: x})	❌	dm-bip compositor output. Needs preprocessing.

Pre-validation normalization

_preprocess_class_derivations (in transformer.py) already converts both dict-keyed and compact-key formats into the canonical explicit list format. Reusing this as a pre-validation step gives us backward-compatible validation for all existing trans-spec formats.

Blockers

1. linkml-validate CLI crashes on our schema due to a bug in JsonSchema.add_lax_def — it assumes all classes have a required field, but abstract classes (like ElementDerivation) don't. Filed upstream: linkml-validate crashes with KeyError in add_lax_def when schema has abstract classes linkml#3366. Workaround: Use JsonSchemaGenerator.serialize() + jsonschema directly (works fine, ~10 lines).

2. YAML type coercion — publication_date: 2025-08-14 parses as a datetime.date, version: 0.1 as a float. Either enforce quoting in specs or add a coercion step before validation.

Proposed implementation

A linkml-map validate-spec (or lint) CLI command that:

1. Loads raw YAML
2. Runs _preprocess_class_derivations to normalize sugar formats
3. Validates against the JSON schema generated from transformer_model.yaml
4. Optionally accepts --source-schema for deeper reference validation (as the triage summary suggests)

The structural validation piece (steps 1-3) is practically free — no schema changes, no model changes, just wiring up existing pieces.