Support derived slot bindings and hide for intermediate computations

[amc-corey-cox]

Summary

Enable slot derivations to reference previously computed (derived) values, not just source fields, and implement the existing hide: true field to suppress intermediate slots from output.

Motivation

When a transformation needs multiple mapping stages — e.g., map two coded fields through lookup tables, then combine the results — there's no clean way to do it today:

• Throwaway intermediate slots produce unwanted output fields
• Inline case() expressions are unreadable for complex mappings

The hide field already exists on SlotDerivation in the spec but is not implemented. Combined with derived slot bindings, it enables a cascading pattern:

slot_derivations:
  _visit_label:
    hide: true
    populated_from: visit_code
    value_mappings:
      "1": "SCREENING"
      "7": "BASELINE"
  _site_label:
    hide: true
    populated_from: site_code
    value_mappings:
      "A": "Boston"
      "B": "Framingham"
  visit_site:
    expr: "slot('_visit_label') + ' at ' + slot('_site_label')"

Two pieces

1. Implement hide: true

Trivial — skip writing to tgt_attrs when slot_derivation.hide is True. Hidden slots still need to be computed.

2. Derived slot bindings via slot() function

Reference previously computed slot values using a slot() function in expressions, avoiding namespace collisions with source field names. Function names can't collide with data field names, unlike {_variable} syntax which could match a source column.

visit_site:
  expr: "slot('_visit_label') + ' at ' + slot('_site_label')"

Design considerations

• Ordering: Derived bindings imply that slot derivation evaluation order matters. Today slot_derivations is a dict (insertion-ordered in Python). This may need to be formalized.
• Reversibility: Chains of value_mappings with hide are more naturally reversible than complex case() expressions. The inverter could potentially reverse these with simple hints.
• Complexity: Derived bindings are a significant change to expression evaluation. This should be driven by concrete use cases, not implemented speculatively.

Related

• Support value_mappings as input to expr (mapping-then-expression pipeline) #160 — value_mappings with expr values (simpler, independent, should land first)
• Implement expression_to_expression_mappings in transformer runtime #127 — expression_to_expression_mappings discussion

[github-actions]

Issue Summary

Opened: 2026-03-25 by @amc-corey-cox
Status: Open — spun out from #160 on the same day

---

What this issue proposes

Two tightly coupled features that together enable a cascading slot derivation pattern, where intermediate computed values can feed into final expressions without polluting the output object:

1. Implement hide: true on SlotDerivation

The hide field already exists on SlotDerivation in the spec but is dead code — it is never checked in object_transformer.py. The fix is simple: when slot_derivation.hide is True, skip writing the derived value to tgt_attrs. Critically, hidden slots must still be computed so downstream derivations can reference them.

2. Derived slot bindings in expr

Currently, {name} placeholders in expr fields resolve against source object fields only. This issue proposes that they should also resolve against previously-computed derived slot values. This enables the cascading pattern:

slot_derivations:
  _visit_label:
    hide: true
    populated_from: visit_code
    value_mappings:
      "1": "SCREENING"
      "7": "BASELINE"
  _site_label:
    hide: true
    populated_from: site_code
    value_mappings:
      "A": "Boston"
      "B": "Framingham"
  visit_site:
    expr: "{_visit_label} + ' at ' + {_site_label}"

The _visit_label and _site_label slots are computed but suppressed from output; visit_site assembles the final value from them.

---

Design considerations called out in the issue

Concern	Detail
Ordering	Derived bindings create an implicit dependency order. slot_derivations is a dict (insertion-ordered in Python 3.7+). This behaviour may need to be formally documented or enforced.
Reversibility	A chain of value_mappings with hide: true intermediates is more mechanically reversible than a complex case() expression. The inverter could potentially handle these with simple hints — a meaningful advantage.
Complexity	Derived bindings are a non-trivial change to expression evaluation. The issue explicitly recommends being driven by concrete use cases (i.e., real BioData Catalyst / FHS harmonization work), not speculative implementation.

---

Implementation sketch

• hide: true — trivial: add if not slot_derivation.hide: guard around the tgt_attrs[…] = … assignment in map_object. Computed value still needs to be stored in a side dict.
• Derived bindings — the expression evaluation context (the Bindings dict passed to eval_expr) needs to accumulate already-computed slot values as derivations run. This changes the loop structure in map_object so it builds a running derived_bindings dict and merges it into the expression context for each subsequent derivation.

---

Related issues

Issue	Title	Status	Relationship
#160	Support value_mappings as input to expr	Open	Sibling/predecessor — this issue was spun out of #160's expanded scope discussion on 2026-03-25. #160 is now narrowed to: extend value_mappings values to support an expr field (literal key → expression result). Should land before #164.
#127	Implement expression_to_expression_mappings in transformer runtime	Open	Related — but the 2026-03-25 design discussion on #127 recommends removing expression_to_expression_mappings and expression_to_value_mappings from the spec before 1.0, in favour of extended value_mappings (#160) and case(). This issue is independent of that decision.

---

Suggested implementation order

1. Support value_mappings as input to expr (mapping-then-expression pipeline) #160 first — extend value_mappings values to support expr (self-contained, no architecture change)
2. Support derived slot bindings and hide for intermediate computations #164 second — implement hide: true (trivial) + derived slot bindings in expressions (more involved)
3. Implement expression_to_expression_mappings in transformer runtime #127 resolution — remove expression_to_expression_mappings / expression_to_value_mappings from the spec pre-1.0

---

Assessment

Well-motivated by a real production use case (BioData Catalyst harmonized variables, FHS visit/site identifier construction). The two pieces are clearly separated in the issue: hide is nearly free; derived bindings require real design work around evaluation order and expression context. The reversibility argument for this pattern vs. inline case() is a genuine advantage worth preserving in the design.

[amc-corey-cox]

Implementation note: slot() function via closure

slot('name') can be implemented as a closure over tgt_attrs in map_object, injected into the expression evaluator's function registry:

def _slot_lookup(name):
    return tgt_attrs.get(name)