Skip to content

msggen: Add UnionField support to generators for oneOf schemas #8964

@4xvgal

Description

@4xvgal

Problem

msggen has a UnionField class (model.py:369-400) designed to represent oneOf JSON schemas, but no generator in the pipeline can actually render it. As a result, every schema field that uses oneOf is manually overridden to a single type (usually ArrayField or PrimitiveField), silently dropping variants.

This means the typed APIs (cln-rpc, gRPC, Python) cannot express what the C implementation and JSON schema both support. For example, invoice.exposeprivatechannels accepts true, ["1x1x3"], or "1x1x3" in JSON-RPC, but cln-rpc only exposes Option<Vec<ShortChannelId>>.

Related: ElementsProject/lightning#8961

Root Cause

  1. UnionField.from_js() has a constructor bug — does not pass added/deprecated to __init__
  2. CompositeField.from_js() has no oneOf detection — top-level oneOf fields hit the "type" not in ftype" warning and are skipped
  3. No generator handles UnionField — all 6 generators (rust.py, proto.py, convert.py, unconvert.py, grpc2py.py, notification.py) only dispatch on PrimitiveField, ArrayField, EnumField, and CompositeField
  4. Traversal utilities skip UnionFieldpatch.py, checks.py, and both gather_subfields functions don't recurse into union variants

Because of (2)-(4), every oneOf field requires a manual override in model.py:537-560, and every override is lossy.

Affected Fields (18 schemas use oneOf)

Override Schema oneOf What's Lost
Invoice.exposeprivatechannels boolean | array[scid] | scid boolean and single scid variants
Invoice.label (×5 methods) string | integer integer variant
Datastore.key (×4 methods) array[string] | string single string variant
Offer.amount msat_or_any | currency currency variant
SetConfig.val string | integer | boolean integer and boolean variants
Pay.exclude array[scid_dir | pubkey] pubkey variant (array-item oneOf)

Fix

  1. model.py: Fix UnionField.from_js() constructor, add oneOf detection in CompositeField.from_js()
  2. patch.py, checks.py: Add UnionField traversal
  3. rust.py: Generate #[serde(untagged)] enums
  4. proto.py: Generate oneof blocks with wrapper messages for array variants
  5. convert.py / unconvert.py: Generate bidirectional From impls for union types
  6. grpc2py.py: Generate WhichOneof()-based conversion
  7. Remove 12 overrides that exist solely because oneOf was not handled

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions