Skip to content

[Bug]: Managed transforms API may produce obscure errors during SnakeYAML processing #37460

@chamikaramj

Description

@chamikaramj

What happened?

Seems like if a config parameter passes an incorrectly formatted JavaBean object (may be by accident), a very non-actionable error like following will be raised.

Exception in thread "main" org.yaml.snakeyaml.error.YAMLException: No JavaBean properties found in com.google.common.collect.ImmutableMap$Builder
        at org.yaml.snakeyaml.introspector.PropertyUtils.getPropertiesMap(PropertyUtils.java:108)
        at org.yaml.snakeyaml.introspector.PropertyUtils.createPropertySet(PropertyUtils.java:135)
        at org.yaml.snakeyaml.introspector.PropertyUtils.getProperties(PropertyUtils.java:128)
        at org.yaml.snakeyaml.introspector.PropertyUtils.getProperties(PropertyUtils.java:121)
        at org.yaml.snakeyaml.representer.Representer.getProperties(Representer.java:258)
        at org.yaml.snakeyaml.representer.Representer$RepresentJavaBean.representData(Representer.java:73)
        at org.yaml.snakeyaml.representer.BaseRepresenter.representData(BaseRepresenter.java:122)
        at org.yaml.snakeyaml.representer.BaseRepresenter.representMapping(BaseRepresenter.java:174)
        at org.yaml.snakeyaml.representer.SafeRepresenter$RepresentMap.representData(SafeRepresenter.java:330)
        at org.yaml.snakeyaml.representer.BaseRepresenter.representData(BaseRepresenter.java:111)
        at org.yaml.snakeyaml.representer.BaseRepresenter.represent(BaseRepresenter.java:81)
        at org.yaml.snakeyaml.Yaml.dumpAll(Yaml.java:285)
        at org.yaml.snakeyaml.Yaml.dumpAs(Yaml.java:334)
        at org.yaml.snakeyaml.Yaml.dumpAsMap(Yaml.java:357)
        at org.apache.beam.sdk.schemas.utils.YamlUtils.yamlStringFromMap(YamlUtils.java:184)
        at org.apache.beam.sdk.managed.Managed$ManagedTransform.expand(Managed.java:235)
        at org.apache.beam.sdk.managed.Managed$ManagedTransform.expand(Managed.java:176)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:559)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:490)
        at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:352)
        at com.google.cloud.dataflow.integration.KafkaToIceberg.main(KafkaToIceberg.java:153)

We should update SnakeYAML processing to produce a better error message in this case.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Labels

    IcebergIOIcebergIO: can only be used through ManagedIOiojava

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions