rust snowflake: bulk ingest with the ADBC_INGEST_OPTION_MODE_CREATE_APPEND mode does not fail during schema updates

[Pranav2612000]

What happened?

• Using the rust adbc snowflake driver, bulk ingest some data with the schema

    let schema = Schema::new(vec![
        Field::new("id", DataType::Int32, false),
        Field::new("name", DataType::Utf8, false),
    ]);

• Update this schema to

    let extended_schema = Schema::new(vec![
        Field::new("id", DataType::Int32, false),
        Field::new("name", DataType::Utf8, false),
        Field::new("age", DataType::Int32, false),
    ]);

• bulk ingest more data into the same table, with ADBC_INGEST_OPTION_MODE set to ADBC_INGEST_OPTION_MODE_CREATE_APPEND

Expected behaviour

• The ingestion should fail with the ADBC_STATUS_ALREADY_EXISTS error

Observed behaviour

• Ingestion completes with partial data return, and no error is returned

Stack Trace

How can we reproduce the bug?

https://github.com/Pranav2612000/snowflake-adbc-test/blob/master/src/main.rs

Environment/Setup

adbc_core = "0.22.0"
adbc_driver_manager = "0.22.0"
arrow = { version = "57.0.0", features = ["prettyprint"] }
arrow-array = { version = "57.0.0", features = ["chrono-tz"] }
arrow-schema = "57.1.0"

[Pranav2612000]

Are there any obvious configuration changes that I'm missing causing this issue?

[Pranav2612000]

This seems to be the default Snowflake behaviour ( https://community.snowflake.com/s/article/How-to-block-uploads-if-the-schema-of-a-Parquet-file-and-the-schema-of-a-table-do-not-match )

As the community answer suggests, we may need to check the schema first and then proceed with appending the data if it matches. Is this something we want as part of the driver? I'll be happy to contribute.
( Not sure about the effect it may have on ingestion speeds )

[amoeba]

Hi @Pranav2612000 thanks for the issue and taking the time to create a reproduction.

I think your code running without error is expected as the driver is just matching Snowflake's behavior and I think you'd get the same behavior if you were using another Snowflake client. Due to that, I don't think adding an option to the driver is really warranted.

I'm going to close this issue. If you run into anything else with this driver, please feel free to file an issue at https://github.com/adbc-drivers/snowflake/issues.