Build: Bump datafusion to 52.3.0 and pyiceberg-core to 0.9.0

[kevinjqliu]

Rationale for this change

Upgrade deps

Prevent dependabot from auto update datafusion, we do it manually

Are these changes tested?

Are there any user-facing changes?

[Fokko]

Nice

[kevinjqliu]

this test failed when i ran it locally 🤔

tests/table/test_datafusion.py F                                                                                                            [100%]

==================================================================== FAILURES =====================================================================
____________________________________________________ test_datafusion_register_pyiceberg_table _____________________________________________________

catalog = default (<class 'pyiceberg.catalog.sql.SqlCatalog'>)
arrow_table_with_null = pyarrow.Table
bool: bool
string: large_string
string_long: large_string
int: int32
long: int64
float: float
double: do...
timestamptz: [[2023-01-01 19:25:00.000000Z,null,2023-03-01 19:25:00.000000Z]]
date: [[2023-01-01,null,2023-03-01]]
...

    def test_datafusion_register_pyiceberg_table(catalog: Catalog, arrow_table_with_null: pa.Table) -> None:
        catalog.create_namespace_if_not_exists("default")
        iceberg_table = catalog.create_table_if_not_exists(
            "default.dataset",
            schema=arrow_table_with_null.schema,
        )
        iceberg_table.append(arrow_table_with_null)
    
        ctx = SessionContext()
>       ctx.register_table("test", iceberg_table)

tests/table/test_datafusion.py:52: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = SessionContext: id=c02bce81-8a6a-4716-a5ad-159899b50529; configs=[
        datafusion.catalog.create_default_catalog_and_sche..._decimal = false
        datafusion.sql_parser.recursion_limit = 50
        datafusion.sql_parser.support_varchar_with_length = true]
name = 'test'
table = dataset(
  1: bool: optional boolean,
  2: string: optional string,
  3: string_long: optional string,
  4: int: optio...optional fixed[16]
),
partition by: [],
sort order: [],
snapshot: Operation.APPEND: id=6251928478140635918, schema_id=0

    def register_table(
        self,
        name: str,
        table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset,
    ) -> None:
        """Register a :py:class:`~datafusion.Table` with this context.
    
        The registered table can be referenced from SQL statements executed against
        this context.
    
        Args:
            name: Name of the resultant table.
            table: Any object that can be converted into a :class:`Table`.
        """
>       self.ctx.register_table(name, table)
E       ImportError: Incompatible libraries. DataFusion 52.0.0 introduced an incompatible signature change for table providers. Either downgrade DataFusion or upgrade your function library.

.venv/lib/python3.10/site-packages/datafusion/context.py:827: ImportError
============================================================= short test summary info =============================================================
FAILED tests/table/test_datafusion.py::test_datafusion_register_pyiceberg_table - ImportError: Incompatible libraries. DataFusion 52.0.0 introduced an incompatible signature change for table providers. Either downgrade DataF...
================================================================ 1 failed in 0.52s ================================================================

[kevinjqliu]

alsooooo CI check didnt trigger 🤔

[kevinjqliu]

Error
The action astral-sh/setup-uv@v7 is not allowed in apache/iceberg-python because all actions must be from a repository owned by your enterprise, created by GitHub, or match one of the patterns:

https://github.com/apache/iceberg-python/actions/runs/23358400193

[kevinjqliu]

oof, apache/infrastructure-actions#550