Iceberg 版本 | Apache Iceberg 0.12.1 發(fā)布

Apache Iceberg 0.12.1 已于2021年11月8日發(fā)布。

Apache Iceberg 是一種用于大型分析數據集的開放式表格式。Iceberg 為包括 Spark、Trino、PrestoDB、Flink 和 Hive 在內的多種計算引擎提供高性能的 SQL 表格式。

Apache Kyuubi(Inubating) 基于 Apache Spark 為包括 Iceberg 在內的數據湖三劍客均提供了支持。

該版本提供的 spark3-runtime Jar 僅支持 Spark 3.0 和 3.1。下一次主要版本的發(fā)布將會帶來 Spark 3.2?的支持，也可以嘗試基于 master 分支代碼的每日構建版嘗鮮 Spark 3.2?的支持。

Apache Iceberg 0.12.1 包含以下主要的問題修復和改進：

#3264 fixes validation failures that occurred after snapshot expiration when writing Flink CDC streams to Iceberg tables.

#3264 fixes reading projected map columns from Parquet files written before Parquet 1.11.1.

#3195 allows validating that commits that produce row-level deltas don't conflict with concurrently added files. Ensures users can maintain serializable isolation for update and delete operations, including merge operations.

#3199 allows validating that commits that overwrite files don't conflict with concurrently added files. Ensures users can maintain serializable isolation for overwrite operations.

#3135 fixes equality-deletes using DATE, TIMESTAMP, and TIME types.

#3078 prevents the JDBC catalog from overwriting the jdbc.user property if any property called user exists in the environment.

#3035 fixes drop namespace calls with the DyanmoDB catalog.

#3273 fixes importing Avro files via add_files by correctly setting the number of records.

#3332 fixes importing ORC files with float or double columns in add_files.