FIXED_LEN_BYTE_ARRAY + DELTA_BYTE_ARRAY for Decimals

[4ertus2]

Arrow C++ reader had troubles with FLBA + DELTA_BYTE_ARRAY. There're issues in Decimal readings with such encodings in ClickHouse, DuckDB, Velox right now. It would be great if you make a sample of decimal columns with different precisions for the encoding combination.

[wgtmac]

Could you please elaborate the trouble in FLBA + DELTA_BYTE_ARRAY? How can we reproduce it?

[4ertus2]

Just make a parquet file with Decimal(19-38, x) with FLBA+DeltaByteArray encoding. And try to read it with non-arrow C++ reader (ClickHouse, DuckDB, Velox).

[wgtmac]

Thanks for the information! Have you tried to use https://github.com/apache/parquet-java/tree/master/parquet-cli to read it? Unfortunately, I don't have a configured setup for these non-arrow C++ reader (ClickHouse, DuckDB, Velox). It would be helpful to provide the error message or even stacktrace here to help understanding the issue.

[4ertus2]

CH ClickHouse/ClickHouse#62141 (comment)

DuckDB:

Invalid Error:
Delta Byte Array encoding is only supported for string/blob data

Velox:

Encoding not supported yet: DELTA_BYTE_ARRAY

[wgtmac]

Thanks. I thought it was a bug produced by the Parquet writer and now it is clear that it is a missing feature from those engines.