When loading large amounts of data, polars is much faster. we can use polars, but for spotfire to ingest the data, we need to convert it to pandas from polars, which consumes memory and extra compute. Allowing spotfire to ingest polars dataframes directly would be a major performance boost.
i'm happy to take a stab at this issue.
When loading large amounts of data, polars is much faster. we can use polars, but for spotfire to ingest the data, we need to convert it to pandas from polars, which consumes memory and extra compute. Allowing spotfire to ingest polars dataframes directly would be a major performance boost.
i'm happy to take a stab at this issue.