Update dependency org.jetbrains.kotlinx:dataframe to v0.15.0

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
org.jetbrains.kotlinx:dataframe	0.13.1 -> 0.15.0

---

Release Notes

Kotlin/dataframe (org.jetbrains.kotlinx:dataframe)

v0.15.0: v0.15: Experimental new CSV-, and Geographic integrations and many other fixes

This release contains several new features, tons of fixes and two new exciting experimental new integrations:

• Experimental new CSV parser based on Deephaven-CSV. See below for more information.
• Experimental new GeoDataFrame class for working with geographical data (from GeoJson/Shapefile) and plotting it with Kandy. See below for more information.
• Full BigInteger support:
  Just like we support the BigDecimal numbers, DataFrame now also supports BigInteger in parsing, converting, statistics, column arithmetics, etc.
• Custom SQL DataBase registration
• Improved parsing:
  Parsing and converting String columns to other types is now faster. We also introduce the new experimental ParserOptions.useFastDoubleParser setting which uses FastDoubleParser for faster and more flexible Double parsing.
• We continue improving our Compiler Plugin with every release. See below for more information.
• See this notebook for some more information about the changes.

New Experimental CSV integration

DataFrame's CSV parsing has been based on Apache Commons CSV from the beginning. While this has been sufficient for most applications, it had some issues like running out of memory, performance, and our API lacking in clarity, documentation, and completeness.

For DataFrame 0.15, we introduce a new separate package org.jetbrains.kotlinx:dataframe-csv which tries to solve all these issues at once. It's based on Deephaven-CSV which makes it faster and more memory efficient. And since we built it from the ground up, we made sure the API was complete, predictable, and documented carefully.

To try it yourself, explicitly add the dependency org.jetbrains.kotlinx:dataframe-csv to your project. In notebooks you can add enableExperimentalCsv=true to the %use-magic, like %use dataframe(enableExperimentalCsv=true).
Use the new DataFrame.readCsv()/DataFrame.readTsv()/DataFrame.readDelim() functions over the old DataFrame.readCSV() ones.

We happily await your feedback!

New Experimental Geo integration

Kandy v0.8 introduces geo-plotting which allows you to visualize geospatial/geographical data using the awesome Kandy DSL. To make working with this geographical data (from GeoJson/Shapefile) easier, we happily accepted the GeoDataFrame PR from the Kandy team.

To try it yourself, explicitly add the dependency org.jetbrains.kotlinx:dataframe-geo to your project or notebook (with the repository maven("https://repo.osgeo.org/repository/release")) and use GeoDataFrame.readGeoJson() or GeoDataFrame.readShapeFile() to get started!

Features

• New CSV implementation by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/903
• GeoDataFrame init by @​AndreiKingsley in https://github.com/Kotlin/dataframe/pull/909
• Change default flatten parent-child separator to "_" by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/920
• Split OpenAPI in module needed for user projects and module needed for code-generation by @​koperagen in https://github.com/Kotlin/dataframe/pull/916
• Support read unstructured excel file by @​khm0651 in https://github.com/Kotlin/dataframe/pull/901
• Fast double parser by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/935
• Implemented custom SQL DB registration by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/917
• Render FormattedFrame stored inside columns as HTML by @​koperagen in https://github.com/Kotlin/dataframe/pull/944
• Adding some missing converters by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/958
• Full BigInteger support by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/972

Compiler Plugin

• [Compiler plugin] Lower frontend generated implicit receivers by @​koperagen in https://github.com/Kotlin/dataframe/pull/869
• Generate valid code in transform(call) when interpret(call) fails by @​koperagen in https://github.com/Kotlin/dataframe/pull/907
• [Compiler plugin] Support dataFrameOf(Pair<String, List) by @​koperagen in https://github.com/Kotlin/dataframe/pull/908
• [Compiler plugin] Add a mechanism to handle function calls to stdlib that can appear as df api arguments by @​koperagen in https://github.com/Kotlin/dataframe/pull/914
• [Compiler plugin] Generate ColumnName annotations on frontend for all names that contain illegal characters by @​koperagen in https://github.com/Kotlin/dataframe/pull/913
• Revert insertGenericTreeImpl by @​koperagen in https://github.com/Kotlin/dataframe/pull/923
• [Compiler plugin] Propagate nullability in toDataFrame tree conversion by @​koperagen in https://github.com/Kotlin/dataframe/pull/942
• Add castTo(Function) overload for workflows that use compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/948
• [Compiler plugin] Setup call transformer pipeline to handle (...) -> DataRow functions by @​koperagen in https://github.com/Kotlin/dataframe/pull/918
• Compiler plugin read improvements by @​koperagen in https://github.com/Kotlin/dataframe/pull/949
• [Compiler plugin] Support valueCounts by @​koperagen in https://github.com/Kotlin/dataframe/pull/951

Fixes

• Adding contracts for Anycol.isValueColumn etc. for smart-casting by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/882
• Fix publish indexes docs by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/885
• Update algolia index builder by @​koperagen in https://github.com/Kotlin/dataframe/pull/895
• Find KSP Configurations that are Added Later by @​mgroth0 in https://github.com/Kotlin/dataframe/pull/881
• Partially inline AnyFrame typealias in return type position by @​koperagen in https://github.com/Kotlin/dataframe/pull/888
• Deprecating DataFrame.read("", delimiter =) by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/902
• Parsing improvements by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/874
• Fixed local classes being inferred as Any by changing visibility check by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/929
• Open File in readExcel in read-only mode by @​koperagen in https://github.com/Kotlin/dataframe/pull/931
• Adds the binary compatibility validator plugin by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/938
• Fixes nulls in framecols and improves column creation situation by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/925
• Specify slf4j-api instead of slf4j-simple. by @​erikogenvik in https://github.com/Kotlin/dataframe/pull/934
• [Important Fix!] Parse started to removed unselected columns by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/947
• Fixed error message for ColumnAccessor by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/953
• fix crs by @​AndreiKingsley in https://github.com/Kotlin/dataframe/pull/955
• Added inferNullability test for other databases by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/954
• Fix: disabled FastDoubleParser debug logs overload in the tests by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/956
• describe() fixes by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/937
• Fix IO closing & add new useful extensions by @​AndreiKingsley in https://github.com/Kotlin/dataframe/pull/960
• Remove dependency on fuel by @​koperagen in https://github.com/Kotlin/dataframe/pull/969
• [fix] Parser should be skipped after it fails to parse value by @​koperagen in https://github.com/Kotlin/dataframe/pull/975
• small fix for file clash at impl/io/readDelim.kt by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/976
• Fixed csv dependencies by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/977
• Bumped deprecations of startsWith and endsWith in CS DSL to Error by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/978
• Version bumps for 0.15 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/980

Docs and Examples

• READMEs by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/900
• Make it clear that dataframes are immutable by @​dmcg in https://github.com/Kotlin/dataframe/pull/924
• Add examples for rename by @​koperagen in https://github.com/Kotlin/dataframe/pull/952

New Contributors

• @​khm0651 made their first contribution in https://github.com/Kotlin/dataframe/pull/901
• @​erikogenvik made their first contribution in https://github.com/Kotlin/dataframe/pull/934
• @​dmcg made their first contribution in https://github.com/Kotlin/dataframe/pull/924
• @​AndreiKingsley made their first contribution in https://github.com/Kotlin/dataframe/pull/909

Full Changelog: Kotlin/dataframe@v0.14.2...v0.15.0

v0.14.2: v0.14: Kotlin 2.0 and many stability improvements

This release can mostly be described as a quality-of-life release. While there are not many new groundbreaking features at the moment, almost every part of the library has had some improvement. See the full list of changes below, but to highlight a few:

• We now officially support Kotlin 2.0+. The library is built with 2.0.20 now, so it will work with KSP 2.0.20 too.
• We've continued our work on the DataFrame Kotlin Compiler Plugin. While it is still experimental, it introduces an exciting new approach to working with your data in a zero-boilerplate, type safe way leveraging the amazing power the Kotlin 2.0 compiler gives us. See this demo project to experiment with it yourself.
• See this notebook for some of the small yet exciting features of the 0.14 release!

0.14.2

Includes the fix: https://github.com/Kotlin/dataframe/pull/934 which removes the slf4j-simple dependency, keeping just slf4j-api.

0.14.1

Includes the fix: https://github.com/Kotlin/dataframe/pull/872 which fixes compatibility with Kandy v0.7.1.

Features

• Compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/729
• added toDataFrame for float- and double iterables by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/631
• Allow any ArrowReader implementation to be use for reading Arrow data #​627 by @​fb64 in https://github.com/Kotlin/dataframe/pull/628
• add random parameter to shuffle by @​koperagen in https://github.com/Kotlin/dataframe/pull/643
• apply ksp to multiplatform configs in multiplatform modules by @​mgroth0 in https://github.com/Kotlin/dataframe/pull/647
• Add separator parameter to DataFrame.flatten by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/667
• POJO toDataFrame support (and array improvements) by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/650
• Add JDBC credentials extraction from env variables and improve exception handling by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/692
• Added MS SQL support for the dataframe-jdbc module by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/689
• Update SQL all table/schemas reading functions to return maps with table names by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/718
• Add a support for H2 modes by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/720
• Add delimiter parameter to readDelimStr by @​koperagen in https://github.com/Kotlin/dataframe/pull/743
• Add an option to read Excel cell values as a String regardless of their content type by @​koperagen in https://github.com/Kotlin/dataframe/pull/745
• Add castTo to help working with implicitly generated schemas in notebooks and plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/747
• Replace Klaxon with kotlinx-serialization by @​devcrocod in https://github.com/Kotlin/dataframe/pull/603
• Add df.convertTo(schemaFrom) overload by @​koperagen in https://github.com/Kotlin/dataframe/pull/764
• Add Convert.asFrame function by @​koperagen in https://github.com/Kotlin/dataframe/pull/781
• Add extension functions for the ResultSet by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/772

Work on the compiler plugin

• then operation in pivot column selection DSL inside aggregate by @​koperagen in https://github.com/Kotlin/dataframe/pull/617
• Compiler plugin fixes by @​koperagen in https://github.com/Kotlin/dataframe/pull/740
• Compiler plugin update by @​koperagen in https://github.com/Kotlin/dataframe/pull/755
• Adding utils to help ensure that compile time schema ~ runtime schema by @​koperagen in https://github.com/Kotlin/dataframe/pull/767
• Improve codegen for stdlib <-> df interop workflow by @​koperagen in https://github.com/Kotlin/dataframe/pull/763
• Add initial support for CS DSL in the compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/783
• Refactor toDataFrame implementation in compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/782
• [Compiler plugin] Avoid throwing debugging exceptions in user projects because of false positives by @​koperagen in https://github.com/Kotlin/dataframe/pull/788
• [Compiler plugin] silently abort interpretation in case of invariant errors by @​koperagen in https://github.com/Kotlin/dataframe/pull/812
• [Compiler plugin ] Support ColumnName annotation in extension properties codegen by @​koperagen in https://github.com/Kotlin/dataframe/pull/818
• Update compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/832

Fixes

• Added "debug mode" to catch type mismatches in columns during testing only by @​Jolanrensen in https://github.com/Kotlin/dataframe/issues/713
• Removing dangerous exception in ConvenienceSchemaGeneratorPlugin by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/655
• add gradle wrapper validation workflow by @​sullis in https://github.com/Kotlin/dataframe/pull/648
• Fix #​640: Jupyter integration conflicts with variable type converters from other integrations by @​ark-1 in https://github.com/Kotlin/dataframe/pull/641
• KDoc fixes for IntelliJ 2024.X by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/613
• Fix some things that break in the build with K2 enabled by @​koperagen in https://github.com/Kotlin/dataframe/pull/665
• Add nullability inference support to dataframe-jdbc by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/672
• Serialize BufferedImages as base64 by @​ermolenkodev in https://github.com/Kotlin/dataframe/pull/694
• Downsize test images to speed up unit tests execution by @​ermolenkodev in https://github.com/Kotlin/dataframe/pull/709
• KTNB-693 Send the full dataframe schema as metadata by @​cmelchior in https://github.com/Kotlin/dataframe/pull/706
• K2 preparation by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/708
• File and URL names are used in JDK / suggested in IDE auto-import. Rename our classes to avoid conflict by @​koperagen in https://github.com/Kotlin/dataframe/pull/725
• isComparable() fix for double describe() by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/726
• Fixes std() not inferring column types by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/728
• Pivot fix by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/735
• Enforce the project to be built with Java 11 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/736
• Rename private properties with names matching library classes by @​koperagen in https://github.com/Kotlin/dataframe/pull/737
• Add a test for valueCounts by @​koperagen in https://github.com/Kotlin/dataframe/pull/756
• Revert "Add a test for valueCounts" by @​koperagen in https://github.com/Kotlin/dataframe/pull/757
• Revert filter condition, fixes broken logic by @​koperagen in https://github.com/Kotlin/dataframe/pull/759
• Add a test for valueCounts by @​koperagen in https://github.com/Kotlin/dataframe/pull/758
• Add a notice about external urls by @​koperagen in https://github.com/Kotlin/dataframe/pull/774
• Make sortBy(ColumnReference) accept pathOf without extra cast by @​koperagen in https://github.com/Kotlin/dataframe/pull/779
• Update MarkersExtractor.kt by @​GeorgCantor in https://github.com/Kotlin/dataframe/pull/791
• Build config and Debug mode by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/796
• This fixes Nothing by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/795
• Small convertTo fix by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/800
• Small implode fix by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/801
• GroupBy aggregate fix by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/803
• Support for the serialization of DataframeConvertible values in ValueColumns has been added to enhance visualization in the KTNB plugin. by @​ermolenkodev in https://github.com/Kotlin/dataframe/pull/823
• Hide properties of intermediate objects and remove data class attributes by @​koperagen in https://github.com/Kotlin/dataframe/pull/828
• Fixes reading CSV files with BOM characters by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/831
• Improve dataframe sorting in KTNB UI by handling non-comparable columns by @​ermolenkodev in https://github.com/Kotlin/dataframe/pull/836
• Empty csv fix by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/835
• Added test suite for PostgreSQL local tests with covering case with URLs by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/798
• Enforcing kotlinx datetime bias by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/843
• allow applying ksp to custom configs by @​mgroth0 in https://github.com/Kotlin/dataframe/pull/842
• Fix duplicated jvm signature by @​koperagen in https://github.com/Kotlin/dataframe/pull/868
• Fixed flaky jsonWrite test by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/873
• Rerunning notebooks for 0.14.0-RC1 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/870
• Remove samplesTest task from kover report so "build" don't trigger it by @​koperagen in https://github.com/Kotlin/dataframe/pull/865
• Improved SQL<->JDBC mapping by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/855
• Add Language annotations for better IDE experience by @​koperagen in https://github.com/Kotlin/dataframe/pull/861

Docs and Examples

• Update docs and readme for 0.13.1 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/629
• Fixed a documentation for SQL database connection in Kotlin Notebooks by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/639
• Updated parse documentation by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/652
• Updated split documentation by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/653
• Updated mentions of DataFrame to represent objects by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/664
• Remove incomplete documentation topics by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/681
• Update comments and doc to provide more details about ResultSet by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/682
• Introduce OtherSamples.kt to generate tables for writerside docs outside Korro by @​koperagen in https://github.com/Kotlin/dataframe/pull/695
• Enable sitemap.xml by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/698
• Update README.md by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/714
• Column Selector Grammar in docs by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/619
• Column selector docs function descriptions by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/711
• KDocs GH actions by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/731
• Korro gh actions by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/824
• [KDocs] auto preprocess only when publishing by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/749
• google analytics script by @​azhebel in https://github.com/Kotlin/dataframe/pull/786
• Highlight work in progress in README by @​koperagen in https://github.com/Kotlin/dataframe/pull/789
• Small update to the linter information for the documentation website by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/810
• Adding KDoc preprocessing guide by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/806
• Changed documentation regarding extension properties api by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/651
• Update links to demos by @​koperagen in https://github.com/Kotlin/dataframe/pull/833
• Explain the use-case for toDataFrame(columnName) in docs by @​koperagen in https://github.com/Kotlin/dataframe/pull/837
• Add documentation for Kotlin Notebook Plugin usage and troubleshooting by @​ermolenkodev in https://github.com/Kotlin/dataframe/pull/839

Version Updates

• Dev version bump to 0.14.0 & updating deprecations by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/636
• update ksp to 1.9.23-1.0.20 by @​koperagen in https://github.com/Kotlin/dataframe/pull/654
• set docPreprocessor to 0.3.6 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/716
• Update Kotlin version for compiler plugin by @​koperagen in https://github.com/Kotlin/dataframe/pull/744
• Update kotlinDatetime version by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/753
• Replacing the linter by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/754
• Update to Kotlin 2.0 by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/792
• Update database URLs and dependencies by @​zaleslaw in https://github.com/Kotlin/dataframe/pull/802
• Gradle bump to 8.10 and housekeeping by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/846
• 0.14 version bumps by @​Jolanrensen in https://github.com/Kotlin/dataframe/pull/848

Known Issues

• Incompatibility with Kandy 0.7.0 for some functions requiring statBin. Resolved in Kandy 0.7.1 and DF 0.14.1.
• isOpenApiStr logger leaks to Gradle in https://github.com/Kotlin/dataframe/issues/785
• Compiler plugin is still experimental in https://github.com/Kotlin/dataframe/issues/704
• Jupyter importDataSchema() does not generate column accessors in https://github.com/Kotlin/dataframe/issues/696
• Describe breaks on Number column (and other statistics inconsistencies) in https://github.com/Kotlin/dataframe/issues/558
• CSV reading can be slow and needs some rework in https://github.com/Kotlin/dataframe/issues/827
• KDocs are lacking in https://github.com/Kotlin/dataframe/issues/811
• Many more, see issues and feel free to help us :)

New Contributors

• @​mgroth0 made their first contribution in https://github.com/Kotlin/dataframe/pull/647
• @​sullis made their first contribution in https://github.com/Kotlin/dataframe/pull/648
• @​ark-1 made their first contribution in https://github.com/Kotlin/dataframe/pull/641
• @​azhebel made their first contribution in https://github.com/Kotlin/dataframe/pull/786
• @​GeorgCantor made their first contribution in https://github.com/Kotlin/dataframe/pull/791

Full Changelog: Kotlin/dataframe@v0.13.1...v0.14.0

---

Configuration

📅 Schedule: Branch creation - "after 9am every weekday,before 5pm every weekday" in timezone Europe/London, Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.