0.128.0

[0.128.0] - 2024-05-25

❤️ csv,conf,v8 Edition 🎉
🏇🏽 ¡Ándale! ¡Ándale! ¡Arriba! ¡Arriba! 💨

Yii-hah! We're Mexico bound as we head to csv,conf,v8 to present and share qsv with fellow data-makers and wranglers from all over!

And we've packed a lot into this release for the occasion:

• search got a lot of love as it now powers qsv pro's new search feature to get near-instant search results even on large datasets.
• stats - the ❤️ of qsv, now has several cache fine-tuning options with --cache-threshold. It now also computes max_precision for floats and is_ascii for strings. It also has a new --round 9999 sentinel value to suppress rounding of statistics.
• schema & tojsonl are now faster thanks to stats --cache-threshold autoindex & cache creation/deletion logic.
• We upgraded Polars to 0.40.0 to unlock additional capabilities in the count, joinp & sqlp commands.
• count now has an additional blazing fast counting mode using Polars' read_csv() table function.
• frequency gets some micro-optimizations for even faster frequency analysis.
• luau is now bundled with luau 0.625 from 0.622. We also upgraded the bundled LuaDate library from 2.2.0 to 2.2.1. All of this, while making it ~10% faster!

Overall, qsv manages to keep its performance edge despite the addition of new capabilities and features. We'll give a whirlwind tour of qsv and these updates in our talk at csv,conf,v8.

We'll also preview what we've been calling the People's APPI - our "Answering People/Policymaker Interface" in qsv pro.

This is a new way to interact with qsv that's more conversational and less command-line-y using a natural language interface. It's a way to make qsv more accessible to more people, especially those who are not comfortable with the command line.

We're excited to share all these qsv innovations with the csv,conf,v8 community and the wider world! Nos vemos en Puebla!

¡Ándele! ¡Ándele! ¡Epa! ¡Epa! ¡Epa!

---

Added

• count: additional Polars-powered counting mode using read_csv() SQL table function 05c5809
• input: add --quote-style option df3c8f1
• joinp: add --coalesce option 8d142e5
• search: add --preview-match option #1785
• search: add --json output option #1790
• search: add "match-only" --flag option mode #1799
• search: add --not-one flag for not using exit code 1 when no match by @rzmk in #1810
• sqlp: add --decimal-comma option #1832
• stats: add --cache-threshold option #1795
• stats: add --cache-threshold autoindex creation/deletion logic #1809
• stats: add additional mode to --cache-threshold 63fdc55
• stats: now computes max_precision for floats #1815
• stats: add --round 9999 sentinel value support to suppress rounding #1818
• stats: add is_ascii column #1824
• added new benchmarks for search command 58d73c3

Changed

• count: document three count modes 3d5a333
• describegpt: update --max-tokens type for LLMs with larger context sizes by @rzmk #1841
• excel: use simpler range::headers() to get headers 069acbf
• frequency: ensure --other-sorted works with --other-text 7430ad7
• frequency: microoptimize hot loop d9c01e1, 7c9f925 and
• luau: improve usage text cb6b4d9
• luau: we now bundle luau 0.625 from 0.622 4060975
• luau: update vendored LuaDate library from 2.2.0 to 2.2.1 #1840
• schema: adjust to reflect stats --cache-threshold option 92fed86
• slice: move json output helpers to util 1f44b48
• tojsonl: refactor boolcheck helper 74d5f5a
• docs: cross-reference split & partition commands #1828
• contrib(bashly): update completions.bash for qsv v0.127.0 by @rzmk in #1776
• contrib(bashly): update completions.bash for qsv v0.128.0 by @rzmk in #1838
• deps: upgrade to polars 0.40.0 #1831
• build(deps): bump actix-web from 4.5.1 to 4.6.0 by @dependabot in #1825
• build(deps): bump anyhow from 1.0.82 to 1.0.83 by @dependabot in #1798
• build(deps): bump anyhow from 1.0.83 to 1.0.85 by @dependabot in #1823
• build(deps): bump anyhow from 1.0.85 to 1.0.86 by @dependabot in #1826
• build(deps): bump cached from 0.50.0 to 0.51.0 by @dependabot in #1789
• build(deps): bump cached from 0.51.0 to 0.51.1 by @dependabot in #1793
• build(deps): bump cached from 0.51.1 to 0.51.2 by @dependabot in #1802
• build(deps): bump cached from 0.51.2 to 0.51.3 by @dependabot in #1805
• build(deps): bump crossbeam-channel from 0.5.12 to 0.5.13 by @dependabot in #1827
• build(deps): bump csvs_convert from 0.8.9 to 0.8.10 by @dependabot in #1808
• build(deps): bump data-encoding from 2.5.0 to 2.6.0 by @dependabot in #1780
• build(deps): bump file-format from 0.24.0 to 0.25.0 by @dependabot in #1807
• build(deps): bump flate2 from 1.0.28 to 1.0.29 by @dependabot in #1778
• build(deps): bump flate2 from 1.0.29 to 1.0.30 by @dependabot in #1784
• build(deps): bump hashbrown from 0.14.3 to 0.14.5 by @dependabot in #1781
• build(deps): bump itertools from 0.12.1 to 0.13.0 by @dependabot in #1822
• deps: bump forked jsonschema from 0.17.1 to 0.18.0 f02620f
• build(deps): bump mimalloc from 0.1.41 to 0.1.42 by @dependabot in #1829
• build(deps): bump mlua from 0.9.7 to 0.9.8 by @dependabot in #1821
• build(deps): bump qsv-stats from 0.16.0 to 0.17.1 by @dependabot in #1813
• build(deps): bump qsv-stats from 0.17.1 to 0.17.2 by @dependabot in #1814
• build(deps): bump qsv-stats from 0.17.2 to 0.18.0 by @dependabot in #1816
• build(deps): bump ryu from 1.0.17 to 1.0.18 by @dependabot in #1801
• build(deps): bump semver from 1.0.22 to 1.0.23 by @dependabot in #1800
• build(deps): bump serde from 1.0.198 to 1.0.199 by @dependabot in #1777
• build(deps): bump serde from 1.0.199 to 1.0.200 by @dependabot in #1787
• build(deps): bump serde from 1.0.200 to 1.0.201 by @dependabot in #1804
• build(deps): bump serde from 1.0.201 to 1.0.202 by @dependabot in #1817
• build(deps): bump serde_json from 1.0.116 to 1.0.117 by @dependabot in #1806
• build(deps): bump serial_test from 3.1.0 to 3.1.1 by @dependabot in #1779
• build(deps): bump simple-expand-tilde from 0.1.5 to 0.1.6 by @dependabot in #1811
• build(deps): bump sysinfo from 0.30.11 to 0.30.12 by @dependabot in #1797
• build(deps): bump titlecase from 3.0.0 to 3.1.1 by @dependabot in #1791
• build(deps): bump jql-runner from 7.1.8 to 7.1.9 by @dependabot in #1839
• apply select clippy suggestions
• updated several indirect dependencies
• pin Rust nightly to 2024-05-14
• bump MSRV to 1.78

Fixed

• luau: correct example when using --colindex cbbed21
• search: fix --json output #1792
• pass through docopt messages without a prefix #1835
• apply Polars SQL count(*) group by fix #1837

Full Changelog: 0.127.0...0.128.0