qsv 0.136.0 #193278
Merged
qsv 0.136.0 #193278
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
rust
stefanb
approved these changes
Oct 8, 2024
|
🤖 An automated task has requested bottles to be published to this PR. |
github-actions
bot
added
the
CI-published-bottle-commits
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Created by
brew bumpCreated with
brew bump-formula-pr.release notes
tap the power of qsv to query, analyze, enrich, scrub and transform huge Excel files and multi-gigabyte CSV files in seconds, without having to deal with the command-line.
designed to make data publishing easier for portal operators and data stewards using the
allows you to build custom node-based flows and data pipelines using a visual interface.
features an ever-expanding library of reusable scripts for common data-wrangling use cases.
Natural Language Interface (RAG), Polars SQL query support, an API, Python/Luau support, automatic Data Dictionaries, DCAT 3 metadata profile inferencing, along with a retinue of other cloud-based services (e.g. customizable street-level geocoding, data feeds, reference data lookups, geo-ip lookups, cloud storage support,
.qsvfile format, etc.) that will be unveiled in future versions.Like qsv, we're iterating rapidly with qsv pro, so your feedback is essential. Give it a try!
Other highlights:
excel: new--tableoption for XLSX files; new--header-rowoption; expanded--rangeoption, adding support for Named Ranges and absolute ranges (e.g.Sheet2!$A$1:$J$10); and expanded metadata export now including Named Ranges and Tables (for XLSX files)apply,datefmt,tojsonlandvalidate) through automatic batch size optimizationvalidate:dynamicEnumcustom JSON Schema keyword in validate command (renamed fromdynenum) and enhanced email validationschema: automatic JSON Schemaconstinferencing for columns with just one valueAdded
apply,datefmt,tojsonl,validate: added logic to automatically determine optimal batch size for better parallelization Automatically determine optimal batch size dathere/qsv#2178enum: added--new-columnsupport for all enum modes, not just--increment2172 enum new column dathere/qsv#2173excel: new--tableoption for XLSX filesexcel: new--tableoption for XLSX files dathere/qsv#2194excel: new--header-rowoption dathere/qsv@458f79aexcel: expanded range and metadata optionsexcel: expanded range and metadata options dathere/qsv#2195schema: added JSON Schema automaticconstinferencingschema: addconstsupport; fixenumso it only adds a list when the number of unique values >--enum-thresholddathere/qsv#2180contrib(completions): add--tableoption toqsv excelby @rzmk incontrib(completions): add--tableoption toqsv exceldathere/qsv#2197completions: add--header-rowoption toqsv exceldathere/qsv@e8794d5apply operations sentimentbenchmark dathere/qsv@b745e64docs: added indexing section to PERFORMANCE.md dathere/qsv@804145aChanged
stats: various minor micro-optimizations dathere/qsv@62d95fc dathere/qsv@2c2862avalidate: renamed custom keyworddynenumtodynamicEnumto be more consistent with JSON schema naming conventions dathere/qsv@0.135.0...master#diff-9783631cdad9e1f47f60266303dc2d56a6e7a486784b61c40961601e8192f7cfvalidate: optimizations for increased performance; replace serde_json with simd_json dathere/qsv@0.135.0...master#diff-9783631cdad9e1f47f60266303dc2d56a6e7a486784b61c40961601e8192f7cfclippy::ref_optionlint to Config::new API apply newclippy::ref_optionlint to Config::new API dathere/qsv#2192deps: bumpcalaminefrom 0.25 to 0.26 dathere/qsv@b42279adeps:jsonschemause latest 0.22.3 upstream with unreleased features/fixesdeps:polarsuse latest 0.43.1 upstream with unreleased features/fixesdeps: created our own fork of unmaintained vader_sentiment crate dathere/qsv@b426761deps: useserde_jsonupstream with unreleased perf improvement/fixes https://github.com/jqnatividad/qsv/blob/1c1174b3b8b65d9dfd9c841597366fb09d0a047c/Cargo.toml#L221Fixed
schema: fixenumso it only adds a list when the number of unique values >--enum-thresholdschema: addconstsupport; fixenumso it only adds a list when the number of unique values >--enum-thresholddathere/qsv#2180Full Changelog: dathere/qsv@0.135.0...0.136.0