Skip to content

Commit

Permalink
sqlp: fix read_csv optimization
Browse files Browse the repository at this point in the history
the if condition to check if there were no sqlp options passed was faulty.

This caused `sqlp` to ALWAYS use the slow path, rather than using the fast path read_csv optimization when no options were passed

Also removed the deeplink to polars-sql source code and just point to Polars SQL reference manual. The manual is maintained and shows the Polars SQL syntax, even though its targeted to py-polars users.
  • Loading branch information
jqnatividad committed Oct 18, 2024
1 parent c452d94 commit 803721e
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 8 deletions.
11 changes: 4 additions & 7 deletions src/cmd/sqlp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,8 @@ Polars SQL is a SQL dialect, converting SQL queries to fast Polars LazyFrame exp
(see https://docs.pola.rs/user-guide/sql/intro/).
For a list of SQL functions and keywords supported by Polars SQL, see
https://github.com/pola-rs/polars/blob/ee9bafbdef7d62baa06d469f42e6cec0755eb544/crates/polars-sql/src/functions.rs#L32 and
https://github.com/pola-rs/polars/blob/ee9bafbdef7d62baa06d469f42e6cec0755eb544/crates/polars-sql/src/keywords.rs.
https://docs.pola.rs/py-polars/html/reference/sql/index.html also provides a more readable
version of the SQL functions and keywords, though be aware that it's for the Python version
of Polars, so there will be some minor syntax differences.
https://docs.pola.rs/py-polars/html/reference/sql/index.html though be aware that it's for
the Python version of Polars, so there will be some minor syntax differences.
Returns the shape of the query result (number of rows, number of columns) to stderr.
Expand Down Expand Up @@ -679,12 +676,12 @@ pub fn run(argv: &[&str]) -> CliResult<()> {
&& delim == b','
&& !args.flag_no_optimizations
&& !args.flag_try_parsedates
&& args.flag_infer_len != 10_000
&& args.flag_infer_len == 10_000 // make sure this matches the usage text default
&& !args.flag_streaming
&& !args.flag_low_memory
&& !args.flag_truncate_ragged_lines
&& !args.flag_ignore_errors
&& args.flag_rnull_values.is_empty()
&& rnull_values == vec![PlSmallStr::EMPTY]
&& !args.flag_decimal_comma
&& comment_char.is_none()
&& std::path::Path::new(&args.arg_input[0])
Expand Down
3 changes: 2 additions & 1 deletion tests/test_sqlp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1532,7 +1532,7 @@ fn sqlp_length_fns() {
let expected = vec![
svec!["words", "n_chrs1", "n_chrs2", "n_chrs3", "n_bytes", "n_bits"],
svec!["Cafe", "4", "4", "4", "4", "32"],
svec!["", "", "", "", "", ""],
svec!["", "0", "0", "0", "0", "0"],
svec!["東京", "2", "2", "2", "6", "48"],
];

Expand Down Expand Up @@ -1666,6 +1666,7 @@ fn sqlp_string_replace() {
let expected = vec![
svec!["words"],
svec!["English breakfast tea is the best tea"],
svec![""],
];

assert_eq!(got, expected);
Expand Down

0 comments on commit 803721e

Please sign in to comment.