Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix reading gzipped file in Julia 1.11 on Windows #1144

Merged
merged 3 commits into from
Oct 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ jobs:
- os: ubuntu-latest
arch: x64
version: 'nightly'
- os: windows-latest
arch: x64
version: 'lts'

steps:
- run: git config --global core.autocrlf false
Expand Down
8 changes: 7 additions & 1 deletion src/file.jl
Original file line number Diff line number Diff line change
Expand Up @@ -336,8 +336,14 @@
ctx.debug && println("types after parsing: $types, pool = $(ctx.pool)")
# for windows, it's particularly finicky about throwing errors when you try to modify an mmapped file
# so we just want to make sure we finalize the input buffer so users don't run into surprises
# on Julia 1.11 the underlying memory needs to be finalized to unmmap the file.
# Ref: https://github.com/JuliaLang/julia/pull/54210
if !chunking && Sys.iswindows() && ctx.stringtype !== PosLenString
finalize(ctx.buf)
if VERSION ≥ v"1.11"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

finalize(ctx.buf.ref.mem)

Check warning on line 343 in src/file.jl

View check run for this annotation

Codecov / codecov/patch

src/file.jl#L343

Added line #L343 was not covered by tests
else
finalize(ctx.buf)
end
end
# check if a temp file was generated for parsing
if !chunking && ctx.tempfile !== nothing && ctx.stringtype !== PosLenString
Expand Down
17 changes: 2 additions & 15 deletions test/basics.jl
Original file line number Diff line number Diff line change
Expand Up @@ -378,21 +378,8 @@ f = CSV.File(IOBuffer("a,b,c\n1,2,3\n\n"))
f = CSV.File(IOBuffer("zip\n11111-1111\n"), dateformat = "y-m-dTH:M:S.s")
@test (length(f), length(f.names)) == (1, 1)

# Supporting commands across multiple platforms cribbed from julia/test/spawn.jl
catcmd = `cat`
havebb = false
if Sys.iswindows()
busybox = download("https://frippery.org/files/busybox/busybox.exe", joinpath(tempdir(), "busybox.exe"))
havebb = try # use busybox-w32 on windows, if available
success(`$busybox`)
true
catch
false
end
if havebb
catcmd = `$busybox cat`
end
end
# `cat` isn't always available on Windows
catcmd = `$(Base.julia_cmd()) --eval "write(stdout, open(ARGS[1]))"`
f = CSV.File(`$(catcmd) $(joinpath(dir, "test_basic.csv"))`)
@test columntable(f) == columntable(CSV.File(joinpath(dir, "test_basic.csv")))

Expand Down
Loading