windows build #115

amos-lsl · 2023-08-20T13:55:37Z

Is there any config or guidance for compiling on windows?

marcboeker · 2023-08-21T13:56:30Z

Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file.
You could try something like:

Download the latest release from DuckDB for Windows and run the following:

CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main

amos-lsl · 2023-08-23T16:13:33Z

Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file. You could try something like:

Download the latest release from DuckDB for Windows and run the following:
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main

i have try this, but run error, like others who tried in windows

Dimirolz · 2023-10-30T12:46:53Z

Windows seems to be problematic. I've hadn't had success yet. Maybe you could try dynamic linking instead of compiling the amalgamation file. You could try something like:

Download the latest release from DuckDB for Windows and run the following:
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/duckdb.dll" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main

Dynamic linking is working for my, btw.

marcboeker · 2023-10-30T16:28:59Z

I've done extensive work on getting it to link statically on Windows, but had no success. The main problem is, that CGO complains about missing __imp_ prefixed symbols.

undefined reference to `__imp_.....`

They are available in the dll but not in the lib which is btw. implausibly small (~200kb).

ZenLiuCN · 2023-12-17T12:16:03Z

This seems a solution, I mentioned under #51 . Current a success workflow is:

compile duckdb source as static library (tried with Mingw64 v13.2 posix msvcrt )
copy all output libraries to a new folder deps/windows_amd64
- libduckdb_static.a
- libparquet_extension.a
- libduckdb_utf8proc.a
- libduckdb_miniz.a
- libduckdb_hyperloglog.a
- libduckdb_re2.a
- libduckdb_fmt.a
- libduckdb_pg_query.a
- libduckdb_mbedtls.a
- libduckdb_fastpforlib.a
- libduckdb_fsst.a

create a file vendor.go

// Package windows_amd64 is required to provide support for vendoring modules
// DO NOT REMOVE
package windows_amd64

create a file under source root cgo_static_win.go, cause it have many different that linux or darwin

//go:build !duckdb_use_lib && windows && amd64

package duckdb

/*
#cgo LDFLAGS: -lduckdb_static -lws2_32 -lduckdb_fsst -lduckdb_fmt -lduckdb_pg_query -lduckdb_re2 -lduckdb_miniz -lduckdb_utf8proc -lduckdb_hyperloglog -lduckdb_fastpforlib -lduckdb_mbedtls -lparquet_extension -lduckdb_static -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread -lm -L${SRCDIR}/deps/windows_amd64
#include <duckdb.h>
*/
import "C"

Most important changs on duckdb.h at line 18

//===----------------------------------------------------------------------===//
//
//                         DuckDB
//
// duckdb.h
//
//
//===----------------------------------------------------------------------===//

#pragma once

// duplicate of duckdb/main/winapi.hpp
#ifndef DUCKDB_API
#ifdef _WIN32
#if defined(DUCKDB_BUILD_LIBRARY) && !defined(DUCKDB_BUILD_LOADABLE_EXTENSION)
#define DUCKDB_API __declspec(dllexport)
#else
// line 18 is patched
#define DUCKDB_API //__declspec(dllimport) !! comment out this part
#endif
#else
#define DUCKDB_API
#endif
#endif

Notes:
1. ws2_32 is required by libparquet_extension
2. Those seperated static libraries's order are important
3. -lparquet_extension -lduckdb_static cause that parquet_extension also requires duckdb_static
4. Use -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread to avoid ship with extra dll (libstdc++-6.dll, libwinpthread-1.dll, libgcc_s_seh-1.dll)
5. It will plus about 99MB to the finally binary.

Also I tested the example (within the engine ), seems that all functional.

2023/12/17 20:09:17 INFO [{"current_setting('access_mode')":"read_write"}]
2023/12/17 20:09:17 INFO {"lastInsertId":0,"rowsAffected":0}
2023/12/17 20:09:17 INFO {"lastInsertId":0,"rowsAffected":1}
2023/12/17 20:09:17 INFO {"rowsAffected":1,"lastInsertId":0}
2023/12/17 20:09:17 INFO [{"age":99,"height":1.909999966621399,"awesome":true,"bday":"1970-01-01T00:00:00Z","name":"marc"},{"height":1.850000023841858,"awesome":true,"bday":"1951-01-23T00:00:00Z","name":"macgyver","age":70}]

See Those changes

ZenLiuCN · 2023-12-18T11:17:47Z

Test Results:

built in extensions almost ok (maybe not found yet)
external extensions have to compile manually ( official extension seems compile by msvc)

marcboeker · 2023-12-23T17:01:30Z

@ZenLiuCN Thanks for the instructions!
Short update: I'm working on integrating your suggestions into the build process. See #140
There are still some problems, I need to figure out.

marcboeker · 2023-12-23T18:56:25Z

@ZenLiuCN I've tried to integrate the go-duckdb windows deps from your engine repo but I'm getting

go test -v -race -count=1 .
# github.com/marcboeker/go-duckdb.test
C:\Program Files\Go\pkg\tool\windows_amd64\link.exe: running gcc failed: exit status 1
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(logical_types-c.cpp.obj):logical_types-c.cpp:(.text+0xbfe): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(logical_types-c.cpp.obj):logical_types-c.cpp:(.text+0x1187): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Marc/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(prepared-c.cpp.obj):prepared-c.cpp:(.text$_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb5ValueEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy[_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb5ValueEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy]+0x108): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0xec0): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0xf01): undefined reference to `std::__throw_bad_array_new_length()'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/_/Desktop/go-duckdb/deps/windows_amd64/libduckdb_static.a(result-c.cpp.obj):result-c.cpp:(.text+0x1460): more undefined references to `std::__throw_bad_array_new_length()' follow
collect2.exe: error: ld returned 1 exit status

deps/windows_amd64 contains the following:

Directory: C:\Users\Dev\Desktop\go-duckdb\deps\windows_amd64

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          22/12/2023    16:29            490 build.sh
-a---          22/12/2023    16:29             62 copyLib.sh
-a---          22/12/2023    16:29         171252 libautocomplete_extension.a
-a---          22/12/2023    16:29         219416 libduckdb_fastpforlib.a    
-a---          22/12/2023    16:29         655098 libduckdb_fmt.a
-a---          22/12/2023    16:29          45386 libduckdb_fsst.a
-a---          22/12/2023    16:29          40218 libduckdb_hyperloglog.a    
-a---          22/12/2023    16:29         136960 libduckdb_mbedtls.a        
-a---          22/12/2023    16:29         105992 libduckdb_miniz.a
-a---          22/12/2023    16:29         494436 libduckdb_pg_query.a       
-a---          22/12/2023    16:29         821462 libduckdb_re2.a
-a---          22/12/2023    16:29       81849598 libduckdb_static.a
-a---          22/12/2023    16:29         311942 libduckdb_utf8proc.a       
-a---          22/12/2023    16:29         459490 libexcel_extension.a       
-a---          22/12/2023    16:29        1047314 libfts_extension.a
-a---          22/12/2023    16:29       19421390 libicu_extension.a
-a---          22/12/2023    16:29          97340 libinet_extension.a        
-a---          22/12/2023    16:29        2705052 libjson_extension.a        
-a---          22/12/2023    16:29        4036218 libparquet_extension.a     
-a---          22/12/2023    16:29        1759702 libsqlsmith_extension.a    
-a---          22/12/2023    16:29        2593182 libtpcds_extension.a       
-a---          22/12/2023    16:29        1186888 libtpch_extension.a        
-a---          22/12/2023    16:29            199 mergeStaticLib.sh
-a---          22/12/2023    16:29            872 vendor.go

And the duckdb.h is also modified by removing __declspec(dllimport).

Do you have any idea why there are still missing references? Do the tests work for you? Even running make test inside engine/sqlx/duckdb/go-duckdb yields the same errors as above.

ZenLiuCN · 2023-12-24T04:04:48Z

I was use mingw64_posix_seh. Does it the same? PS use msvc maybe a better option, I can't get mysql fdw to compiled by mingw.

ZenLiuCN · 2023-12-24T04:24:57Z

@marcboeker 13.2.0 is the gcc version which I used. I saw that you was using 10.3.0. It may the difference.
Notes:

Any loadable extensions are not provided by offical extension repository
MySQL-FDW extension not success to port to mingw
Postgres extension is fine (without using vcpkg)
Spartial extension haven't try yet
The Test result with go test -v -race -count=1 .

=== RUN   TestAppender
--- PASS: TestAppender (0.68s)
=== RUN   TestOpen
=== PAUSE TestOpen
=== RUN   TestConnPool
--- PASS: TestConnPool (0.01s)
=== RUN   TestConnInit
--- PASS: TestConnInit (7.10s)
=== RUN   TestExec
=== PAUSE TestExec
=== RUN   TestQuery
=== PAUSE TestQuery
=== RUN   TestStruct
=== PAUSE TestStruct
=== RUN   TestMap
=== PAUSE TestMap
=== RUN   TestList
=== PAUSE TestList
=== RUN   TestDecimal
=== PAUSE TestDecimal
=== RUN   TestUUID
=== PAUSE TestUUID
=== RUN   TestENUMs
=== PAUSE TestENUMs
=== RUN   TestHugeInt
=== PAUSE TestHugeInt
=== RUN   TestVarchar
=== PAUSE TestVarchar
=== RUN   TestBlob
=== PAUSE TestBlob
=== RUN   TestBoolean
=== PAUSE TestBoolean
=== RUN   TestJSON
=== PAUSE TestJSON
=== RUN   TestDate
=== PAUSE TestDate
=== RUN   TestTimestamp
=== PAUSE TestTimestamp
=== RUN   TestInterval
=== PAUSE TestInterval
=== RUN   TestEmpty
=== PAUSE TestEmpty
=== RUN   TestTypeNamesAndScanTypes
=== RUN   TestTypeNamesAndScanTypes/BOOLEAN
=== RUN   TestTypeNamesAndScanTypes/TINYINT
=== RUN   TestTypeNamesAndScanTypes/SMALLINT
=== RUN   TestTypeNamesAndScanTypes/INTEGER
=== RUN   TestTypeNamesAndScanTypes/BIGINT
=== RUN   TestTypeNamesAndScanTypes/UTINYINT
=== RUN   TestTypeNamesAndScanTypes/USMALLINT
=== RUN   TestTypeNamesAndScanTypes/UINTEGER
=== RUN   TestTypeNamesAndScanTypes/UBIGINT
=== RUN   TestTypeNamesAndScanTypes/FLOAT
=== RUN   TestTypeNamesAndScanTypes/DOUBLE
=== RUN   TestTypeNamesAndScanTypes/TIMESTAMP
=== RUN   TestTypeNamesAndScanTypes/DATE
=== RUN   TestTypeNamesAndScanTypes/TIME
=== RUN   TestTypeNamesAndScanTypes/INTERVAL
=== RUN   TestTypeNamesAndScanTypes/HUGEINT
=== RUN   TestTypeNamesAndScanTypes/VARCHAR
=== RUN   TestTypeNamesAndScanTypes/BLOB
=== RUN   TestTypeNamesAndScanTypes/DECIMAL(30,17)
=== RUN   TestTypeNamesAndScanTypes/TIMESTAMP_S
=== RUN   TestTypeNamesAndScanTypes/TIMESTAMP_MS
=== RUN   TestTypeNamesAndScanTypes/TIMESTAMP_NS
=== RUN   TestTypeNamesAndScanTypes/VARCHAR[][]
=== RUN   TestTypeNamesAndScanTypes/STRUCT("key1"_VARCHAR,_"key2"_INTEGER,_"key3"_DOUBLE)
=== RUN   TestTypeNamesAndScanTypes/STRUCT("key1_(,)_\_\""_""""_"""_INTEGER)
=== RUN   TestTypeNamesAndScanTypes/MAP(INTEGER,_VARCHAR)
=== RUN   TestTypeNamesAndScanTypes/UUID
--- PASS: TestTypeNamesAndScanTypes (0.02s)
   --- PASS: TestTypeNamesAndScanTypes/BOOLEAN (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TINYINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/SMALLINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/INTEGER (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/BIGINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/UTINYINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/USMALLINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/UINTEGER (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/UBIGINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/FLOAT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/DOUBLE (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TIMESTAMP (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/DATE (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TIME (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/INTERVAL (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/HUGEINT (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/VARCHAR (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/BLOB (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/DECIMAL(30,17) (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_S (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_MS (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/TIMESTAMP_NS (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/VARCHAR[][] (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/STRUCT("key1"_VARCHAR,_"key2"_INTEGER,_"key3"_DOUBLE) (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/STRUCT("key1_(,)_\_\""_""""_"""_INTEGER) (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/MAP(INTEGER,_VARCHAR) (0.00s)
   --- PASS: TestTypeNamesAndScanTypes/UUID (0.00s)
=== RUN   TestMultipleStatements
--- PASS: TestMultipleStatements (0.01s)
=== RUN   TestParquetExtension
--- PASS: TestParquetExtension (0.05s)
=== RUN   TestPrepareQuery
--- PASS: TestPrepareQuery (0.01s)
=== RUN   TestPrepareWithError
--- PASS: TestPrepareWithError (0.02s)
=== CONT  TestOpen
=== CONT  TestHugeInt
=== CONT  TestBoolean
=== CONT  TestBlob
=== CONT  TestVarchar
=== CONT  TestJSON
=== CONT  TestTimestamp
=== CONT  TestInterval
=== RUN   TestOpen/without_config
=== RUN   TestBlob/select_as_string
=== RUN   TestInterval/bind_interval
=== RUN   TestBlob/select_as_hex
=== RUN   TestVarchar/size_is_smaller_than_12
=== RUN   TestBoolean/scan
=== RUN   TestInterval/scan_interval
=== RUN   TestTimestamp/epoch
=== RUN   TestInterval/scan_interval/simple_interval
=== RUN   TestVarchar/size_is_exactly_12
=== RUN   TestHugeInt/scan_hugeint
=== RUN   TestHugeInt/scan_hugeint/0
=== RUN   TestTimestamp/before_1970
=== RUN   TestInterval/scan_interval/interval_arithmetic
=== RUN   TestVarchar/size_is_13
=== RUN   TestBoolean/bind
=== RUN   TestHugeInt/scan_hugeint/1
=== RUN   TestTimestamp/after_1970
=== RUN   TestVarchar/size_is_greater_than_12
=== RUN   TestInterval/scan_interval/timestamp_arithmetic
=== RUN   TestJSON/select_empty_JSON
=== RUN   TestHugeInt/scan_hugeint/-1
=== RUN   TestTimestamp/HH:MM:SS
=== RUN   TestJSON/select_from_marshalled_JSON
=== RUN   TestTimestamp/HH:MM:SS.DDDD
=== RUN   TestHugeInt/scan_hugeint/9223372036854775807
=== RUN   TestHugeInt/scan_hugeint/-9223372036854775808
=== RUN   TestHugeInt/scan_hugeint/170141183460469231731687303715884105727
=== RUN   TestJSON/select_JSON_array
=== RUN   TestHugeInt/scan_hugeint/-170141183460469231731687303715884105727
=== RUN   TestHugeInt/bind_hugeint
--- PASS: TestBoolean (0.01s)
   --- PASS: TestBoolean/scan (0.00s)
   --- PASS: TestBoolean/bind (0.00s)
=== CONT  TestEmpty
--- PASS: TestVarchar (0.02s)
   --- PASS: TestVarchar/size_is_smaller_than_12 (0.00s)
   --- PASS: TestVarchar/size_is_exactly_12 (0.00s)
   --- PASS: TestVarchar/size_is_13 (0.00s)
   --- PASS: TestVarchar/size_is_greater_than_12 (0.00s)
=== CONT  TestDate
--- PASS: TestInterval (0.02s)
   --- PASS: TestInterval/bind_interval (0.00s)
   --- PASS: TestInterval/scan_interval (0.00s)
       --- PASS: TestInterval/scan_interval/simple_interval (0.00s)
       --- PASS: TestInterval/scan_interval/interval_arithmetic (0.00s)
       --- PASS: TestInterval/scan_interval/timestamp_arithmetic (0.00s)
=== CONT  TestUUID
=== RUN   TestOpen/with_config
--- PASS: TestBlob (0.02s)
   --- PASS: TestBlob/select_as_string (0.00s)
   --- PASS: TestBlob/select_as_hex (0.00s)
=== CONT  TestQuery
--- PASS: TestTimestamp (0.02s)
   --- PASS: TestTimestamp/epoch (0.00s)
   --- PASS: TestTimestamp/before_1970 (0.00s)
   --- PASS: TestTimestamp/after_1970 (0.00s)
   --- PASS: TestTimestamp/HH:MM:SS (0.00s)
   --- PASS: TestTimestamp/HH:MM:SS.DDDD (0.00s)
=== CONT  TestENUMs
--- PASS: TestJSON (0.02s)
   --- PASS: TestJSON/select_empty_JSON (0.00s)
   --- PASS: TestJSON/select_from_marshalled_JSON (0.00s)
   --- PASS: TestJSON/select_JSON_array (0.00s)
=== CONT  TestDecimal
--- PASS: TestHugeInt (0.02s)
   --- PASS: TestHugeInt/scan_hugeint (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/0 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/1 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/-1 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/9223372036854775807 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/-9223372036854775808 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/170141183460469231731687303715884105727 (0.00s)
       --- PASS: TestHugeInt/scan_hugeint/-170141183460469231731687303715884105727 (0.00s)
   --- PASS: TestHugeInt/bind_hugeint (0.00s)
=== CONT  TestStruct
=== RUN   TestDate/before_1970
=== RUN   TestDate/after_1970
=== RUN   TestDate/epoch
=== RUN   TestQuery/simple
--- PASS: TestEmpty (0.01s)
=== RUN   TestDecimal/decimal_widths
=== CONT  TestList
=== RUN   TestQuery/large_number_of_rows
--- PASS: TestDate (0.01s)
   --- PASS: TestDate/before_1970 (0.00s)
   --- PASS: TestDate/after_1970 (0.00s)
   --- PASS: TestDate/epoch (0.00s)
=== CONT  TestExec
=== RUN   TestOpen/with_invalid_config
=== RUN   TestStruct/scan_single_struct
--- PASS: TestOpen (0.03s)
   --- PASS: TestOpen/without_config (0.02s)
   --- PASS: TestOpen/with_config (0.01s)
   --- PASS: TestOpen/with_invalid_config (0.00s)
=== CONT  TestMap
=== RUN   TestStruct/scan_slice_of_structs
--- PASS: TestUUID (0.02s)
--- PASS: TestENUMs (0.02s)
=== RUN   TestList/integer_list
--- PASS: TestStruct (0.01s)
   --- PASS: TestStruct/scan_single_struct (0.00s)
   --- PASS: TestStruct/scan_slice_of_structs (0.00s)
=== RUN   TestMap/select_map
=== RUN   TestMap/select_nested_map
--- PASS: TestExec (0.01s)
=== RUN   TestMap/insert_map
=== RUN   TestDecimal/multiple_decimal_types
--- PASS: TestMap (0.01s)
   --- PASS: TestMap/select_map (0.00s)
   --- PASS: TestMap/select_nested_map (0.00s)
   --- PASS: TestMap/insert_map (0.00s)
--- PASS: TestList (0.02s)
   --- PASS: TestList/integer_list (0.01s)
=== RUN   TestDecimal/huge_decimal
=== RUN   TestDecimal/decimal_to_float64
--- PASS: TestDecimal (0.03s)
   --- PASS: TestDecimal/decimal_widths (0.01s)
   --- PASS: TestDecimal/multiple_decimal_types (0.00s)
   --- PASS: TestDecimal/huge_decimal (0.00s)
   --- PASS: TestDecimal/decimal_to_float64 (0.00s)
=== RUN   TestQuery/wrong_syntax
=== RUN   TestQuery/missing_parameter
=== RUN   TestQuery/null
--- PASS: TestQuery (0.61s)
   --- PASS: TestQuery/simple (0.00s)
   --- PASS: TestQuery/large_number_of_rows (0.60s)
   --- PASS: TestQuery/wrong_syntax (0.00s)
   --- PASS: TestQuery/missing_parameter (0.00s)
   --- PASS: TestQuery/null (0.00s)
PASS
ok  	github.com/marcboeker/go-duckdb	9.835s

the test for sqlite at duckdb_test.go:45 comment out cause I did not compile this extension.

ZenLiuCN · 2023-12-24T04:47:55Z

I suggest that maybe not merge the windows static link with go-duckdb for using mingw as compile now.

No offical extension support (only windows_amd64 not with windows_amd64_mingw), Those extensions can not run with mingw compiled duckdb
MSVC maybe a better option, but we known that it ship with VisualStduio and must have a license with even commuity version.
Can just left it as a extend mode or just a document, If this porject want to offer an opportunity for windows users, cause it requires some mannully operate to be done. Eg. ship with manully compiled loadable extensions.

ZenLiuCN · 2023-12-24T04:49:46Z

Ps those extensions I compiled is under folder. And the extension compile script not finish yet.

ZenLiuCN · 2023-12-24T04:51:58Z

Ps, those three file are the script I use to compile the lib.

-----------------------------------------------------------------------
Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          22/12/2023    16:29            490 build.sh
-a---          22/12/2023    16:29             62 copyLib.sh 
-a---          22/12/2023    16:29            199 mergeStaticLib.sh

mergeStaticLib.sh not tested. just an Idea

ZenLiuCN · 2023-12-24T05:03:18Z

ilammy/msvc-dev-cmd seems should get a try. I will try on it (it may take some time)

ZenLiuCN · 2023-12-25T02:16:10Z

ilammy/msvc-dev-cmd seems should get a try. I will try on it (it may take some time)

done try. there still no stable method to link a static lib built with msvc .

ZenLiuCN · 2023-12-25T16:09:33Z

I moved relative codes to here, Cause the engine module is too big for pull from go mod server.

I will try to add a github workflow to build the mingw libs, maybe that will do a little help.

But also should notice that no official extensions provided for windows_amd64_mingw toolchain. Maybe requires extra provided extentions.

PS: I suggest that maybe put binaries to a seprated location or as release binary archives (so avoid to reach the go mod size limit , it'll broken everything, if with mingw lib and extensions it will reach the limit, maybe about 50mega bytes after zip compressed )
Merry Christmas ^_^

triiberg · 2024-01-02T14:12:30Z

-Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread

Hi. This very helpful, somehow it doesn't work in my computer. Do you have full build command? I think I'm missing some flags :( It might help others as well who does not have too much experience of compiling stuff.

ZenLiuCN · 2024-01-02T15:28:23Z

-Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread

Hi. This very helpful, somehow it doesn't work in my computer. Do you have full build command? I think I'm missing some flags :( It might help others as well who does not have too much experience of compiling stuff.

Here the the script for compile duckdb with mingw (GCC v13), You not need -Wl,-Bstatic -lstdc++ -Wl,-Bstatic -lpthread for compile. It's only required for link without dyn dlls.

#!/bin/sh
mkdir -p build && \
cmake -G "MinGW Makefiles" \
 -DENABLE_EXTENSION_AUTOLOADING=1 \
 -DENABLE_EXTENSION_AUTOINSTALL=1 \
 -DDUCKDB_EXTENSION_CONFIGS="./.github/config/bundled_extensions.cmake" \
 -DBUILD_SHELL=0 \
 -DBUILD_BENCHMARK=0 \
 -DBUILD_JDBC=0 \
 -DBUILD_TPCH=0 \
 -DBUILD_TPCDS=0 \
 -DBUILD_ODBC=0 \
 -DBUILD_PYTHON=0 \
 -DDISABLE_UNITY=1 \
 -DCMAKE_BUILD_TYPE=Release -B build &&\
cd build &&\
 MAKEFLAGS=-j6 cmake --build . --config Release &&\
cd ../ &&\
mkdir -p lib

ZenLiuCN · 2024-01-02T15:30:19Z

@triiberg HERE the scripts and other test like compile pg-scanner with mingw. I can't get MySQL-scanner to work. good luck.
PS: if you are use to link by cgo, The should have a MingW64 installed with your system. If not Winlibs may help, or just with the orignal MinGW64.

triiberg · 2024-01-03T09:10:30Z

The DuckDb part compiled very well. Thank you! I'm still don't understand how I can include it to my Go project's building process (like the simple Go project that initiates DuckDb database, writes one record). I need to create a Golang microservice using my linux, then compile it to other platforms: win and mac. It compiles to linux very well, no problem at all. But not for windows.

$ GOOS=windows GOARCH=amd64 CGO_ENABLED=1 CGO_LDFLAGS="-L./deps/windows_amd64_mingw" CGO_CFLAGS="-I./deps/includes" CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ go build -tags=duckdb_use_lib -o builds/app.exe cmd/main.go

returns result:

# command-line-arguments
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-g++ failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: cannot find -lduckdb: No such file or directory
collect2: error: ld returned 1 exit status

and the same command but I remove the -tags=duckdb_use_lib I'll got this one:

# github.com/marcboeker/go-duckdb
In file included from duckdb.cpp:2:
duckdb.hpp:1686:12: error: ‘mutex’ has not been declared in ‘std’
 1686 | using std::mutex;
      |            ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:12582:2: error: ‘mutex’ does not name a type
12582 |  mutex lock;
      |  ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:14037:2: error: ‘mutex’ does not name a type
14037 |  mutex lock;
      |  ^~~~~
duckdb.hpp:14679:2: error: ‘mutex’ does not name a type
14679 |  mutex lock;
      |  ^~~~~
In file included from duckdb.cpp:2:
duckdb.hpp:16566:2: error: ‘mutex’ does not name a type
16566 |  mutex producer_lock;
      |  ^~~~~
duckdb.hpp:16611:2: error: ‘mutex’ does not name a type
16611 |  mutex thread_lock;
      |  ^~~~~
duckdb.hpp:16729:2: error: ‘mutex’ does not name a type
16729 |  mutex batch_lock;
      |  ^~~~~
duckdb.hpp:16852:2: error: ‘mutex’ does not name a type
16852 |  mutex executor_lock;
      |  ^~~~~
duckdb.hpp:16853:2: error: ‘mutex’ does not name a type
16853 |  mutex error_lock;
      |  ^~~~~
duckdb.hpp:17907:2: error: ‘mutex’ does not name a type
17907 |  mutex lock;
      |  ^~~~~
duckdb.hpp:18164:49: error: ‘mutex’ was not declared in this scope
18164 | typedef unordered_map<CatalogSet *, unique_lock<mutex>> set_lock_map_t;
      |                                                 ^~~~~
duckdb.hpp:18164:49: error: template argument 1 is invalid
duckdb.hpp:18164:54: error: template argument 2 is invalid
18164 | typedef unordered_map<CatalogSet *, unique_lock<mutex>> set_lock_map_t;
      |                                                      ^~
duckdb.hpp:18164:54: error: template argument 5 is invalid
duckdb.hpp:18277:72: error: ‘mutex’ was not declared in this scope

triiberg · 2024-01-03T09:18:54Z

And if I put the .a files into project own directory (someone told me that -l option doesn't go for other directories?) I got errors like this:

# command-line-arguments
/usr/local/go/pkg/tool/linux_amd64/link: running x86_64-w64-mingw32-g++ failed: exit status 1
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x8d): undefined reference to `__imp_duckdb_append_blob'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0xe9): undefined reference to `__imp_duckdb_append_bool'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x147): undefined reference to `__imp_duckdb_append_double'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x1a4): undefined reference to `__imp_duckdb_append_float'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x200): undefined reference to `__imp_duckdb_append_int16'
/usr/bin/x86_64-w64-mingw32-ld: /tmp/go-link-1052453679/000002.o:appender.cgo2.:(.text+0x258): undefined reference to `__imp_duckdb_append_int32'

ZenLiuCN · 2024-01-03T10:53:26Z

That requires modify the go-duck source, I suggest use dynamic link for windows ( also ship the dll with yourbuilt go service) That will much easier. I doesn't tried cross-compile from linux to windows. See [level 6](#115 (comment)) for details of modify go-duckdb. (Really not suggest use this solution for production , use dynamic linked will more stable, at least for windows)

camperjett · 2024-06-12T09:27:18Z

Hi!
I am working on a go project on windows. How do I use this go-client?

ZenLiuCN · 2024-06-12T10:08:56Z

use dynamic lib link via tags. it is the most stable for current. unless U real need a standalone executable.

marcboeker added the question / help wanted Further information is requested label Aug 21, 2023

gedw99 mentioned this issue Sep 20, 2023

windows release rilldata/rill#3104

Closed

taniabogatsch mentioned this issue Jun 6, 2024

Update README.md #229

Closed

taniabogatsch added the [OS] Windows [OS] Windows label Jun 6, 2024

amos-lsl closed this as completed Jun 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

windows build #115

windows build #115

amos-lsl commented Aug 20, 2023

marcboeker commented Aug 21, 2023

amos-lsl commented Aug 23, 2023

Dimirolz commented Oct 30, 2023

marcboeker commented Oct 30, 2023

ZenLiuCN commented Dec 17, 2023

ZenLiuCN commented Dec 18, 2023

marcboeker commented Dec 23, 2023

marcboeker commented Dec 23, 2023

ZenLiuCN commented Dec 24, 2023 via email

ZenLiuCN commented Dec 24, 2023 •

edited

Loading

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 24, 2023 •

edited

Loading

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 25, 2023

ZenLiuCN commented Dec 25, 2023 •

edited

Loading

triiberg commented Jan 2, 2024

ZenLiuCN commented Jan 2, 2024

ZenLiuCN commented Jan 2, 2024 •

edited

Loading

triiberg commented Jan 3, 2024

triiberg commented Jan 3, 2024

ZenLiuCN commented Jan 3, 2024 via email •

edited

Loading

camperjett commented Jun 12, 2024

ZenLiuCN commented Jun 12, 2024 via email

windows build #115

windows build #115

Comments

amos-lsl commented Aug 20, 2023

marcboeker commented Aug 21, 2023

amos-lsl commented Aug 23, 2023

Dimirolz commented Oct 30, 2023

marcboeker commented Oct 30, 2023

ZenLiuCN commented Dec 17, 2023

ZenLiuCN commented Dec 18, 2023

marcboeker commented Dec 23, 2023

marcboeker commented Dec 23, 2023

ZenLiuCN commented Dec 24, 2023 via email

ZenLiuCN commented Dec 24, 2023 • edited Loading

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 24, 2023 • edited Loading

ZenLiuCN commented Dec 24, 2023

ZenLiuCN commented Dec 25, 2023

ZenLiuCN commented Dec 25, 2023 • edited Loading

triiberg commented Jan 2, 2024

ZenLiuCN commented Jan 2, 2024

ZenLiuCN commented Jan 2, 2024 • edited Loading

triiberg commented Jan 3, 2024

triiberg commented Jan 3, 2024

ZenLiuCN commented Jan 3, 2024 via email • edited Loading

camperjett commented Jun 12, 2024

ZenLiuCN commented Jun 12, 2024 via email

ZenLiuCN commented Dec 24, 2023 •

edited

Loading

ZenLiuCN commented Dec 24, 2023 •

edited

Loading

ZenLiuCN commented Dec 25, 2023 •

edited

Loading

ZenLiuCN commented Jan 2, 2024 •

edited

Loading

ZenLiuCN commented Jan 3, 2024 via email •

edited

Loading