2.2.1 build failure related to libvbyte

[ilovezfs]

Attempting to upgrade the Homebrew formula to 2.2.1, I'm seeing the following build failure:

Undefined symbols for architecture x86_64:
  "_masked_vbyte_decode", referenced from:
      _vbyte_uncompress_unsorted32 in libupscaledb.a(vbyte.o)
  "_masked_vbyte_decode_delta", referenced from:
      _vbyte_uncompress_sorted32 in libupscaledb.a(vbyte.o)
  "_masked_vbyte_search_delta", referenced from:
      _vbyte_search_lower_bound_sorted32 in libupscaledb.a(vbyte.o)
  "_masked_vbyte_select_delta", referenced from:
      _vbyte_select_sorted32 in libupscaledb.a(vbyte.o)
ld: symbol(s) not found for architecture x86_64

https://gist.github.com/ilovezfs/76655b07db888bba145f636ea7abd4da

[ilovezfs]

I can hack around this with

diff --git a/3rdparty/libvbyte/vbyte.cc b/3rdparty/libvbyte/vbyte.cc
index 6faf792..46255b8 100644
--- a/3rdparty/libvbyte/vbyte.cc
+++ b/3rdparty/libvbyte/vbyte.cc
@@ -24,10 +24,6 @@ typedef signed char int8_t;
 #  include <stdint.h>
 #endif
 
-#if defined(__AVX__) || defined(__AVX2__) || defined(__SSE4__)
-#  define USE_MASKEDVBYTE 1
-#endif
-
 #include "vbyte.h"
 #include "varintdecode.h"

[cruppstahl]

Thanks for looking into it... The SSE/AVX code in MaskedVbyte should only be compiled if ./configure recognizes SSE4 support: if grep -q sse4 /proc/cpuinfo; then AM_CONDITIONAL(ENABLE_SSE4, true) settings="$settings (simd-sse4)" else settings="$settings (no simd-sse4)" fi Maybe this check does not work on MacOS?

[ilovezfs]

@cruppstahl in Homebrew here's how we check CPU features on Linux and macOS:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/extend/os/linux/hardware/cpu.rb
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/extend/os/mac/hardware/cpu.rb

So the equivalent of checking /proc/cpuinfo is checking sysctls such as

bash-3.2$ sysctl -n hw.optional.sse4_1
1
bash-3.2$ sysctl -n hw.optional.sse4_2
1
bash-3.2$ sysctl -n hw.optional.sse3
1
bash-3.2$

[cruppstahl]

Great, thank you. I'll provide a fix tonight.

[cruppstahl]

I've pushed a new commit to the master branch: 68ca8c3
Can you give it a try please?

[ilovezfs]

@cruppstahl the cpu thing seems fixed but the C++ thing does not:

Making all in unittests
clang++ -DHAVE_CONFIG_H -I. -I..  -I../include -I../include -I../src -std=c++11 -DUPS_ENABLE_REMOTE -I/usr/local/Cellar/upscaledb/HEAD-68ca8c3_5/libexec/libuv/include -msse2 -flax-vector-conversions -msse4 -flax-vector-conversions -I/usr/local/Cellar/upscaledb/HEAD-68ca8c3_5/libexec/libuv/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/include/darwin -DNDEBUG -std=c++0x -c -o simd.o simd.cpp
simd.cpp:31:20: error: implicit instantiation of undefined template 'std::__1::array<unsigned short, 16>'
  std::array<T, S> values;
                   ^
simd.cpp:47:3: note: in instantiation of function template specialization 'test_linear_search_sse<unsigned short, 16>' requested here
  test_linear_search_sse<uint16_t, 16>();
  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:116:65: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
                                                                ^
simd.cpp:31:20: error: implicit instantiation of undefined template 'std::__1::array<unsigned int, 16>'
  std::array<T, S> values;
                   ^
simd.cpp:52:3: note: in instantiation of function template specialization 'test_linear_search_sse<unsigned int, 16>' requested here
  test_linear_search_sse<uint32_t, 16>();
  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:116:65: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
                                                                ^
simd.cpp:31:20: error: implicit instantiation of undefined template 'std::__1::array<unsigned long long, 4>'
  std::array<T, S> values;
                   ^
simd.cpp:57:3: note: in instantiation of function template specialization 'test_linear_search_sse<unsigned long long, 4>' requested here
  test_linear_search_sse<uint64_t, 4>();
  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:116:65: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
                                                                ^
simd.cpp:31:20: error: implicit instantiation of undefined template 'std::__1::array<float, 16>'
  std::array<T, S> values;
                   ^
simd.cpp:62:3: note: in instantiation of function template specialization 'test_linear_search_sse<float, 16>' requested here
  test_linear_search_sse<float, 16>();
  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:116:65: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
                                                                ^
simd.cpp:31:20: error: implicit instantiation of undefined template 'std::__1::array<double, 4>'
  std::array<T, S> values;
                   ^
simd.cpp:67:3: note: in instantiation of function template specialization 'test_linear_search_sse<double, 4>' requested here
  test_linear_search_sse<double, 4>();
  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__tuple:116:65: note: template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
                                                                ^
5 errors generated.
make[2]: *** [simd.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
bash-3.2$

[cruppstahl]

This seems to be a different issue as the previous C++ syntax error... Which clang version do you use? I'll try to replicate your setup on Ubuntu.
You might be able to fix this by specifying --disable-simd as an option for the ./configure script.

[ilovezfs]

Without --disable-simd:
https://gist.github.com/ilovezfs/7883ed96495a6291197b28f607134c21

With --disable-simd:
https://gist.github.com/ilovezfs/f8b0d1e66117a537f23faaa4a6f67324

iMac-TMP:~ joe$ clang --version
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
iMac-TMP:~ joe$ brew config
HOMEBREW_VERSION: 1.3.0-17-g1cae6dd
ORIGIN: https://github.com/Homebrew/brew
HEAD: 1cae6dd6777aa6657c0df500a32ef68cefb2b746
Last commit: 3 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 8c2e2088b0200fb485e96da20490f4e1f46305e1
Core tap last commit: 2 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: octa-core 64-bit skylake
Homebrew Ruby: 2.0.0-p648 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby
Clang: 8.0 build 800
Git: 2.10.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: 1.8.0_121
macOS: 10.11.6-x86_64
Xcode: 8.2.1
CLT: 8.2.0.0.1.1480973914
X11: 2.7.11 => /opt/X11
iMac-TMP:~ joe$