Merge branch 'master' into feature/ax

.github/workflows/build.yml

@@ -139,19 +139,19 @@ jobs:
     - uses: actions/checkout@v2
     - name: install
       shell: bash
+      # brew boost-python3 installs a "Keg-only" version of python which is
+      # not installed to PATH. We must manually provide the location of the
+      # required python installation to CMake through a hint variable which
+      # is exported in install_macos.sh
       run: ./ci/install_macos.sh
     - name: install
       shell: bash
       run: ./ci/install_blosc.sh 1.5.0
     - name: build
       shell: bash
-      # brew boost-python installs a "Keg-only" version of python which is
-      # not insatlled to PATH. Until this becomes the default, we must
-      # manually provide the location of the require python installation
-      # See https://formulae.brew.sh/formula/[email protected]
       # Also need to disable compiler warnings for ABI 6 and above due to
       # the version of clang installed
-      run: ./ci/build.sh Release 7 ON SSE42 "core,python,bin,test" -DPython_ROOT_DIR=/usr/local/opt/[email protected] -DOPENVDB_CXX_STRICT=OFF
+      run: ./ci/build.sh Release 7 ON SSE42 "core,python,bin,test" -DOPENVDB_CXX_STRICT=OFF
     - name: test
       shell: bash
       run: ./ci/test.sh

CHANGES

@@ -13,6 +13,7 @@ Version 7.1.1 - In Development
       packed geometry were also being converted.
     - Fixed a bug where a Houdini SOP's verb would not be correctly associated
       with the corresponding node if the node's internal name was changed.
+    - Fixed bug where OpenVDB Convert SOP could revert the name attribute.
 
     Bug fixes:
     - Fixed a bug which could cause recursive compile time instantiations of

ci/install_macos.sh

@@ -13,3 +13,13 @@ brew install cppunit
 brew install tbb
 brew install zlib
 brew install glfw
+brew install jq # for trivial parsing of brew json
+
+# Alias python version installed by boost-python3 to path
+py_version=$(brew info boost-python3 --json | \
+    jq -cr '.[].dependencies[] | select(. | startswith("python"))')
+echo "Using python $py_version"
+# export for subsequent action steps (note, not exported for this env)
+echo "Python_ROOT_DIR=/usr/local/opt/$py_version" >> $GITHUB_ENV
+echo "/usr/local/opt/$py_version/bin" >> $GITHUB_PATH
+

doc/changes.txt

@@ -18,6 +18,7 @@ Houdini:
   packed geometry were also being converted.
 - Fixed a bug where a Houdini SOP's verb would not be correctly associated
   with the corresponding node if the node's internal name was changed.
+- Fixed bug where OpenVDB Convert SOP could revert the name attribute.
 
 @par
 Bug Fixes:

openvdb_houdini/openvdb_houdini/SOP_OpenVDB_Convert.cc

@@ -1017,9 +1017,13 @@ SOP_OpenVDB_Convert::Cache::convertVDBType(
     const UT_String& outPrecStr,
     hvdb::Interrupter& boss)
 {
+    GA_RWHandleS name_h(gdp, GA_ATTRIB_PRIMITIVE, "name");
     for (hvdb::VdbPrimIterator it(&dst, group); it; ++it) {
         if (boss.wasInterrupted()) return;
 
+        if (name_h.isValid())
+            it->getGrid().setName(static_cast<const char *> (name_h.get(it->getMapOffset())));
+
         const UT_VDBType inType = it->getStorageType();
         const UT_String inTypeName = getVDBTypeName(inType);
         const int inBits = getVDBPrecision(inType);

tsc/meetings/2020-09-22.md

@@ -0,0 +1,89 @@
+Minutes from 64th OpenVDB TSC meeting, Sep 22nd, 2020, (EDT)
+
+Attendees: *Nick* A., *Jeff* L., *Ken* M., *Dan* B.
+
+Additional Attendees: Johannes Meng (Intel), JT Nelson (Blender),
+Andre Pradhana (DW), Bruce Cherniak (Intel)
+
+Regrets: *Peter* C.
+
+Agenda:
+
+1) Confirm quorum
+2) Secretary
+3) Forum
+4) NanoVDB
+5) PRs Outstanding
+6) Checksums
+7) Next Meeting
+
+
+1) Quorum was confirmed.
+
+2) Secretary was Jeff Lait
+
+3) Forum
+
+Ken still to reply to the out of core question.  The answer to the question is straightforward, but what would we like in the future?
+
+4) NanoVDB
+
+NanoVDB also has interest in delayed loading.
+
+Jeff does not want NanoVDB to be more complicated than it has to be.
+
+There are some additional changes still coming to support DirectX.
+
+An improved C-port also exists.
+
+5) PRs Outstanding
+
+What is the process for external PR.  Does the second Reviewer merge or the first?  Or someone else does?  We will continue to work ad-hoc.
+
+Feature branches, however, must be merged by the owner of the feature branch.
+
+Likewise, if a PR is for a particular person's domain, that person will be left with the merge.
+
+a) PR 829, sync names.  To be merged
+
+b) PR 823, make delayed loading/iostreams optional.
+
+This disables delayed loading.  This does change the File::Impl, but this should be name-spaced and internal, so not cause actual ABI changes.
+
+Should we have ENABLE or DISABLE?  First we should have config flags in the source file only where possible.  Flag names depending on cmake defaults is scary.  Our Find Module is not yet powerful enough yet.  So we want to keep Find Module as simple as possible.
+
+If we tried to move the delayed loading to the C file we'd lose the ability to remove the mutex.
+
+Configuration of the build system is USE_FOO.  These Cmake variables then drive -D variables.  These vary currently.  Should half and delayed loading be off by default?  Apparently our models on the web are all half...
+
+Ideally the default is no dependency.
+
+The define will be DISABLE for delayed loading.
+
+C) PR 819.  Remove half.
+
+Should we just bring in a Half.C/Half.h rather than removing Half.h?
+
+In future version of OpenEXR it will be its own header we can include.
+
+What should we do with missing dependencies?  A runtime check is required because we don't know the source of the grid, it can come from another library that has support.  We will go for runtime errors, not compile time or linker error.
+
+D) PR 818, all of AX
+
+This has AX based CI.  It has to install multiple versons of LLVM, which eventually evicts the Houdini caches.  But as LLVM isn't EULA based, we should be able to pre-populate the containers with them.
+
+We should read over this prior to the next TSC, especially any questions inside.
+
+6) Checksums
+
+Ken is researching CRC-variants, some are parallel, so fast.  Three modes have been added.  No checksum.  Checksum for the metadata only.  Full checksum.  Separate CRC are for grid, tree, root, and else.
+
+One use is debugging, quickly verifying things are identical.
+
+Validation tool for NanoVDB to verify no pointers go out of scope.  The security member approves.
+
+7) Next Meeting
+
+Focus on AX.
+
+Tuesday September 29th 2020, 1-2pm EDT (GMT-4)

tsc/meetings/2020-09-29.md

@@ -0,0 +1,179 @@
+Minutes from 64th OpenVDB TSC meeting, Sep 29th, 2020, (EDT)
+
+Attendees: *Nick* A., *Jeff* L., *Ken* M., *Dan* B.
+
+Additional Attendees: Johannes Meng (Intel), JT Nelson (Blender),
+Andre Pradhana (DW), Robin Rowe (Cinepaint)
+
+Regrets: *Peter* C.
+
+Agenda:
+
+1) Confirm quorum
+2) Secretary
+3) Forum
+4) PR 818 AX Review
+5) Simd
+6) Next Meeting
+
+1) Quorum was confirmed.
+
+2) Secretary was Nick Avramoussis
+
+3) Forum
+
+Still need to reply to some forum posts, Dan to take a look. A few issues w.r.t
+VDB 7.x dependent software not building as users are not realizing C++14 is now
+required. Dan to put in a compile time check in Platform.h to explicitly fail
+if C++14 is not enabled.
+
+4) PR 818 AX Review
+
+Most of this meeting was spent reviewing the comments made on PR 818. Replies
+are copied here for convenience.
+
+AX CI
+Currently the AX CI builds LLVM from source and caches it on the available
+github actions cache. Dan has initiated conversations with the LF to see if they
+can provide docker containers for different LLVM versions.
+
+Precision of integers and strict type binding
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496308313
+
+AX currently support short, int and long signed integer values which are aliased
+to 16, 32 and 64 bit respectively. This is primarily required due to the way AX
+strictly binds to grids/attributes of the same type (i.e short@myattrib). Agreed
+that, realistically, 32 or 64 bits should be exposed. There are some performance
+considerations making everything 64 bit on some architectures. Best solution was
+to first allow @ to bind to the same type category, remove short, rename int and
+long to int32 and int64 and alias int to either int32 or int64 at compile time.
+Same comments were made w.r.t float and double, however GPU support will
+definitely need float and the lack of float for CPU code will make it impossible
+to produce deterministic results.
+
+$ syntax in Houdini
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496311518
+
+The AX Houdini SOP supports some vex functions like ch/chramp etc. It evaluates
+the raw string to avoid $ expanding at the cost of no backtick support (though
+this can be added). Ultimately the presence of ch means that this isn't an issue.
+
+Attrib dependencies
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496312075
+
+No evaluation is performed on code values, so dead/unreachable code paths still
+contribute to attribute/grid creation and dependencies.
+
+C vs python style modulo
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496312423
+
+Currently uses LLVM's default IR builder modulo instruction which will be C
+style remainder op. Agreed to instead switch to python style "true" modulo op.
+
+Line directives, pre-processing and runtime errors
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496315058
+
+Currently no # directive support. Agreed adding it would be good. Catching
+exceptions is not an ideal way to control top level behavior. Agreed that
+custom logging would be better. This actually already exists but needs to be
+upstreamed. This was going to be done as a subsequent PR but may as well be
+bundled into this now.
+
+lerp()
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496317012
+fit()
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496318424
+
+Agreed to investigate better formulation of lerp() and fit()
+
+signbit()
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496319359
+
+Agreed to change to sign()
+
+Function signature type support
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496317548
+
+Depending on the int precision work, this work will be simplified. Signatures
+should really exist for all types unless it really doesn't make sense for some
+specifically. Agreed to add a maxComponent style method for vector elements.
+
+Determinant checking of polardecomp
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496319863
+
+Need to check the behavior of openvdbs polardecomposition function if the
+calculated det is negative and document this.
+
+AX namespace, API/ABI
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496322339
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496449902
+
+Agree on the two options stated - either AX should use its own versioned namespace
+or be locked to VDBs. This includes lib versioning too. Preference is to lock to
+VDBs namespace. Will peruse this and see if any issues arise.
+
+VDB changes
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496433146
+
+Although minor, agreed to split out VDB specific changes to a different PR.
+
+Deprecated methods and odd implementations
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496434030
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496441704
+
+For backwards compatibility and internal reasons, some design decisions were
+made which now look odd in the open source version of AX. These still need to
+exist in the standalone repo but can be removed and re-worked for the merge into
+VDB. Agreed to action these.
+
+Executable terminology
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496439529
+
+Agreeded to rename to Binary and see what it looks like.
+
+Executable member interface and usage
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496444627
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496443272
+
+In the current design, the executables store all settings on a sub struct unique
+ptr. This produces a clean API with separation of the trivial execute calls
+from the member settings. However it means that they must be copied to make
+modifications if the exe is const (though this copying is cheap) and requires
+explicit copy constructors/assignment operators. Another option would be to have
+users provide these settings on each call to execute. This better ties in some
+settings to the grid data being executed, produces a trivially copyable object
+and means the executable can be const. Nick, will play around with these ideas.
+
+Combination of Compiler to Executable
+https://github.com/AcademySoftwareFoundation/openvdb/pull/818#discussion_r496446789
+
+The templated compile methods exist for internal reasons and can be removed. It
+may be better to introduce static creators on the executables which take a compiler
+vs the friend implementation. No clear advantage currently, but agreeded that
+the workflow between the code generators, compiler and exes could use some
+attention.
+
+ax::run() with multiple grids/attribute bindings
+
+Agreed that this particular signature could use some more explicit behavior.
+Agreed to error if points and volumes are provided. Custom way to bind attributes
+to grids should exist that don't rely on the grid name (i.e. what if the grid is
+const).
+
+Below are the main action points which block the initial merge:
+ - Integer precision changes
+ - Modulo implementation
+ - Deprecated code removal
+ - lerp(), fit(), signbit() changes
+ - AX Versioning changes
+
+5) Simd
+
+Ken, working on some more intrinsic functionality for VDB. Nick, has code he'd
+like to upstream which uses SIMD wrappers. TSC is open to including external
+SIMD intrinsic wrapper software in VDB.
+
+6) Next Meeting
+
+Skipping next week.
+Next meeting is October 13th, 2020. 1pm-2pm EDT (GMT-4).