Preliminary notes on surface representations

[matthew-brett]

By @effigies, August 9, 2021, 8:03pm

The basic problem of working with surfaces is that you have three types of data (previous notes at RFP: SurfaceImage API · Issue #936 · nipy/nibabel · GitHub:

• Topology - a graph of neighboring points/vertices, typically defined as triangles/faces
• Geometry - the coordinates of each point
• Data - scalars or vectors (including time series) defined on each point or face

For the purposes of efficiency, these will often be stored in separate files. Surface files typically contain topology and geometry, while data files typically purely contain one or more data vectors and no geometrical information at all.

Not all components are needed for many tasks. For example, vertex-wise calculations can typically be performed with one or more data arrays, without reference to the geometry. Spatial smoothing and plotting will require topological and geometric data as well.

In most cases, files are associated by keeping them in a common directory structure, which requires tooling to know about the directory conventions of each potential format.

The file formats currently supported in nibabel are:

• GIFTI - Topology, geometry and data are “data arrays” in an XML file. A given file may contain any combination of data arrays.
• FreeSurfer
  Surface files contain topology and geometry, are maintained within the surf/ subdirectory. It is standard to have multiple geometries, each with the same topology, mapping to locations in the brain volume or in more abstract spaces used for visualization (inflated) or registration (sphere).
  Different formats for parcellation (label) data, morphometric data, and generic vectors (MGH format is re-purposed here, IIRC).
• CIFTI-2 - These are data arrays with flexible coordinates. For example, a position along an axis may correspond to a vertex on a surface, a voxel in a volume, an ROI in an atlas, or a point in time. Generally requires a wbspec file to associate the data array with geometry and topology.

There are many more, I suspect most comprehensively supported by Surf Ice.

Nilearn has adopted an extremely simple API:

Mesh = namedtuple("mesh", ["coordinates", "faces"])
Surface = namedtuple("surface", ["mesh", "data"])

This has the limitation:

• Multiple surfaces cannot currently be associated, e.g. L+R hemisphere
• Lack of a proxying mechanism means geometry is expensive to keep associated with surface data, especially when that involves copying

CIFTI’s BrainModelAxis and WBSpec may provide a reasonable model to associate multiple surfaces (or other structures).

[matthew-brett]

By @neurolabusc, August 10, 2021, 6:32pm

The formats supported by Surfice are listed here. It simply supports popular formats, and should not be seen as an endorsement (e.g. STL format does not reuse vertices, so is inherently inefficient for both disk storage and modern GPUs).

The supported formats fall into 3 classes:

• Triangulated meshes.
• Streamlines (for diffusion data).
• Graph based nodes and edges (e.g. an edge might be the correlation between two specific nodes).

With regards to streamlines, several people have noted that Python routines are slow and can be accelerated for existing formats. @frheault also sketched out a new format that could address many of the limitations with existing formats.

[matthew-brett]

By fangq December 2, 2021, 3:45pm

Wow, that’s an impressively comprehensive list of mesh formats - I assume the test data were one of the sample files from surf-ice. Can you let me know which file were used for the table? I’d love to try it on my JSON/UBJSON based mesh formats in combination with various built-in compression options.

[matthew-brett]

By @ neurolabsc

@ fangq - those tests used native code. I have just created a JavaScript test - feel free to try and optimize any of the format readers.

[matthew-brett]

By @ fangq March 28, 2022, 2:46am

hi @ neurolabusc, I tested JSON based .jmsh and binary JSON based .bmsh files and added the new mesh files, parsers and benchmark results in this PR
github.com/neurolabusc/MeshFormatsJS

on a laptop running Ubuntu 20.04 (Ryzen 7 4800H), the benchmark results are updated as below.

fangq@rainbird:~/space/git/Temp/MeshFormatsJS$ node ./meshtest.js
gifti.gii	Size	4384750	Time	2143
gz.mz3	Size	3259141	Time	545
raw.mz3	Size	5898280	Time	28
obj.obj	Size	13307997	Time	5236
stl.stl	Size	16384084	Time	1053
zlib.jmsh	Size	4405604	Time	625
zlib.bmsh	Size	3259049	Time	479
raw.min.json	Size	12325881	Time	1239
raw.bmsh	Size	5898902	Time	38
Done

[matthew-brett]

By @fangq, December 2, 2021, 3:06pm

I wrote and have maintained a matlab based 3D mesh generator, Iso2Mesh (http://iso2mesh.sf.net, GitHub - fangq/iso2mesh: Iso2Mesh - a 3D surface and volumetric mesh generator for MATLAB/Octave), over the past years. We have also developed a brain-specific mesh generation pipeline built over iso2mesh - named Brain2Mesh (Brain2Mesh - A one-liner brain mesh generator). These toolboxes were build upon a number of other open-source meshing utilities - Tetgen, CGAL, Cork, meshfix … A big part of my computational work also involves mesh generation and tissue shape modeling (FEM based, or mesh-based Monte Carlo).

In a drafted JSON-based mesh-data exchange specification - JMesh - I put together some of the typical data structures that I would expect to encounter in shape/surface/mesh based analysis tools, you can see the full document here
github.com

specifically, the specification currently covers 1) discretized geometries (surfaces, solid elements, NURBS), 2) shape primitives (cubes, bricks, spheres, cylinders…), 3) textures (1D,2D,3D) and 4) constructive solid geometries (CSG). In one of my TODOs, I would like to include time-frame related keywords to represent shape changes over time (like what Blender/3DS handling of animations), but it will be less relevant to neuroimaging applications.

Example categories of keywords from the JMesh spec are listed below, showing the scope of the geometry data that I would like to encapsulate using this format.

• Mesh grouping: MeshGroup, MeshObject, MeshPart
• Vertices: MeshVertex1,MeshVertex2,MeshVertex3,MeshVertex4
• Lines: MeshLine,MeshEdge,MeshBSpline2D
• Surfaces: MeshTri3,MeshQuad4,MeshPLC,MeshNURBS
• Solid: MeshTet4,MeshHex8,MeshPyramid5,MeshTet10
• Flexible containers: MeshNode,MeshSurf,MeshPoly,MeshElem
• CSG operators: CSGObject,CSGUnion,CSGIntersect,CSGSubtract
• Texture: Texture1D,Texture2D,Texture3D
• 2-D shape primitives: ShapeBox2,ShapeDisc2,ShapeEllipse,ShapeLine2,
  ShapeArrow2,ShapeAnnulus,ShapeGrid2
• 3-D shape primitives: ShapeLine3,ShapePlane3,ShapeBox3,ShapeDisc3,
  ShapeGrid3,ShapeSphere,ShapeCylinder,ShapeEllipsoid,ShapeTorus,ShapeCone,
  ShapeConeFrustum,ShapeSphereShell,ShapeSphereSegment
• Extrusion and revolving: ShapeExtrude2D,ShapeExtrude3D,ShapeRevolve3D
  Properties: Color,Normal,Size,Label,Value

Please let me know if this is of interest to you. Majority of these support of these data structures are available in Iso2mesh for MATLAB/Octave.