Reputation: 21
I'm working on a project that involves constructing a watertight triangle mesh from a point cloud (potentially using alpha shapes), optimizing point positions (with minimal recomputation of the mesh), projecting the mesh to 2D and finding boundary points, preventing self-intersections, calculating mesh volume, and integrating all this into a differentiable machine learning pipeline. I am looking to find a mesh library which will assist me. I'm choosing between Open3D and PyTorch3D currently, but am open to using both or using any other libraries which I have not yet come across.
I have looked at the documentation for both and my observations are as follows.
Open3D provides functionality to create a mesh from a point cloud using alpha shapes (create_from_point_cloud_alpha_shape
), check if a mesh is watertight (is_watertight
), and calculate its volume (get_volume
). It also includes an ML add-on, though this seems focused on batch processing and dataset handling rather than enabling backpropagation, and so to perform backpropagation, I would need to backpropagate through the point cloud to get new points, and then compute a new mesh based on these updated points.
On the other hand, PyTorch3D integrates well with PyTorch, making it fully compatible with a differentiable pipeline. However, it lacks built-in support for alpha shape-based mesh construction, watertightness checks, and direct volume calculation (though volume could be implemented manually using a 3D shoelace formula).
I’d appreciate any advice, alternative suggestions, or insights on whether these concerns are over- or under-emphasized.
Upvotes: 1
Views: 64
Reputation: 7937
If I were you, I would use Open3D to create the initialization of the mesh, after mesh is created, I would not adjust vertex interconnection, i.e. not changing edges via optimization.
Then convert this mesh into PyTorch3D, optimize vertex positions with custom written / found on the Internet loss on mesh volume, self-intersection, etc. Optimize for a few hundreds steps, convert to Open3D, use it to do whatever deterministic algorithms (e.g. simplify mesh, compute volume exactly for sanity check, etc).
Then convert new mesh to PyTorch3D mesh and restart optimization (with resetting optimizer state for mesh data).
"I underestimating the effort required to extend one of them?"
You do, it's very hard to do most of mesh manipulations differentiably, so extending Open3D for differentiability is not an option
Writing custom mesh algorithms that you want on top of PyTorch3D is more realistic, but I would anyway suggest to stay away from it if possible
Upvotes: 2