I noticed that there are many useful Python APIs designed for image preprocessing (grooming), which gives us great flexibility. However, when it comes to optimization stage, the Python APIs are just calling the optimize command of ShapeWorks, which reads the project sheet with the shape filenames and optimization parameter values. I wonder whether we can have a more detailed control of the optimization process.
I am now trying to develop a combined model using PyTorch, which incorporates SSM (not DeepSSM) with some other deep learning models. Thus I want to take a tensor (of some 3D image) as the input, and retrieve the loss in SSM optimization stage. ShapeWorks uses a loss function H(Z) - ΣH(Xn) to find optimal particles on the input surfaces, and I wonder whether it is possible to get that loss, and use it in PyTorch for model optimization.
If there is no APIs to do so currently, do you think it is possible to retrieve that information with a reasonable amount of manual work? What I have is input 3D images of some shape, and what I want is a SSM model which takes that shape and generates particles (correspondences) on the surface, with minimal loss H(Z) - ΣH(Xn). Furthermore, I would like to optimize the SSM using this loss using the PyTorch framework. Do you think it’s possible to do so?
We have some preliminary work along these lines with goals similar to what you’ve described. We provide a per-iteration Python callback function and some APIs for getting and setting the current particle positions and update vectors. This allows the python client to swap out the objective function. Does this sound like what you are looking for?
I have a set of shapes (3D binary segmentations), and I am interested in the task described here (How to Optimize Your Shape Model? - ShapeWorks), i.e., getting the optimal correspondences (particle positions) on the surfaces of the given 3D shapes, and also retrieve the cost Q = αH(Z) - ΣH(Xn). It would be better if I can use the cost to update the model (the particle positions) in a “PyTorch” way, just like updating a neural network with backpropagation using the loss. (I am not sure what approach is currenly adopted in ShapeWorks for optimization. Gradient descent based on the cost Q or some other method?)
But the current version of ShapeWorks seems to only support doing the optimization either in ShapeWorks Studio (GUI), or via command line. Essentially it’s executing C++ programs, and what we can do is just to specify paramters (number of particles, relative weighting, etc.) in an xml/excel file and pass them to the C++ program. I wonder whether Python code exists for the same task, using which we can create a model instance with specified parameters, and update the model by minimizing the loss function.