Easier CAD through DL+Search/Optimization

CAD modeling is a procedural modeling task, where CAD designers compose geometric primitives through operations in the modeling toolbox, to produce the final shapes. Equivalently, CAD modeling is programming in a domain-specific language (DSL) that specifies the set of geometric primitives and operations. Programming directly in this low-level DSL is not easy, just like programming with assembly language is not easy. We develop tools to facilitate CAD modeling, with the overall idea of providing more intuitive higher-level languages/interfaces that abstract away the construction details and can still be faithfully translated into the specific implementations.

Toward the above goal, we have developed two Sketch-to-CAD systems (Sketch2CAD, Free2CAD), where the higher-level interface is freehand sketch in 2D. We have also developed a ComplexGen framework that reconstructs B-Rep structures directly from point clouds containing scattered points without any additional structure. The B-Rep is more accessible than points, and foundational for higher-level CAD like recovering operations.

Technically, the mappings from sketches to CAD ops or from points to B-Reps both involve recovering structures from unstructured data. Indeed, most computational tasks that are worthwhile involve searching for meaningful structures from unstructured/corrupted/noisy data. In our challenge, the target structures are very algebraic, e.g. the CAD ops have types and parameters, and the B-Reps contain vertices, edges, faces that consistently connect into chain complexes. Such algebraic structures have strict validity requirements, and their correspondences with the unstructured data consist of input segmentation and much more. We model such structure extraction problems as translation tasks done through the flexible transformer neural networks, and further ensure their algebraic validity through constrained optimizations of various flavors (online fitting for sketch-to-CAD, global integer linear optimization for ComplexGen).

Through these projects we have demonstrated the power of hybrid computation containing both neural networks and classical search/optimization, where the NN learning part addresses the hard problems of translating between very different domains and providing rough guesses, and the search/optimization part utilizes these guesses and finalizes the proper solutions. Such a paradigm should be helpful for more general questions than CAD.

On the other hand, clearly the goal of enhanced programming is only touched by these projects. There are many more questions to resolve with developing more efficient ways of (assisted) programming, for CAD and many other domains that can be formalized computationally.