Help solving a geometrical matching issue with Graph Neural Networks

1 reply · 4K views · last reply
lauripro56
Messages
1
Reaction score
0
TL;DR
I wish to understand which lines and vertices in different 2D orthographic views of a 3D object correspond to each other. This information would also later be used to construct a 3D model from the 2D orthographic views. I wish to utilize graphs and graph neural networks.
Hello!

I wish to understand which lines and vertices in different 2D orthographic views of a 3D object correspond to each other. This information would also later be used to construct a 3D model from the 2D orthographic views.
Orthographic mechanical drawing of a stepped block with blue and orange arrows linking matching edges and vertices across the front and side views; dimensions 3.00, 2.00, 5.00, 10.00, 3.00 and 10.00 shown, with a 0.3 chamfer note.

Blue shows matched edges/lines. Orange shows matched nodes/vertices.
Mechanical engineering drawing of a flanged cylinder: front view with a 30.00 diameter outer circle, 15.00 inner circle, four 4.00 holes and a 10.00 dimension; side view with 100.00 length and 20.00 height. Orange matching marks link vertices.

Circular objects seem especially difficult.​
So far it seems like it would be sensible to use a graph neural network to solve this task. Initial ideas, structure, features are as follows (general, more certain):
  • Each vertex is a node in the graph
    • Node feature vector would include the x-y coordinates relative to the view
  • Each line on the drawing is an edge between nodes in the graph
    • Edge feature vector would include:
      • Edge type (in addition to straight lines there are are also circles and arcs)
      • Edge length
      • If dimension text is defined next to the edge (this is a mechanical engineering drawing related property, with the importance being that equivalent edges in a mechanical engineering drawings should have the length defined for them only once)
Do you have any suggestions for the following:
  • What network architecture(s) would be worth a try?
  • Should a hierarchical graph structure (and GNN) be used?
    • A hypernode representing the entire view, which is connected to all other nodes in the view
    • A global node connected to all hypernodes, in order to capture the relation between different views
Left: a multigraph drawing with blue, red and green edges connecting circular nodes, with a legend for the three edge types. Right: a hierarchy diagram with a global node, hypernodes, and nodes connected by dashed edges, with shaded groups.

Schematic of more complex graphs. (https://distill.pub/2021/gnn-intro/)
  • Any thoughts about other relevant edge, node and potentially global features?
  • How would You define this task? Is it link prediction, node classification, graph matching, etc.?
    • This task can probably be approached in many different ways, what seems logical to You?
  • Engineering drawings often also contain an isometric view, could this be relevant somehow?
    • Notice that an entirely isometric view dependent solution does not work for all drawings then, however it could be still relevant if works with high accuracy or does not require too much “side-tracking”.
Feel free to ask any additional questions or engage in discussion (some more uncertain ideas left out to not cause unnecessary confusion / make the post too long).

Thanks for any help!