It is widely recognized that the performance of neural networks depends on their architecture 1. On one hand, the design of these structures is inspired by biological neural networks. For instance, ...

The solutions found by neural networks to solve a task are often inscrutable. We have little insight into why a particular structure emerges in a network. By reverse engineering neural networks from ...