Upper Generalization Bounds for Neural Oscillators

Neural oscillators, originating from the second-order ordinary differential equations (ODEs), have shown empirical success in learning the mappings between loads and responses of complex nonlinear structural systems. However, theoretically quantifying the generalization capacity of their neural network architectures remains a significant challenge. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived. Specifically, the generalization error is shown to be bounded by the sum of the neural oscillator's approximation error and the Rademacher complexity of the loss function for learning the neural oscillator. Subsequently, it is demonstrated that the Rademacher complexity can be bounded by the expected supremum of a sub-Gaussian process. Next, the expected supremum of the sub-Gaussian process is bounded using the covering number of a neural oscillator class. Finally, by combining the covering number and the theoretical approximation errors previously derived by the authors, the upper generalization bounds of the neural oscillator are established. Theoretical results reveal that the estimation errors in the PAC generalization bounds exhibit polynomial growth with increasing MLP sizes, thereby avoiding the curse of parametric complexity.