Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification - LAAS - Laboratoire d'Analyse et d'Architecture des Systèmes Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification

Résumé

This paper is concerned with the computation of the local Lipschitz constant of feedforward neural networks (FNNs) with activation functions being rectified linear units (ReLUs). The local Lipschitz constant of an FNN for a target input is a reasonable measure for its quantitative evaluation of the reliability. By following a standard procedure using multipliers that capture the behavior of ReLUs,we first reduce the upper bound computation problem of the local Lipschitz constant into a semidefinite programming problem (SDP). Here we newly introduce copositive multipliers to capture the ReLU behavior accurately. Then, by considering the dual of the SDP for the upper bound computation, we second derive a viable test to conclude the exactness of the computed upper bound. However, these SDPs are intractable for practical FNNs with hundreds of ReLUs. To address this issue, we further propose a method to construct a reduced order model whose input-output property is identical to the original FNN over a neighborhood of the target input. We finally illustrate the effectiveness of the model reduction and exactness verification methods with numerical examples of practical FNNs.

Dates et versions

hal-04247204 , version 1 (18-10-2023)

Identifiants

Citer

Yoshio Ebihara, Xin Dai, Victor Magron, Dimitri Peaucelle, Sophie Tarbouriech. Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification. 2023. ⟨hal-04247204⟩
77 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More