Analisi di sensibilità in reti neurali profonde

A seguito di una domanda a cui è già stata data risposta ( Estrazione dell'importanza del peso dalla rete di feed-forward a un livello ) Sto cercando un'inferenza sulla pertinenza degli input nelle reti neurali.

Considerando una rete profonda, dove ricostruire l'importanza dell'input andando indietro attraverso gli strati dal nodo di output di interesse può essere difficile o richiedere tempo, mi chiedevo se ci fosse un quadro teorico nell'esecuzione dell'analisi di sensibilità per la rete neurale, sostanzialmente cambiando leggermente un input e considerare come cambia il nodo di interesse di fuoriuscita.

Esiste un modo canonico per eseguire una sorta di analisi di sensibilità nelle reti neurali?

Gradirei davvero un po 'di codice Python per farlo, se ce ne fosse

— Tommaso Guerrini
fonte

L'analisi di sensibilità suggerita corrisponde all'esame delle derivate parziali degli output rispetto agli input. Pronuncia il vettore di output è dato da , dove è il vettore di input e è la funzione implementata dalla rete. Il giacobino delle uscite contro gli ingressi è: $y \in \mathbb{R}^m$ $y= f(x)$ $x \in \mathbb{R}^d$ $f$

J_{i j} (x) = \frac{\partial}{\partial x_{j}} f_{i} (x)

$J_{ij}(x) = \frac{\partial}{\partial x_j} f_i(x)$

The Jacobian gives the local rate of change of each output w.r.t. each input, so it tells us how $f$ will behave in response to infinitesimal perturbations. If we start with input $x$ and add an infinitesimal value $\Delta$ to the $j$ th input, we expect the $i$ th output to increase by $\Delta J_{ij}(x)$ .

If $J_{ij}(x)$ has large magnitude, it means that output $i$ is sensitive to input $j$ in the vicinity of $x$ . Because $f$ is, in general, nonlinear, this notion of sensitivity depends on the input; it may be large in some regions and near zero in others. If you want some kind of summary measure of how strongly the outputs depend on the inputs, you'd have to aggregate over multiple input values. For example, you could take the absolute value of the Jacobian, averaged over all inputs in the training set (which acts as a surrogate for the expected value w.r.t. the underlying distribution of inputs). Of course, this kind of summary will end up discarding information, so could be misleading in some circumstances.

You can use the chain rule to derive an expression for the Jacobian, similarly to how you'd derive the gradient of the loss function w.r.t. the parameters for use with backprop. You can also compute it using automatic differentiation, using a library like Theano, TensorFlow, etc. There's not much reason to perform finite differencing (i.e. actually simulate the perturbation and measure the change in output), unless the function your network implements is nondifferentiable (in which case the Jacobian doesn't exist).

A couple caveats: If the inputs have different units/scales than each other, the sensitivities will also have different units/scales, and can't be directly compared. Standardizing/scaling the inputs is one possible solution. It's also important to keep in mind is that this type of analysis tells us about the model itself, but not necessarily the underlying distribution that generated the data. For example, if two inputs are correlated, the model might end up using the first but not the second. In this case, we'd find that the sensitivity is high for the first input and low for the second, but should not conclude that the first input is inherently more important for predicting the output in general.

This article should be of interest.

— user20160
fonte

great answer and great article! if anyone is interested in implementing this method, you can find a nice implementation of the jacobian calculation here: medium.com/unit8-machine-learning-publication/…

— pcko1