Softmax as a Neural Networks Activation Function - Sefik Ilkin Serengil

(a) (5pts) Prove that softmax is invariant to constant offset in the input, that is, for any input vector x and any constant c,softmax(x) = softmax(x c)where x c means adding the constant c to every dimension of x. Remember thatsoftmax(x)i = (e^xi)/(e^xj)Note: In practice, we make use of this property and choose c = - maxi xi when computing softmax probabilities for numerical stability (i.e., subtracting its maximum element from all elements of x).