Numpy - Softmax

Soft-max function by Numpy

Numpy

---

---

<img width=”509” height=”318” alt=”Image” src=”” {: .align-center width=”400”}

Reference:
Logistic and Soft-max Regression

Activation Funcition

Base Code

exp_x = np.exp(x)
softmax = exp_x / np.sum(exp_x)

For Numerical Stability

np.exp(x - np.max(x))

Elaborate

np.exp(x) # Exponentiate the value
exp(x) / sum(exp(x)) # Normalize by dividing by total sum
exp(x - max(x)) / sum(...) # Prevent overflow and Maintain numerical stability

Pytorch

torch.softmax(tensor, dim=0)