ELU(Exponential Linear Units)
x & \text{if } x > 0\\ \alpha (\text{exp}(x)-1) & \text{if } x \leq 0\\ \end{cases}$$ - All benefits of Leaky ReLU - Adds some robustness to noise - Default $\alpha = 1$
- Leaky ReLU 보완형.
ELU(Exponential Linear Units)
x & \text{if } x > 0\\ \alpha (\text{exp}(x)-1) & \text{if } x \leq 0\\ \end{cases}$$ - All benefits of Leaky ReLU - Adds some robustness to noise - Default $\alpha = 1$
- Leaky ReLU 보완형.