Maxout

Maxout

• Generalizes ReLU and Leaky ReLU
  $g(x)=\text{max}({\mathrm{a}}_1^{\top }\mathrm{x}+b_1,{\mathrm{a}}_2^{\top }\mathrm{x}+b_2)$
• Increases the number of parameters per neuron