ELU activation: A comprehensive analysis - Tung M Phungs Blog Exponential Linear Unit (ELU), proposed by Djork-Arné in 2015, is a variant of the so-called ReLU nonlinearity Through various experiments, ELU is accepted by many researchers as a good successor of the original version (ReLU)