Skip to main content

All Questions

Tagged with
Filter by
Sorted by
Tagged with
237 votes
9 answers
293k views

What are the advantages of ReLU over sigmoid function in deep neural networks?

The state of the art of non-linearity is to use rectified linear units (ReLU) instead of sigmoid function in deep neural network. What are the advantages? I know that training a network when ReLU is ...
RockTheStar's user avatar
  • 13.1k