In simple words: Nonlinearity helps a network represent more complicated relationships.
After the weighted sum
An activation function changes a neuron’s calculated value before sending it forward. Without nonlinear activations, stacking ordinary linear layers still behaves like one linear calculation.
Common names
Rectified Linear Unit (ReLU) turns negative values into zero and keeps positive values. Sigmoid maps a number between zero and one. Softmax converts several scores into class probabilities. They serve different purposes; there is no single best function for every task.
See the flow
A model judging an image may combine curves and corners. That pattern is more complex than one straight-line rule.
Remember: Nonlinearity helps a network represent more complicated relationships.