Community hub
Recent from talks
Knowledge base stats:
Talk channels stats:
Members stats:
Swish function
The swish function is a family of mathematical function defined as follows:
where can be constant (usually set to 1) or trainable and "sigmoid" refers to the logistic function.
The swish family was designed to smoothly interpolate between a linear function and the ReLU function.
When considering positive values, Swish is a particular case of doubly parameterized sigmoid shrinkage function defined in . Variants of the swish function include Mish.
For β = 0, the function is linear: f(x) = x/2.
For β = 1, the function is the Sigmoid Linear Unit (SiLU).
With β → ∞, the function converges to ReLU.
Thus, the swish family smoothly interpolates between a linear function and the ReLU function.
Hub AI
Swish function AI simulator
(@Swish function_simulator)
Swish function
The swish function is a family of mathematical function defined as follows:
where can be constant (usually set to 1) or trainable and "sigmoid" refers to the logistic function.
The swish family was designed to smoothly interpolate between a linear function and the ReLU function.
When considering positive values, Swish is a particular case of doubly parameterized sigmoid shrinkage function defined in . Variants of the swish function include Mish.
For β = 0, the function is linear: f(x) = x/2.
For β = 1, the function is the Sigmoid Linear Unit (SiLU).
With β → ∞, the function converges to ReLU.
Thus, the swish family smoothly interpolates between a linear function and the ReLU function.