Table 3 Activation function and derivation
From: Artificial Neural network-based prediction of SCF at the Intersection of CFST Y-joints
Activation function | f(x) | f`(x) |
|---|---|---|
Sigmoid | \(f(x)=\frac{1}{1+{\mathrm{e}}^{\hbox{-} x}}\) | \(f(x)=\frac{{\mathrm{e}}^{-x}}{{\left(1+{\mathrm{e}}^{-x}\right)}^2}\) |
Tanh | \(f(x)=\frac{{\mathrm{e}}^x-{\mathrm{e}}^{-x}}{{\mathrm{e}}^x+{\mathrm{e}}^{-x}}\) | \(f(x)=\frac{4}{{\left({\mathrm{e}}^x+{\mathrm{e}}^{-x}\right)}^2}\) |
ReLU | \(f(x)=\left\{\begin{array}{c}0\kern0.5em x<0\\ {}x\begin{array}{cc}& x>0\end{array}\end{array}\right.\) | \(f(x)=\left\{\begin{array}{c}0\begin{array}{cc}& x<0\end{array}\\ {}1\begin{array}{cc}& x>0\end{array}\end{array}\right.\) |