Element-wise ReLU function

Symbolic element-wise ReLU function for LazyTensor objects or standard element-wise ReLU function otherwise.

Usage

relu(x)

Arguments

x

input for relu.default() or relu.LazyTensor().

Value

See value of relu.default() or relu.LazyTensor()

Details

The ReLU function (Fukushima 1975) is defined as follows: relu(x) returns 0 if x < 0, x if x >= 0.

If x is a LazyTensor, see relu.LazyTensor(), else see relu.default().

References

Fukushima K (1975). “Cognitron: A self-organizing multilayered neural network.” Biological Cybernetics, 20(3), 121--136. ISSN 1432-0770, doi:10.1007/BF00342633 .

See also

relu.default(), relu.LazyTensor()

Author

Chloe Serre-Combe, Amelie Vernay

Examples

if (FALSE) {
# Numerical input
relu(4)
relu(-10:10)
# LazyTensor symbolic element-wise square
x <- matrix(runif(150 * 3), 150, 3) # arbitrary R matrix, 150 rows, 3 columns
x_i <- LazyTensor(x, index = 'i')   # creating LazyTensor from matrix x, 
                                    # indexed by 'i'
ReLU_x <- relu(x_i)                 # symbolic matrix, 150 rows and 3 columns
}