bee6030e31e42a9394ac567da170a89a98d2062f,modeling.py,,gelu,#Any#,265
Before Change
Returns:
`input_tensor` with the GELU activation applied.
cdf = 0.5 * (1.0 + tf.erf(input_tensor / tf.sqrt(2.0)))
return input_tensor * cdf
After Change
Returns:
`x` with the GELU activation applied.
cdf = 0.5 * (1.0 + tf.tanh(
(np.sqrt(2 / np.pi) * (x + 0.044715 * tf.pow(x, 3)))))
return x * cdf
def get_activation(activation_string):
In pattern: SUPERPATTERN
Frequency: 3
Non-data size: 5
Instances
Project Name: google-research/bert
Commit Name: bee6030e31e42a9394ac567da170a89a98d2062f
Time: 2019-02-07
Author: jacobdevlin@google.com
File Name: modeling.py
Class Name:
Method Name: gelu
Project Name: tensorflow/models
Commit Name: a97304d53e873ce5949b486c8fcbf5d4fcab7a3b
Time: 2017-05-03
Author: rsepassi@google.com
File Name: adversarial_text/adversarial_losses.py
Class Name:
Method Name: _scale_l2
Project Name: tensorflow/models
Commit Name: a97304d53e873ce5949b486c8fcbf5d4fcab7a3b
Time: 2017-05-03
Author: rsepassi@google.com
File Name: adversarial_text/layers.py
Class Name: Embedding
Method Name: _normalize