# How to add regularizations in TensorFlow?

I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.

My q are:

Is there a more elegant or recommended way of regularization than doing it manually?

I also find that

`get_variable`

has an argument`regularizer`

. How should it be used? According to my observation, if we pass a regularizer to it (such as`tf.contrib.layers.l2_regularizer`

, a tensor representing regularized term will be computed and added to a graph collection named`tf.GraphKeys.REGULARIZATOIN_LOSSES`

. Will that collection be automatically used by TensorFlow (e.g. used by optimizers when training)? Or is it expected that I should use that collection by myself?

As you say in the second point, using the `regularizer`

argument is the recommended way. You can use it in `get_variable`

, or set it once in your `variable_scope`

and have all your variables regularized.

The losses are collected in the graph, and you need to manually add them to your cost function like this.

```
reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
reg_constant = 0.01 # Choose an appropriate one.
loss = my_normal_loss + reg_constant * sum(reg_losses)
```

Hope that helps!

From: stackoverflow.com/q/37107223

**★ Back to homepage or read more recommendations:**