Difference between a -= b and a = a - b in Python

I have recently applied this solution for averaging every N rows of matrix. Although the solution works in general I had problems when applied to a 7x1 array. I have noticed that the problem is when using the -= operator. To make a small example:

    import numpy as np

    a = np.array([1,2,3])
    b = np.copy(a)

    a[1:] -= a[:-1]
    b[1:] = b[1:] - b[:-1]

    print a
    print b

which outputs:

    [1 1 2]
    [1 1 1]

So, in the case of an array a -= b produces a different result than a = a - b. I thought until now that these two ways are exactly the same. What is the difference?

How come the method I am mentioning for summing every N rows in a matrix is working e.g. for a 7x4 matrix but not for a 7x1 array?

Note: using in-place operations on NumPy arrays that share memory in no longer a problem in version 1.13.0 onward (see detailshere). The two operation will produce the same result. This answer only applies to earlier versions of NumPy.

Mutating arrays while they're being used in computations can lead to unexpected results!

In the example in the question, subtraction with -= modifies the second element of a and then immediately uses that modified second element in the operation on the third element of a.

Here is what happens with a[1:] -= a[:-1] step by step:

  • a is the array with the data [1, 2, 3].

  • We have two views onto this data: a[1:] is [2, 3], and a[:-1] is [1, 2].

  • The in-place subtraction -= begins. The first element of a[:-1], 1, is subtracted from the first element of a[1:]. This has modified a to be [1, 1, 3]. Now we have that a[1:] is a view of the data [1, 3], and a[:-1] is a view of the data [1, 1] (the second element of array a has been changed).

  • a[:-1] is now [1, 1] and NumPy must now subtract its second element which is 1 (not 2 anymore!) from the second element of a[1:]. This makes a[1:] a view of the values [1, 2].

  • a is now an array with the values [1, 1, 2].

b[1:] = b[1:] - b[:-1] does not have this problem because b[1:] - b[:-1] creates a new array first and then assigns the values in this array to b[1:]. It does not modify b itself during the subtraction, so the views b[1:] and b[:-1] do not change.

The general advice is to avoid modifying one view inplace with another if they overlap. This includes the operators -=, *=, etc. and using the out parameter in universal functions (like np.subtract and np.multiply) to write back to one of the arrays.

From: stackoverflow.com/q/35036126

Back to homepage or read more recommendations: