Why is (2^31) >> 32 not 0?

Question

My problem is that

np.array([2**31], dtype=np.uint32) >> 32

does not return 0, but returns array([2147483648], dtype=uint32) instead. The same is true for

np.right_shift(np.array([2**31], dtype=np.uint32), 32)

(so I believe this is simply how >> is implemented).

Interestingly, all these alternatives seem to work as expected, returning some kind of 0:

print(
    2**31 >> 32,
    np.uint32(2**31) >> 32,
    np.array(2**31, dtype=np.uint32) >> 32,
    np.right_shift(2**31, 32),
    np.right_shift([2**31], 32),
    np.right_shift(np.uint32(2**31), 32),
    np.right_shift(np.array(2**31, dtype=np.uint32), 32),
)

In particular, what is different between Numpy arrays representing 2147483648 and [2147483648]?

I have seen this issue in JavaScript (Why does << 32 not result in 0 in javascript?) and C++ (Weird behavior of right shift operator (1 >> 32), Why is `int >> 32` not always zero?), but not yet in Python/Numpy. In fact, neither Python nor Numpy docs seem to be documenting this behavior:

• https://docs.python.org/3/reference/expressions.html#shifting-operations

• https://docs.scipy.org/doc/numpy/reference/generated/numpy.right_shift.html

Answer 1

While not documented, numpy is mostly implemented in C and the shift operator in C (and C++) is not defined for shifts greater than or equal to the number of bits. So the result can be arbitrary.

If you look at the types of the examples that work you'll see why they work:

print(
    type(2**31 >> 32),
    type(np.uint32(2**31) >> 32),
    type(np.array(2**31, dtype=np.uint32) >> 32),
    type(np.right_shift(2**31, 32)),
    np.right_shift([2**31], 32).dtype,
    type(np.right_shift(np.uint32(2**31), 32)),
    type(np.right_shift(np.array(2**31, dtype=np.uint32), 32)),
)

<class 'int'> <class 'numpy.int64'> <class 'numpy.int64'> <class 'numpy.int64'> int64 <class 'numpy.int64'> <class 'numpy.int64'>

The first uses Python's own int type, while the others are all converted to numpy.int64, where the behavior for a 32-bit shift is correct. This is mostly due to the fact that scalar (zero-dimensional) arrays behave differently. And in the list case that the default integer type for numpy is not numpy.uint32.

On the other hand

print((np.array([2**31], dtype=np.uint32) >> 32).dtype)

uint32

So you run into the undefined behavior here.