Writing columns into a NumPy array

Question

I have initialized a numpy array with zeros for memory management, and I am trying to write data into each column within the loop.

I come from a Matlab background, so my code goes something like:

myArray = np.zeros((250000, 100))

for i in range(100):
    x = np.random.rand(250000) # random just to show behavior
    myArray[:,i] = x

I'm getting a Value Error: could not broadcast input array from shape (250000,1) into shape (250000).

I see that myArray[:,i].shape is (250000,) and I'm not sure how to get to a (250000,1). Matlab does this implicitly.

It's the other way round, you should assign `= x.ravel()`. But since `x` is the same on every loop, you can simply do `myArray[:] = x`. — a_guest, Jul 29 '19 at 21:58
The usual way to reshape an array is to use `array.`[reshape()](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.reshape.html) — G. Anderson, Jul 29 '19 at 21:59
What is `x`? That is the source of your problem. It works fine if `x` is a constant, for example. — Alexander, Jul 29 '19 at 22:05
```myArray[:] = x``` would be great if I wanted each column to be identical (but then, why have a loop?). X is changing dynamically in the loop, but is always the same shape. I could loop through each of the items in x (i.e. having each row and column have its own loop), but there must be a better way. — Michelle Greene, Jul 29 '19 at 22:14
Welcome to SO! There should be no problem running the code you present, as no error comes out to any of us. Both `x` and `myArray[:,i]` are shaped `(250000,)`. Check this post to know more about shapes: https://stackoverflow.com/questions/22053050/difference-between-numpy-array-shape-r-1-and-r — David, Jul 29 '19 at 22:35

hpaulj · Answer 1 · 2019-07-29T23:58:54.327

Your example does not produce the error:

In [120]: arr = np.zeros((4,3))                                                                              
In [121]: arr[:,0] = np.random.rand(4)                                                                       
In [122]: arr                                                                                                
Out[122]: 
array([[0.81792002, 0.        , 0.        ],
       [0.47090337, 0.        , 0.        ],
       [0.20433628, 0.        , 0.        ],
       [0.66201335, 0.        , 0.        ]])

However if you generated a different random array:

In [124]: arr[:,0] = np.random.rand(4,1)                                                                     
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-124-86b2b8982a84> in <module>
----> 1 arr[:,0] = np.random.rand(4,1)

ValueError: could not broadcast input array from shape (4,1) into shape (4)

arr[:,0] produces a 1d array; indexing with a scalar reduces dimensions. MATLAB does that too - with 3d or higher (but it has a 2d lower bound).

The error is produced when trying to fit a (n,1) array into a (n,) slot. Broadcasting rules can add leading dimensions, but trailing ones have to be explicit. In numpy leading dimensions are the outer ones (reverse of MATLAB). So a (n,) can broadcast to (1,n).

Indexing with a list or array preserves dimensions, so this puts a (4,1) into a (4,1):

arr[:,[0]] = np.random.rand(4,1)

and a (3,) fits into a (1,3):

arr[[0],:] = np.random.rand(3)

A (1,n) fits into a (n,) as well:

arr[:,0] = np.random.rand(1,4)

Writing columns into a NumPy array

1 Answers1