9

I have some data that Im reading from a h5 file as a numpy array and am doing some analysis with. For context, the data plots a spectral response curve. I am indexing the data (and a subsequent array I have made for my x axis) to get a specific value or range of values. Im not doing anything complex and even the little maths I'm doing is pretty basic. However I get the following warning error in a number of places

"VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 44 but corresponding boolean dimension is 17"

even though the output I get is the correct one when I check it.

Can someone explain what this warning means and whether I need to be more concerned about it than I currently am?

Im not sure example code would shed much light on this, but seeing as it is a warning that occurs when I index and slice arrays, here is some anyway:

data = h5py.File(file,'r')
dset = data['/DATA/DATA/'][:]
vals1 = dset[0]

AVIRIS = numpy.linspace(346.2995778, 2505.0363678, 432)
AVIRIS1 = AVIRIS[vals1>0]
AVIRIS1 = AVIRIS[vals1<1]
Nathan Thomas
  • 1,344
  • 6
  • 21
  • 44
  • In your example, `AVIRIS` has 432 elements (`AVIRIS.shape[0]`). What's the shape of `vals1`? I'm guessing less than 432. And what version of `numpy` are you running? – hpaulj Feb 17 '16 at 19:40
  • Indeed. Vals1 originally has 432 but then I slice it down to 385. An error brought on by reusing variable names then? – Nathan Thomas Feb 17 '16 at 19:47
  • Or correct the shape of `AVIRIS`. – hpaulj Feb 17 '16 at 19:52

2 Answers2

15

Previous questions on this warning:

VisibleDeprecationWarning: boolean index did not match indexed array along dimension 1; dimension is 2 but corresponding boolean dimension is 1

https://stackoverflow.com/a/34296620/901925

I think this is something new in numpy 1.10, and is the result of using boolean index that is shorter than array. I don't have that version installed so can't give an example. But in an earlier numpy

In [667]: x=np.arange(10)
In [668]: ind=np.array([1,0,0,1],bool)
In [669]: ind
Out[669]: array([ True, False, False,  True], dtype=bool)
In [670]: x[ind]
Out[670]: array([0, 3])

runs ok, even though ind is shorter than x. It effectively pads ind with False. I think newer versions continue to do the calculation, but issue this warning. I need to find a commit that changed this or a SO question that discusses it.

It is possible to suppress warnings - see the side bar. But you really should check the shape of the offending arrays. Do they match, or is the boolean index too short? Can you correct that?

Github discussion

https://github.com/numpy/numpy/issues/4980 Boolean array indexing fails silently #4980

Pull request

https://github.com/numpy/numpy/pull/4353 DEP: Deprecate boolean array indices with non-matching shape #4353

To suppress the warning use something like:

import warnings
warnings.filterwarnings("ignore", category=np.VisibleDeprecationWarning)

you may have to tweak the category name to get it right.

Community
  • 1
  • 1
hpaulj
  • 175,871
  • 13
  • 170
  • 282
  • Thanks this is useful. As everything works I am trying hard to resist the temptation to ignore the warnings. Im am trying to match the shapes of my arrays now – Nathan Thomas Feb 17 '16 at 20:08
  • 1
    Took the time to sort all the array shapes out and its no longer giving me warnings - just needed to be sure what their cause was – Nathan Thomas Feb 17 '16 at 21:56
  • Helpful answer! Here is a [related question](http://stackoverflow.com/q/41310108/3904031) about `VisibleDeprecationWarning`. – uhoh Dec 24 '16 at 03:44
3

To suppress the warning you can:

  1. (evil)

add something like this to your .bashrc or whereever you set environmental variables to turn off visible deprecation warnings globally:

export PYTHONWARNINGS="ignore::DeprecationWarning:simplejson"

  1. (bad)

Turn of warnings when running a single script:

python -W ignore thisbetterworks.py

  1. (okayish)

Run a block without warnings:

import warnings with warnings.catch_warnings(): warnings.warn("Let this be your last warning") warnings.simplefilter("ignore") < your code >

Of course you do run the risk of this failing when deprecation turns to absence, so you may want to make sure it doesn't end up in long-term code.

Matthias Winkelmann
  • 13,735
  • 5
  • 55
  • 67