Issue when trying to plot after applying PCA on a dataset

Question

I am trying to plot the results of PCA of the dataset pima-indians-diabetes.csv. My code shows a problem only in the plotting piece:

import numpy
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import pandas as pd

# Dataset Description:
#    1. Number of times pregnant
#    2. Plasma glucose concentration a 2 hours in an oral glucose tolerance test
#    3. Diastolic blood pressure (mm Hg)
#    4. Triceps skin fold thickness (mm)
#    5. 2-Hour serum insulin (mu U/ml)
#    6. Body mass index (weight in kg/(height in m)^2)
#    7. Diabetes pedigree function
#    8. Age (years)
#    9. Class variable (0 or 1)

path = 'pima-indians-diabetes.data.csv'
dataset = numpy.loadtxt(path, delimiter=",")
X = dataset[:,0:8]
Y = dataset[:,8]

features = ['1','2','3','4','5','6','7','8','9']
df = pd.read_csv(path, names=features)

x = df.loc[:, features].values          # Separating out the values
y = df.loc[:,['9']].values              # Separating out the target
x = StandardScaler().fit_transform(x)   # Standardizing the features


pca = PCA(n_components=2)
principalComponents = pca.fit_transform(x)
# principalDf = pd.DataFrame(data=principalComponents, columns=['pca1', 'pca2'])
# finalDf = pd.concat([principalDf, df[['9']]], axis = 1)

plt.figure()
colors = ['navy', 'turquoise', 'darkorange']
lw = 2
for color, i, target_name in zip(colors, [0, 1, 2], ['Negative', 'Positive']):
    plt.scatter(principalComponents[y == i, 0], principalComponents[y == i, 1], color=color, alpha=.8, lw=lw,
                label=target_name)
plt.legend(loc='best', shadow=False, scatterpoints=1)
plt.title('PCA of pima-indians-diabetes Dataset')

The error is located at the following line:

Traceback (most recent call last):
  File "test.py", line 53, in <module>
    plt.scatter(principalComponents[y == i, 0], principalComponents[y == i, 1], color=color, alpha=.8, lw=lw,
IndexError: too many indices for array

Kindly, how to fix this?

jdamp · Accepted Answer · 2019-06-04T08:13:48.713

As the error indicates some kind of shape/dimension mismatch, a good starting point is to check the shapes of the arrays involved in the operation:

principalComponents.shape

yields

(768, 2)

while

(y==i).shape

(768, 1)

Which leads to a shape mismatch when trying to run

principalComponents[y==i, 0]

as the first array is already multidimensional, therefore the error is indicating that you used too many indices for the array.

You can fix this by forcing the shape of y==i to a 1D array ((768,)), e.g. by changing your call to scatter to

    plt.scatter(principalComponents[(y == i).reshape(-1), 0],
                principalComponents[(y == i).reshape(-1), 1],
                color=color, alpha=.8, lw=lw, label=target_name)

which then creates the plot for me

For more information on the difference between arrays of the shape (R, 1)and (R,) this question on StackOverflow provides a nice starting point.

Issue when trying to plot after applying PCA on a dataset

1 Answers1