Indexing a dataframe by date and trouble when using reset_index

Question

I do have a dataframe to calculate daily means of a measurement quantity.

index_date = pd.Index(t, name = 'dates') # t is an array of datetime objects

df = pd.DataFrame(gas, index_date, columns=['A'], dtype=float)

daily_mean = df.resample('D', how='mean')

In order to be able plot the data I reset the index and create numpy arrays

daily_mean.reset_index(inplace=True)

date = np.array(daily_mean['dates'])

means = np.array(daily_mean['A'])

My question or problem now is: The timing is not correct. For example the datetime object datetime.datetime(1979, 8, 27, 10, 0) which is the first object in t transforms to '1979-08-27T01:00:00.0000+0100' in the np.array date. Why does this happen and how to I get the correct time?

Thanks in advance for any ideas or helpful comments.

Answer 1

The time is not 'incorrect', it is just a numpy peculiarity: numpy prints datetime64 values in the local timezone

For example, for me being located in a GMT+01 timezone:

In [24]: np.datetime64('2014-01-01 00:00:00+00:00')
Out[24]: numpy.datetime64('2014-01-01T01:00:00+0100')

But the value itself is not changed. You can eg see this by converting it back to a datetime.datetime:

In [26]: np.datetime64('2014-01-01 00:00:00+00:00').item()
Out[26]: datetime.datetime(2014, 1, 1, 0, 0)

In [27]: print np.datetime64('2014-01-01 00:00:00+00:00').item()
2014-01-01 00:00:00

---

For the plotting part: you can also just do daily_mean['A'].plot()

Answer 2

There is no need for you to convert your data to numpy arrays in order to plot it. just use e.g.

fig, ax = plt.subplots()
ax.plot(daily_mean.index, daily_mean.values)

plt.show()

Why the change happens when you convert the datetime index to a numpy array, I do not know, but it seems to be a difficult question (Converting between datetime, Timestamp and datetime64). When you put the array with dates back into the dataframe, they will be in the correct form again.