0

I have some pandas code that works fine in Python 2.7. I recently upgraded to Python 3.8 (on a new laptop). Both reading large csv-files and merging 'sizable' dataframes now gives:

"MemoryError: Unable to allocate 322. MiB for an array with shape (92, 459319) and data type float64".

I have 32 GB of RAM, so that should not be the issue. And I even got this error for a small df, where Python complained it could not allocate 1.4 MiB!! When I use the exact same code in 2.7 on my old laptop, it runs like a charm.

Do you know if I can allocate more memory to avoid these errors? I read that Python allocates memory dynamically, so that would tell me there is no way to increase this. If that is the case, I would be forced to move back to 2.7.

Thanks in advance for your help.

Erjen
  • 11
  • 4
  • Which operating system? Python 32bit or 64bit? Version of pandas (on 2.7 and 3.8)? – Matthias Apr 05 '20 at 19:34
  • And you might want to read: [How do I fix memory allocation problem with pandas?](https://stackoverflow.com/questions/60259246/how-do-i-fix-memory-allocation-problem-with-pandas) – Matthias Apr 05 '20 at 19:39
  • That's it! I'm not sure why I even installed the 32-bit version, but apparently I did. I'm surprised the 32-bit version sometimes can't deal with dataframes that are only a few MiB. I replaced the 32-bit installation with the 64-version and now it works again. Thanks for the tip! – Erjen Apr 06 '20 at 07:53

0 Answers0