Lost important .py file (overwritten as 0byte file), but the old version still LOADED IN IPYTHON as module -- can it be retrieved?

Question

While managing several different screen sessions with vim open in many of them, in the process of trying to "organize" my sessions I somehow managed to overwrite a very important .py script with a 0Byte file.

However, I have an ipython instance open that, when running that same .py file as a module, still remembers the code that used to be there!

So did I just learn a hard lesson about backups (my last one was done by vim about a week ago, which would leave me with a lot of work to do), or is there any possible, conceivable way to retrieve the .py file from an already loaded module? I probably deserve this for being so cavalier, but I'm seriously desperate here.

Answer 1

As noted in comments, inspect.getsource will not work because it depends on the original file (ie, module.__file__).

Best option: check to see if there's a .pyc file (ex, foo.pyc should be beside foo.py). If there is, you can use Decompile Python 2.7 .pyc to decompile it.

The inspect modules also caches the source. You may be able to get lucky and use inspect.getsource(module), or inspect.getsourcelines(module.function) if it has been called in the past.

Otherwise you'll need to rebuild the module "manually" by inspecting the exports (ie, module.__globals__). Constants and whatnot are obvious, and for functions you can use func.func_name to get its name, func.__doc__ to get the docstring, inspect.getargspec(func) to get the arguments, and func.func_code to get details about the code: co_firstlineno will get the line number, then co_code will get the code. There's more on decompiling that here: Exploring and decompiling python bytecode

For example, to use uncompyle2:

>>> def foo():
...     print "Hello, world!"
...
>>> from StringIO import StringIO
>>> import uncompyle2
>>> out = StringIO()
>>> uncompyle2.uncompyle("2.7", foo.func_code, out=out)
>>> print out.getvalue()
print 'Hello, world!'

But, no — I'm not aware of any more straight forward method to take a module and get the source code back out.

Answer 2

With the process still running, you can look to your namespace to find candidates to restore:

>>> dir()
['__builtins__', '__doc__', '__name__', '__package__', 'readline', 'rlcompleter', 'test']

Let's peek at what we have in store for test:

>>> help(test)

Help on module test:

NAME
    test

FILE
    /Users/tfisher/code/ffi4wd/test.py

FUNCTIONS
    call_cat(cat)

DATA
    cat_name = 'commander sprinkles'

Which has cleaner output than looking at the locals inside of test:

>>> dir(test)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'call_cat', 'cat_name', 'json']

Using the inspect module, we can get the argument specifications for functions:

>>> inspect.getargspec(test.call_cat)
ArgSpec(args=['cat'], varargs=None, keywords=None, defaults=None)

or the lines inside our functions:

>>> inspect.getsourcelines(test.call_cat)
(['def call_cat(cat):\n', '    print("Hello %s" % cat)\n'], 5)

Which is reasonably close to the original:

import json

cat_name = 'commander sprinkles'

def call_cat(cat):
    print("Hello %s" % cat)

Which should work if the file gets deleted after being imported and hasn't been replaced by a file of the same name that's newer (getsourcelines uses the object cache if possible):

$ python -V
Python 2.7.10

$ ls | grep test
$

Answer 3

You should be able to use inspect

import inspect in your Ipython session, and, assuming you are trying to recover myModule, do:

q = inspect.getsource(myModule)

and write q to a file.

[Edit] This worked for me simulating the problem using Python 2.7.6, IPython 1.2.1

[Edit #2]