0

I'm trying to cache a python method's output, so for the cache key I'm using hash(method) + the serialized args. I'm using a single memcached server for multiple worker machines.

The problem is, hash(method) has proven inconsistent across these worker machines and processes.

class Foo(object):
    def bar():
       pass

x = Foo()
hash(x.bar) #was inconsistent across machines/processes

id() won't work because that's a memory location and thus works on only one machine.

Thanks!

djs22
  • 1,068
  • 3
  • 13
  • 28
  • @delnan: Oh, we're hashing functions... Nvm – Niklas B. Mar 20 '12 at 20:38
  • @NiklasB. what do you recommend I take the MD5 hash of? The function takes a string input... – djs22 Mar 20 '12 at 20:40
  • djs22: Arguments and some kind of fully-qualified function name. This is a solved problem already, though, just look at the links provided in the second answer. – Niklas B. Mar 20 '12 at 20:42
  • Maybe either hashing the fully qualified name (as linked in a comment below) or using the results of inspect.getsource() would suffice. – DSM Mar 20 '12 at 20:42

3 Answers3

1

I would use the name of the function instead, like x.foo.__name__.

guidoism
  • 7,030
  • 8
  • 37
  • 58
  • Yes but what about functions with the same name in different classes/modules? – djs22 Mar 20 '12 at 20:39
  • @djs22: Obviously you should include the module/class name too in that case. – Niklas B. Mar 20 '12 at 20:41
  • Don't give functions the same name if you'll be hashing them based on their names? – kindall Mar 20 '12 at 20:41
  • 1
    @djs22: add the class/module/package name: http://stackoverflow.com/questions/2020014/get-fully-qualified-class-name-of-an-object-in-python – dgorissen Mar 20 '12 at 20:41
  • Even then it's prone to causing problems, especially with smarter code (any kind of metaprogramming, for instance). It's also arguably dirty to use such a decidedly collision-prone key. –  Mar 20 '12 at 20:43
  • @NiklasB., in the situation I'm using this (inside a decorator), it's not easy to access the module/class name. I used the code linked to in the accepted answer to learn how to do this. – djs22 Mar 21 '12 at 00:22
1

Have you looked at one the the avialable memcached decorators?

dgorissen
  • 5,951
  • 3
  • 36
  • 52
  • Thanks dgorissen! I ended up using the code from 1stvamp's github as a base to write properly implemented fully qualified function names. – djs22 Mar 21 '12 at 00:20
1

Setup a name mangling function that will take the class name and method name to create a string and then take the hash for the string. For example let x be the method then

def method_hash(x):
    return hash("{0}.{1}".format(x.__objclass__.__name__, x.__name__))

This assumes that the method is from the class and not a specific object. If from the object then x.__self__.__name__ could be used instead.

Lance Helsten
  • 7,753
  • 3
  • 14
  • 15
  • The `__class__` of functions is ``, at least in 3.x (it *may* work in 2.x with its "unbound methods" but I wouldn't bet on it). –  Mar 20 '12 at 20:49
  • @delnan Thanks for catching my error on `__class__`, I edited my answer to handle it. – Lance Helsten Mar 20 '12 at 21:26
  • A good answer, but due to reason's not specified in the original question (I need to generate this unique identifier within a decorator), it's not always easy to tell if you're in a class/object/or just module. – djs22 Mar 21 '12 at 00:24