Suds is not reusing cached WSDLs and XSDs, although I expect it to

Question

I'm pretty sure suds is not caching my WSDLs and XSDs like I expect it to. Here's how I know that cached objects are not being used:

1. It takes about 30 seconds to create a client: client = Client(url)
2. The logger entries show consistent digestion of the XSD and WSDL files during the entire 30 seconds
3. Wireshark is showing consistent TCP traffic to the server storing the XSD and WSDL files during the entire 30 seconds
4. I see the files in the cache being updated each time I run my program

I have a small program that creates a suds client, sends a single request, gets the response, then ends. My expectation is that each time I run the program, it should fetch the WSDL and XSD files from the file cache, not from the URLs. Here's why I think that:

1. client.options.cache.duration is set to ('days', 1)
2. client.options.cache.location is set to c:\docume~1\mlin\locals~1\temp\suds and I see the cache files being generated and re-generated each time I run the program
3. For a moment I thought that maybe the cache is not reused between runs of a program, but I don't think a file cache would be used if that were the case, because an in-memory cache would do just fine

Am I misunderstanding how suds caching is supposed to work?

Answer 1

The problem is in the suds library itself. In cache.py, although ObjectCache.get() is always getting a valid file pointer, it's hitting an exception (EOFError) doing pickle.load(fp). When that happens, the file is just downloaded again.

Here's the sequence of events:

DocumentReader.open():

1. Trying http://172.28.50.249/wsdl/billingServices/v3.0/RequestScrubAddress.wsdl
2. Loading ObjectCache 51012453-document
3. Loading pickled object...
4. Exception raised:
5. Got None from cache
6. Downloading... Done
7. Saving FileCache 51012453-document... Done

So it doesn't really matter that the new cache file was saved, because the same thing happens the next time I run. This happens for ALL of WSDL and XSD files.

I fixed that problem by opening the cache file in binary mode when reading and writing. Specifically, the changes I made were in cache.py:

1) In FileCache.put(), change this line:

f = self.open(fn, 'w')

to

f = self.open(fn, 'wb')

2) In FileCache.getf(), change this line:

return self.open(fn)

to

return self.open(fn, 'rb')

I don't know the codebase well enough to know if these changes are safe, but it is pulling the objects from the file cache, the service is still running successfully, and loading the client went from 16 seconds down to 2.5 seconds. Much better if you ask me.

Hopefully this fix, or something similar can be introduced back into the suds main line. I've already sent this to the suds mailing list (fedora-suds-list at redhat dot com).