8

I recently committed a large changeset (~7,000 files) to my svn repository. These 7,000 files only make up 5% of the overall size of the repository which uses FSFS backend and served with svnserve 1.7. Ever since, checkouts to revisions after this mega-commit take 20x longer.

What is Subversion doing internally that is causing the slowdown and is there a way to fix this?

Updates

  1. While manually checking out the bad revision I can see the point at which the checkout starts to slow down. The checkout starts be adding files to the working copy very quickly (you can't read the tty output quickly enough). Once the checkout reaches a certain directory (bad revision adds 2,000 files to this directory (which contains 17,000 files already)) files are added to the working copy significantly slower (like 5 files a second) for the rest of the checkout. The revision just before the bad one adds files to the working copy very quickly the entire time. The files in this directory are about 1KB each.

  2. I compiled my own version of svnserve for versions 1.6 and 1.7 --with-debuging and --with-gprof so that we could get some insight into what's going on. Some further poking shows that some enhancements made in svnadmin 1.7 related to in-memory caching are actually killing it at this revision. I.e., serving the repository with svnserve 1.6 makes this problem go away. I'm guessing it's the in-memory caching discussed at http://subversion.apache.org/docs/release-notes/1.7.html#server-performance-tuning based on the gprof profiles for checkout times at at the bad revision (and the one right before it). At rBAD, certain svn fsfs to memory cache functions are called about 2,000,000,000 times more than in rGOOD.

bahrep
  • 26,679
  • 12
  • 95
  • 136
sholsapp
  • 13,679
  • 8
  • 44
  • 62
  • Overall size in filesize or number of files? – John Carter Aug 22 '12 at 01:00
  • I meant in the number of files. – sholsapp Aug 22 '12 at 01:29
  • Yeah, I thought so, but worth clarifying. – John Carter Aug 22 '12 at 01:34
  • My guess is that a directory with 19,000 files might be a bit much. You could try dumping and loading your Subversion database. – Gilbert Le Blanc Aug 24 '12 at 17:48
  • The revision with 17,000 files doesn't take longer, though, which makes me think the number of files in the directory might not actually be the issue. – sholsapp Aug 24 '12 at 19:24
  • 2
    I'm not sure I'd say the caching calls are killing the performance. It may be spending a lot of time there, but perhaps it's saving you on some expensive operation. OTOH, it could be a bug. Have you raised the issue on the [Subversion dev list](mailto:dev@subversion.apache.org)? It might be worth while. I believe Stefan Fuhrmann implemented the caching features, and he's very concerned about performance. – John Szakmeister Oct 02 '12 at 09:47
  • Did you raise the [cache size](http://subversion.apache.org/docs/release-notes/1.7.html#server-performance-tuning)? The default is pretty puny. (Or set it to zero and see if that solves the problem by restoring 1.6 caching behavior.) – David Schwartz Oct 18 '12 at 13:18

1 Answers1

-1

Yes, lot of performance enhancements are rolled-out with SVN v1.7 release. But there are based on some general assumptions. For extreme cases like huge number of files in single commit or files with huge size, one would need to tweak pre-set parameters as specified in relase notes.

rohit
  • 405
  • 1
  • 5
  • 12