CODEWITHSUNDEEP
performancemercurialhgweb
Chris R
Chris R

Reputation: 17926

Why is my hgweb server so slow?

I am serving up access to many mercurial repositories using hgweb, providing them as a collection:

[collections]
/home/me = /home/me/projects

This serves them up at localhost/projects

I have around 30 repositories at that location, in a source tree with a fair number of other, non-mercurial-managed projects.

hgweb is really slow to respond; it takes about 30 seconds to provide a listing at http://localhost/, and about 30 seconds to open a project, making it painful to use this for sharing purposes.

How can I tune this to make it faster?

I'm running on OSX, if it makes a difference.

Upvotes: 6

Views: 2080

Answers (4)

jwsample
jwsample

Reputation: 2411

The problem is probably the server searching recursively for repos during every request. Sounds like you've got a pretty big directory there so this makes sense.

This notation will work with the preferred [paths] attribute but I am not sure about if it will help the [collections] attribute. Try changing to 

[collections]
/home/me = /home/me/projects/*

so it will only search one level down.

Check here for more on the issue: https://www.mercurial-scm.org/wiki/HgWebDirStepByStep

If that doesn't work it definitely will if you change to [paths] and use the * notation.

Upvotes: 1

marcinkuzminski
marcinkuzminski

Reputation: 1771

As an open-source alternative, you can use RhodeCode http://rhodecode.com it's hgweb replacement written entirely in Python.

  • has it's own middleware to handle mercurial protocol request each request can be logged and authenticated + threaded performance unlikely to hgweb
  • full permissions per project read/write/admin access even on mercurial request
  • mako templates lets you customize look and feel of application.
  • diffs annotations and source code all colored by pygments.
  • mercurial branch graph and yui-flot powered graphs
  • admin interface for performing user/permission management as well as repository management.
  • Additional settings for mercurial web, (hooks editable from admin panel !) also manage paths, archive, remote messages
  • backup scripts can do backup of whole app and send it over scp to desired location
  • setup project descriptions and info inside built in db for easy, non file-system operations
  • Full search text on repository codes
  • added cache with invalidation on push/repo managment for high performance and always upto date data.
  • rss / atom feeds, gravatar support
  • based on pylons 1.0 / sqlalchemy

Upvotes: 7

Niall C.
Niall C.

Reputation: 10908

AFAIK, hgweb will scan all subdirectories of the [collections] entry in its configuration file. Since you've got a lot of non-Mercurial directories in there, it has to do a scan of each subdirectory of each of them. In contrast, it can stop scanning at the top level of a directory tree containing a Mercurial repository because it will see the .hg directory there.

If you're using a newer Mercurial (after 1.1, it looks like), try changing the hgweb.config to use a [paths] section instead, and provide explicit entries for each of the Mercurial repositories.

Upvotes: 5

Chris R
Chris R

Reputation: 17926

Following up on Niall's very helpful answer, above, I realized that I needed a tool to maintain this [paths] section. I ended up going with this (which uses configobj by M. Foord.

#!/usr/bin/env python
from __future__ import print_function

import os

from configobj import ConfigObj

hgweb_path = os.path.join(os.path.expanduser("~"), "Library", "Preferences", "hgweb.config")
projects = os.path.join(os.path.expanduser("~"), "projects")

config = ConfigObj(hgweb_path)

paths = []

def add_mercurial(arg, dirname, names):
  if '.hg' in names:
    paths.append(dirname[len(projects) + 1:])

os.path.walk(projects, add_mercurial, None)

config['paths'] = {}

for path in paths:
  config['paths']["projects/" + path] = os.path.join(projects, path)

config.write()

This script is run by OS X's equivalent of cron every 15 minutes and ensures that my hgweb never gets out of date.

Upvotes: 1

Related Questions