Sphene Community Tools

Hello,

dealing with the MultiHostMiddleware, I've noticed that at every request the database is hit to look for the group: such request can be easily shaved off.

Looking at the middleware code I also see the hard work done to look for a match in the SPH_HOST_MIDDLEWARE_URLCONF_MAP and your comment about caching the result.

I attach here two patches: 'cache-urlconf-search' caches the result of a host search into SPH_HOST_MIDDLEWARE_URLCONF_MAP: further calls will avoid the fat "else" branch, which seems me a good win for a line of code.

The patch 'cache-group' adds a 'get_group()' function to the 'community.models' module and avoids a database hit. The cache is invalidated when the group is modified. The function is used by the middleware.

I am attaching these patches here to hear your opinion about them and more generally to check if you are interested in receiving patches as feedback from the implementation of our website. Tell me if you prefer to have them added to the issue tracker... or instead if you are not interested in this kind of tweaks.

Cheers!

hi,

thanks for the patches - i very much welcome any improvements and fixes.. you can keep posting them in this forum for now.. (i one day should write a simple extension to the forum for simple issue tracking.. this would save me the use of google code :) )

to the patches:

cache-urlconf-search.patch: a good idea to simply but it into the settings variable - but if i'm not missing anything it won't restore the urlconf_params - so if i have a named group in the regular expression .. those parameters will be lossed?

cache-group.patch: do you have experience with caching whole django models? i have not yet investigated it fully.. but is the (de)serialization for the cache really seamless - what happens if i change anything on those cached models and then save them back or something? i have until now avoided caching whole models, because i have nowhere found a good documentation if it is actually intended to do so .. (I'm not saying it's bad, just that I haven't investigated it)


thanks for your efforts,
herbert

uhm, I was actually missing the point of having a named group in the regexp, but I think now I got it. Correct me if I'm wrong, but if you want to implement something like http://yellow.bikeshed.com/ (you can replace "yellow" with every valid html color, such as "c0c0c0") then you need to capture the host name. But now I also see that in this case the website can be flooded by requests to random vhosts which would fill the setting indefinitely, so the patch as it is now is probably not a good idea.

Herbert Poul said @ 2009-06-11 08:38:52:

cache-group.patch: do you have experience with caching whole django models? i have not yet investigated it fully.. but is the (de)serialization for the cache really seamless - what happens if i change anything on those cached models and then save them back or something? i have until now avoided caching whole models, because i have nowhere found a good documentation if it is actually intended to do so .. (I'm not saying it's bad, just that I haven't investigated it)

Django cache is safe for all picklable objects, and django objects should be (the django page about caching says "lists of model objects" as something that an be stored in cache).

The patch contains a signal handler: if the model is changed the cache gets invalidated and the fresh model is loaded from the database at the following request.

If a website is deployed as multi-process, the locmem cache could be a problem because, being process specific, only the process updating the group would have the cache invalidated. But being the group mostly a read-only configuration object (if you want to change the name or the base url you are probably changing other hardcoded settings such as the urls or the webserver configuration, so you probably need a website restart anyway) and because you already advise to set up caching for the website, I think that database hit can be easily avoided.

Cheers!