Hello!
The comments on this profile are unclaimed and thus are unverified.
Do they belong to you? Claim these comments.
Jean-Paul
Is this you? Claim Profile »
4 months ago
in http://brizzled.clapper.org/id/88 on Brizzled
I quickly hacked (~3 line diff, but it was definitely a hack) your Twisted web server version to use 3 cores on a 4 core machine (leaving the last to ab). Performance went from ~1400r/s to ~6000r/s. I assume the main advantage the Scala version has over the rest is that it's actually exploiting most of the available hardware than any of the Python versions, so this change brings the Twisted web version onto even footing with it. :) I realize this isn't in the spirit of your "naive approach" servers, but I thought I'd point it out in case you hadn't considered just using more processes instead of switching all of your development to a new language. This should be easy, since your case seems to involve little or no state shared between requests; lots of Twisted is geared towards the opposite, where it's a bit harder to split things up into different processes.
1 reply
Brian Clapper
Multiple processes is definitely one of the ways we've been scaling--though in our production servers, it's not as easily accomplished (due to some shared state) as it is with these brain-dead test servers of mine. And you're right: Converting all of our code to a new language isn't a palatable idea at all. Much as I would love to be working in Scala, it's difficult to justify a big code conversion.
6 months ago
in multiprocessing.Pool and KeyboardInterrupt on jessenoller.com comments
Why doesn't apply_async have a means for receiving notification of errors from the child process? This isn't just related to C-c, right? Any exception raised in the child will cause the process to go into an unmanageable state - that seems like a terrible way to deal with errors.
2 replies
