Disqus - Friends of tswicegood

Re: Configs for nginx and Apache with mod_wsgi | Brett Hoerner's blog

Brett Hoerner — Fri, 10 Oct 2008 11:08:22 -0000

nginx does blocking I/O, so the only way to mitigate that is to use more worker processes (I happen to run on a Dual CPU Dual Core, so I just use 4).

I don't totally understand how nginx works, though. The idea that it uses blocking I/O and yet each worker can be serving X many requests is confusing as hell to me. That's the extent of the English docs, though.

Re: Setting up a baseline Django environment on Ubuntu | Brett Hoerner's blog

Brett Hoerner — Wed, 18 Mar 2009 08:15:04 -0000

Ah, I only did those that way because I'm running 8.04 LTS. I recently did an 8.10 install and agree that much of this is available in apt.

Re: Setting up a baseline Django environment on Ubuntu | Brett Hoerner's blog

Brett Hoerner — Wed, 15 Apr 2009 11:37:52 -0000

Was it actually my feed, or a feed of Google persistent search results? I've had the same issue with a persistent search lately.

Re: Disqus Forks Into Two Products, Launches Revamped Real-Time Comment System

Brett Hoerner — Tue, 25 Aug 2009 14:42:22 -0000

No really, it's fixed.

Re: Disqus Forks Into Two Products, Launches Revamped Real-Time Comment System

Brett Hoerner — Tue, 25 Aug 2009 15:01:51 -0000

You can use it just about anywhere you want.

Re: Disqus version 3: Introducing Disqus Comments and Disqus Profile

Brett Hoerner — Tue, 25 Aug 2009 20:39:23 -0000

Nope, just go into the control panel and change your theme to "Narcissus" to use all of the new features.

Re: Disqus version 3: Introducing Disqus Comments and Disqus Profile

Brett Hoerner — Wed, 26 Aug 2009 22:44:41 -0000

This is for moderators only (people who run their sites): http://disqus.com/comments/

Re: Trust Agents

Brett Hoerner — Mon, 04 Jan 2010 12:25:00 -0000

This post lacks Amazon affiliate links.

Re: /2010/jan/11/hot-django-wsgi-action-announcing-django-wsgi/

Brett Hoerner — Mon, 11 Jan 2010 19:05:04 -0000

I think you mean django_view(my_wsgi_app) at the end there. :)

I think I'll be using this real soon, thanks.

Re: Wired's Hide and Seek

Brett Hoerner — Sun, 07 Feb 2010 19:35:08 -0000

To think I opened Python to solve your puzzle and that's all it said. :(

Re: When you comment on a comment

Brett Hoerner — Wed, 10 Feb 2010 17:48:21 -0000

I know of at least a couple DISQUS developers that follow your blog, so the feeling is mutual. :)

Re: HTC Incredible | Brett Hoerner's blog

Brett Hoerner — Wed, 12 May 2010 11:47:19 -0000

I've had one over-the-air OS update already. It just notified me that one was downloaded and ready to be applied. I assume larger updates might tell you one is online and wait for Wi-Fi, I'm not sure yet. About them actually being released for each phone, sadly that's in the hand of the manufacturer. HTC is building a pretty good reputation, I hope they know this is important.

There isn't *any* software included out of the box. You don't even plugin to setup all your accounts and stuff. I think that's kind of nice, but agree that there's room for a really awesome non-closed iTunes-a-like syncing station. I've always been a manual music manager, but man, having your desktop and phone know where you last left off in that 2 hour podcast is just so nice, I miss it.

Re: HTC Incredible | Brett Hoerner's blog

Brett Hoerner — Wed, 12 May 2010 13:30:51 -0000

Yeah, it's $80ish for the lowest minutes and all that jazz.

Re: The many faces of Ninja Turtles

Brett Hoerner — Wed, 09 Jun 2010 21:06:02 -0000

Phew, I was starting to worry about what they're teaching kids in America's Hat.

Re: Disqus: The Official Blog

Brett Hoerner — Mon, 02 Aug 2010 17:04:37 -0000

If you ask me, it sounds like you guys hire too many people named David.

PERL4LYFE.

Re: Why I love Disqus

Brett Hoerner — Tue, 21 Sep 2010 16:59:31 -0000

And we use Django, too. :)

http://djangocon.blip.tv/fi...

Re: Microsoft & …?

Brett Hoerner — Wed, 29 Sep 2010 11:34:18 -0000

Would this mean they'd have to tell the honest truth? It's a hard pill to swallow.

Or would they just use the engineers to write bullshit posts that would hopefully bypass the tech-nerd filters?

Re: http://joshbohde.com/blog/async-workers-in-django-with-celery

Brett Hoerner — Fri, 11 Feb 2011 13:18:41 -0000

Did you verify that your DB calls are async? If so, how?

psycopg2, being a C extension, is not patched by eventlet.monkey_patch(), unless I'm missing something. We're using true async psycopg2 a little at Disqus, but it requires a lot more work and we don't use the Django ORM.

Re: http://joshbohde.com/blog/async-workers-in-django-with-celery

Brett Hoerner — Fri, 11 Feb 2011 13:35:07 -0000

Hmm, have you used this under load? I read the psycopg2 documentation when I was working on this, and it states: "Psycopg connections are not green thread safe and can’t be used concurrently by different green threads."

So isn't django.db.connections['default'] (etc) a single DB connection? GreenThread #1 starts a query, then yields control. GreenThread #2 gets control, and starts a query with the same connection, problems ensue? Am I missing something?

Re: http://joshbohde.com/blog/async-workers-in-django-with-celery

Brett Hoerner — Fri, 11 Feb 2011 14:44:54 -0000

Ah, I see.

I'm using eventlet.db_pool with an unpatched psycopg2. That gives me a pool of connections are re-used (and thus capped, lower than the number of GreenThreads). For example, I might have 1000 GreenThreads but only 4 DB connections they take turns using, because the real block is some HTTP APIs we hit, and not our DB (nor would we want 1000 workers hammering the DB at once).

My thought for the ORM was that I could monkey patch django.db.connections to fetch from this pool. For now I just use async with raw queries, and workers that use the ORM still use normal single threaded forked processes.

Re: Yeehaw!

Brett Hoerner — Wed, 16 Feb 2011 16:26:00 -0000

New homepage.

Re: Redis at Disqus | Brett Hoerner's blog

Brett Hoerner — Mon, 21 Feb 2011 18:25:50 -0000

It's not easy, actually. The short answer is that we don't add capacity (because we've only needed to once, and we have tons of room to grow now). The long answer is that I have a switch I can flip that starts incrementing/adding data to a whole new cluster of Redis nodes while it still updates the old ones. We can then backfill all data to the new nodes and when they're setup, flip a switch to read/write only from/to the new nodes. It may sound a bit weird, mostly because it is. Moving sets of random keys from one node to the other while you're expecting live reads/writes is a huge pain, so I just punted on the problem.

It really helps to plan ahead here. What's nice about Redis is that each running (software) server is very, very cheap. When we had to expand the first time, we just put up way more nodes per server than we expected to need. Say you have 5 servers and put 8 nodes on each. They're all splitting the server's resources by 1/8 (effectively), but your code can logically treat them like 40 servers. Now as they grow you can just move those existing (logical, software) servers to their own hardware without changing your hashing at all. In other words, you'd have to fill 40 physical servers worth of RAM until you really feel more pain about having to add new nodes.

Re: Redis at Disqus | Brett Hoerner's blog

Brett Hoerner — Mon, 21 Feb 2011 19:05:23 -0000

There's a lot more discussion on Hacker News: http://news.ycombinator.com...

Re: Redis at Disqus | Brett Hoerner's blog

Brett Hoerner — Tue, 22 Feb 2011 14:20:12 -0000

We use Flot and some other tools. My coworker DZ replied in more detail here: http://news.ycombinator.com...

Re: Redis at Disqus | Brett Hoerner's blog

Brett Hoerner — Tue, 22 Feb 2011 14:25:06 -0000

Sure, but we actually haven't had to tune much of anything. This is random conf from one of our shards, apologies for random order, I just grepped out the default comments and sorted/uniq'd it: https://gist.github.com/839193