DISQUS

Rob,
A number of things have changed since your post on Mashable. Our data quality is improving considerably as we've eliminated a lot of the spam that had infiltrated the index. The index entry barrier is a higher, which is a mixed blessing; less spam is good, missing some legit blogs is bad but we're continuously improving the index quality as well as the coverage. The tag pages rolled out in last month's site update have restored the separation between tag aggregation and keyword search. The changes the year prior that mingled the two were mistaken but we're adapting; the tag pages are once again topical pivots through the blogosphere. Additionally, the crawler we're rolling out makes blogroll and post link distinctions much more reliably. Finally, service response times and availability levels have been greatly improved.

The focus on helping bloggers and brands find who is talking about them, the topics they care about and the markets they serve continues to provide value. However as you've noted, blog search isn't enough, we're focused also on discovery applications, publisher services and ad services. We're also considering ways to bring in more social media to provide a broader 360 degree lens, we'd love to hear from folks about what they would find valuable in that regard. If you haven't explored the blogosphere through Technorati lately, give it a try and let us know what you think, we're listening!
thanks,
-Ian

It's true, changing your URL space does a reset on your authority. Here's the irony: I work at Technorati and have this as an issue because when I upgraded my CMS, my URL space changed; all of my recent links go uncounted towards authority. This is something we expect to work on but in the grand scheme, it's kind of an edge case; the vast majority of the blogs out there have stable URL spaces. Twitter is a different phenomenon but if you have other ideas about how Technorati can help you and your readers, please me know.
thanks,
-Ian
Technorait

Chris,
Thanks for posting this. Our focus over the prior 5 months has been on organizing the important blog posts of the day into major categories. I blogged about the percolator when we first released it in December, it's something I'd been wanting us to build for a few years and last fall, I actually got a chance to do it. Since then, it's been improved a whole lot with more data sources, improved categorization, grouping together related posts and identifying what the important posts are everyday in Technology, Politics, Sports, Entertainment, Business and Life. I really encourage you and your readers to let us know what what changes we've made that you dislike and which ones you liked. We're a small company serving a huge community with a large, volatile, open data set but fundamentally, we want to provide a service that is of value to you and your readers and your feedback helps make that happen. Feel free to contact me directly at ikallen@technorati.com
Thanks,
-Ian

Nice to hear we could be of better service, thanks for posting this.
-Ian

Hi Doug,
You do have 187 fans indicating your blog is a favorite on Technorati, see
http://technorati.com/blogs/www.douglaskarr.com?fans - maybe having Technorati widgetry on your site ain't all that bad ;)

Seriously though, I was saddened to read this when I first came across this post last week. I've been looking into various reports of rankings that've dropped, in most cases it comes down to one or more of a few simple explanations. There's the case of folks who have a handful of posts that received an unusual amount of attention and as those links age out of the 6 month window , their rank drops even as they continue to produce great posts and acquire new readership. I've found folks who were the beneficiaries of cut-and-pasted link sets from last spring's rash of viral link memes saw their ranks drop as time progressed. There are also cases where folks are unwittingly benefiting from a spammer's feed-scraped splogs or a bug in our system that was artificially inflating their links; when a correction is made those gains are reversed.

Suffice to say, you've got a great blog and I'm hoping we can continue to be of service to you and your readers. The Percolator we built is designed to highlight the things that bloggers are paying attention to and highlight the bloggers themselves, I've found some of your posts in there and I'm hoping to see more.
best,
-Ian
Technorati

Hi Zoli,
Your post was actually the first bit of feedback on the index shuffling we've been doing, I just want to say: Thank You for noticing!

Just to clarify: we're *not* deleting any indexes but we've been taking portions of the data offline while we realign some infrastructure. I really commented in that previous post (and here) in an effort to provide some visibility into what you'd observed. As to the index age, Google Blogsearch's index doesn't go back forever either, but yes the oldest hit in their index for that search is 14 months ago, much older than the 5.5 months of the age of our oldest hit. When I said most people "don't notice" it's because most folks aren't using us for historical research; they're using us to find the most recent posts and links. Google's main index has pretty good coverage for historical pages. I'm curious what data set time frame for blog search is important too you (and Dennis)? If we operated with a 18 month index, would that be sufficient? Do you need a historical blog search archive?

Thanks again,
-Ian

We're in the midst of some economization, performance fixes and retooling that have required taking some data offline. The data is not lost but our priorities are to prefer keeping recent data online. Most people don't notice :) We'll probably be bringing that data back online but I don't have an ETA yet.

As far as Allen's questions:
> Why does it sometimes show 0?

Simple. That's a bug. And the UI doesn't degrade gracefully when it occurs. Which is another bug. We know about 'em, we hate 'em too, we'll fix 'em.

> Why does it count spam blogs?

We don't intentionally count spam blogs, when we do spam purges they no longer contribute to counts. On a daily basis, we keep a lot of spam out but amidst the new edges of the blogosphere that emerges continuously that we try to cover, spam slips in. And when we catch it, we clean it out.

> Why does the overall (not authority) count go down?

Counts go down if we purge spam or if we pull some data offline (as has been observed with our keyword search index, so it is with our links). Of course authority fluctuates according to the rolling six month window within which we count unique blogs for a site's authority.

I hope that clears some things up.
thanks,
-Ian

Hi Doug,
The rolling six month window of authority calculation is always going to have ebbs and flows -- it's not just how many unique blogs link to you in that time frame, it's also how many link to the blogs above and below you in the histogram. We also had some operational glitches related to the data center move that we did that made rank calculations stale; fixing that may have caused some sudden jumps/drops but the normal state of affairs is for rank to drift up and down with how much attention one's blog posts receive within the blogosphere.

As far as what blogs should be included, this is becoming a harder issue to deal with. On the one hand we have media conglomerates with editorial teams publishing blogs (that the rest of the blogosphere actively talks about and links to) and on the other hand there are the John Chows selling and bartering links, which submerges what substance people are really paying attention to.

Indeed, we've had a lot of twists and turns at Technorati this year. But all of us at the company are committed from turning the doom and gloom that seems to be the popular characterization at the moment into our finest hour.
Regards,
-Ian
Technorati

When Technorati receives a bunch of pings for lots of URLs on a site and feed discovery is not functioning, indexing can run off of the tracks. If you don't see a feed icon in the URL box in your Firefox toolbar, chances are your feed discovery mechanisms are not in place correctly. Please let me know when that's remedied and send along the actual URL you're pinging with (or the list if you're pinging with more than one), I'll take a look.
thanks,
-Ian
Technorati

When Technorati receives a bunch of pings for lots of URLs on a site and feed discovery is not functioning, indexing can run off of the tracks. If you don't see a feed icon in the URL box in your Firefox toolbar, chances are your feed discovery mechanisms are not in place correctly. Please let me know when that's remedied and send along the actual URL you're pinging with (or the list if you're pinging with more than one), I'll take a look.
thanks,
-Ian
Technorati

There was a serious of unfortunate database losses that hit part of our infrastructure. No data was lost in this incident but for some of the last several days we did have indexing suspended for about 7% of the blogosphere while this was shaken out. Things are recovering now and there's a large back-log of indexing tasks queued up, it's working itself out.

Sorry about the grief this may have brought you.
-Ian

Admittedly, keeping our back-end operations smooth and improving the website take precedence but, fear not, our API users are important to us. Glad to see your widget displaying the rank again, I think that validates that the fix made to the API took effect :)
-Ian
Technorati

Admittedly, changing your blog URL is problematic for us from a book-keeping standpoint. We count links to your blog, if those links continue pointing to your old URL space, re-mapping them to a new one isn't something we have provisioned for.

We're trying to tool up facilities that will at least make it easier for folks to correct adaptations our data needs to make. But I appreciate the feedback and hope that we're providing helpful services.

-Ian
Technorati

Doug,
API calls are limited to 500 day. I tried your widget code, that's really cool! The usage I would recommend is to run it out of cron(1) once per day and publish it to a static javascript file you can include. For instance, an example setup I'd imagine is one where the widget gets published everyday at 1 a.m. like so:

0 1 * * * /usr/local/bin/php /home/dkarr/web/rank_widget.php > /usr/local/apache/htdocs/js/rank_widget.js

(specific paths are made up but hopefully that provides a good starting point)

In general, scaling the your API usage and scaling your website traffic should be decoupled ;)

Hope that helps,
-Ian