DISQUS

Hi again,

No worries, it's pretty hard for me to take offense, and really, I LOVE when people pick PostRank apart. It's from those folks that you often learn the most useful ways to improve what you're doing.

I hadn't seen the research project; thanks for the link. It's a funny balance -- we want the credibility that comes with "just the facts" -- measurement by algorithm. But at the same time we want the nuance and recommendations that come from social connection, which is purely human. (Though, of course, the folks who create the algorithms are human, too...)

And, of course, the same people who want to have the control over how we weight/measure things are usually the first ones to holler about compromised credibility of results, ironically. :)

The discussion about how to best measure "interestingness" has gone on for some time, and 100 people will give you 10 different answers. (I know, cuz I'm the one they tell...) :) The approach we've taken is three-pronged:

- gather as many engagement sources as possible to capture as many expressions of interest as possible
- gather engagement metrics over time to capture stories' real interest arc (some stories get all the attention immediately, others trickle in over time)
- account for the human aspect - the individual interests and random finds.

We've had a lot of feedback about needing to account for that third item, which was the big catalyst behind the Discovery functionality we just launched.

I.e. it's about what YOU find interesting and what YOU read and making it easy to share that with others. Especially since there's way too much info online for any one person to keep on top of.

Cuz if I've learned one thing from perusing the feeds in our system, it's that there are communities online around EVERYTHING, so there's always someone else into what you're into.

@Datenwachschutz -- It's the same in countries besides Germany, too. :)

In any case, we don't collect private data. Everything we collect is anonymous.

I completely agree, and it's something we're working on all the time. It's a slow process, sometimes, since it seems everyone learns differently online. Sometimes text, sometimes graphics, sometimes video...

That said, no one's completely cracked Google's algorithms, and they're doing alright for themselves. :)

Until the day we don't need explanations, I guess I've got some job security. :)

Hi Andy,

I can definitely see the value of comparing the performance of a current post against a post from a year ago if they're on the same topic, for example. That would actually be really cool. Unfortunately, there really isn't a way to isolate the reasons for the engagement those posts get. For example, in a year your blog could have a lot bigger reach, and you might get more comments and diggs and tweets on every post, so the engagement wouldn't necessarily reflect your handling of a particular topic.

The Top Posts calculation isn't quite that simplified or specific, nor is it specifically based on time frame, though that's certainly the default assumption people make (myself included, initially, because it makes the most sense at a glance). The intent of the widget also isn't really as a snapshot of recent posts. It certainly could be, but ideally it brings up content that's a bit deeper than that and, ideally, gets readers into publishers' archives.

It's been really interesting seeing how people approach PostRank. We get the whole gamut, really, from folks who don't care in the least how it works -- they're just happy to trust the algorithms and have less to read -- to folks who want to completely dismantle and understand the entire system and who aren't willing to trust anything. Sometimes they get comfortable with how we do things, sometimes not.

Really, though, the skeptics, like yourself, make the most interesting people to talk to. They often have perspectives and questions we haven't seen before, which sparks great conversations and ideas about new ways to handle features. (So thank you!)

It's certainly one of the motivations behind launching the Analytics Beta -- to get input from people from all over the adoption spectrum.

Melanie Baker's last blog post..A response on how PostRank works

Hi Andy,

Actually, none of those examples are errors. (Whew!) I'll try and explain what you're actually seeing there.

In the first example you list, I can't fully see the two dates those posts were published, but I suspect those two posts you reference were published quite a while apart? We don't compare each new post to every post you've published that's in our system, for a number of reasons, so comparing the PostRank scores and metrics of two posts published a year apart, for example, wouldn't be applicable.

If the two posts you reference had been published in the same week, for example, the second one would definitely have gotten a higher PostRank score. However, the second example post, ranked 4.1 was published after a number of posts that had gotten even more engagement -- more than 5 comments, 2 tweets, etc. When the post ranked 7.0 was published, however, the preceding posts weren't getting as much engagement, so 4 comments was pretty good performance at that time.

Regarding the second example, the short answer is that PostRank scores in the widget and PostRank scores on the website are calculated slightly differently. A more in-depth explanation of that is here: http://blog.postrank.com/2009/01/07/postranks-w....

Hope that helps clear things up. Any additional or outstanding questions, please feel free to give me a holler any time. :)

Melanie Baker's last blog post..Feed Analytics Beta

I'm afraid the extensions only work with Firefox, and we don't have any plans to develop versions for Chrome at this time.

Alrighty, I think we've got this sorted out now. For those watching at home, it appears to have been a comments detection issue. Cross your fingers! :)

The Get Satisfaction problem appears to have been an issue of Mr. Gentry's issue getting swallowed up within someone else's larger issue, which, when that got fixed, we chalked up as taken care of. My bad on that one.

Thanks for including us, Stan!

We're pretty excited about the new site, since so many of the new/changed features are based directly on user requests. So of course we're very much looking for feedback at this point to improve the user experience even more.

We've been seeing awesome stuff from third party developers who've integrated our functionality too, like Daisy Feed for iPhone and Read It Later for pretty much everybody.

We're also working on more tutorials, both non-video (to make them more accessible to some folks), and topic targeted (e.g. "PostRank for brand management"), so requests are welcome there as well.

Hi Nick,

Yeah, we're aware of the issue and have one of our guys checking it out. There was also (terrible timing) an EC2 hardware failure, so we're getting things moved over and caught up as quickly as possible, but unfortunately it takes time. Should be done today, though.

Hi Malcolm -- thanks for the write-up! Definitely a good overview of what we do now, and what the future might hold.

One correction -- we do analyze tweets, and it doesn't matter if the URLs contained are shortened or not. In reality, most Twitter clients auto-shorten tweets, so we pretty much have to be able to parse that. We just don't display tweets in our analysis results.

Was it ever - thanks for the catch! Fixed now.

Hi Javed,

I actually just replied to you on Twitter a little while ago that I'm getting the developers to look into the conflict to see if there's anything we can resolve from our end.

Will let you know what we find out.

Hi James, thanks for writing about us!

It's always interesting seeing how different types of business users put our stuff to work.

Have you tried out the Google Reader extension in Firefox? You might find some interesting comparative/competitive value in the Thematic PostRank functionality, since it lets you rank contents from assorted feeds on topics you select based on comparisons to each other's performance.

Hi Barb,

Weighted according to what we call the "5 Cs" -- more information about that on our PostRank site.

Any additional questions, let me know.

"The problem I wrestle with in this argument is that it must start with the supposition that's every blogger and social media player has the very same goal."

That totally nails the results of a lot of conversations I had with the bloggers covered in the list we analyzed. Some thought it was really engaging; some thought it was pretty cool and had a lot of questions (which were a great education for me); and some reflected your statement above -- that it wasn't necessarily what they track (or at least not the most important thing).

Adjacent to that is the fact that some really valuable stuff -- interpersonal interactions -- take place off the grid, as it were, in formats like email, or evolve from online venues to offline ones, where we can't track systematically at all.

It's definitely part of the opportunity of the metrics "space", I think, given that things are so embryonic yet. Ilya, our CTO, has said he thinks the solution to RSS, filtering, engagement measuring, metrics, etc. will be a variety of applications that can work in combinations, and given how many different sets of priorities I've seen, I'm certainly inclined to agree.

Hi Jevon -- Thanks for the coverage and your tenacious loyalty and high expectations. :)

We're working on this being just the beginning. As you note, there's some amazing opportunity out there, and the "space" is still so embryonic.

Keep the questions and feedback coming and the expectations high. It's exactly what makes us better.

Hi Jevon -- Thanks for the coverage and your tenacious loyalty and high expectations. :)

We're working on this being just the beginning. As you note, there's some amazing opportunity out there, and the "space" is still so embryonic.

Keep the questions and feedback coming and the expectations high. It's exactly what makes us better.

Hi Trevor,

Thanks!

We talked a fair bit about that. We track site interaction data, but we don't track identifying information. We just want to know that blog posts are getting read or responded to, for example. For our purposes there's no reason to know who, specifically, is doing it.

(Though if you write your own blog post or leave a comment you're usually identifying yourself anyway...) :)

Hi Marji,

Those more passive interactions (clicking, reading, etc.) go in the "Clicking" category of our 5Cs of analysis. We try to do our best to record all interactions, even if they aren't the more public or visibly interactive.

Hi Chris,

Thanks for the mention, and congratulations on being the king! :)

We have just launched a whole bunch of cool stuff, so I can finally offer more info on methodologies, logic, etc.

Hope that sheds some light on things. Any questions or whatnot, feel free to give me a holler.

@Shey - We've been doing a fair bit of hashing out re. how to best handle FriendFeed. It's definitely a major hub these days!

Thanks, Marshall. There were definitely a lot of suggestions and "what ifs?" thrown around internally before we set the parameters of this experiment, but since we wanted to get it out the door before next year, we had to stop somewhere and leave more excitement for the future. :)

Rest assured there'll be much more on this subject from us!

And as for your "cred", had you been on the list, based on the overall engagement for marshallk.com, you'd be... #17. w00t! (Jim was just talking about how in the future we'll have to run stats and show who should make the list.)

Hmm... Ahh, in the last link. Good catch. Fixed!

Hi Doug, thanks for the link love!

The API has actually been out a while, but the Programmable Web coverage is new. Can't wait to have a whole lot more mashups to add.

Adding PostRank functionality to mainstream applications is awesome and really enriching, but it's the stuff the third-party devs get up to that really impresses me. :)

Amen! It's a bit of a Murphy's Law in product development that it seems like if you design products with the traditional "user is the end consideration" approach, the stuff you spend the most blood, sweat, and tears getting "right" will be the stuff the users utterly ignore or passionately hate. And the stuff you despise and cringe at incorporating will be their unanimous favourite. :)