Disqus - Latest Comments for jedc_mercury

Re: What you (and even Steve Jobs) could learn about customer support from JetBlue

jedc_mercury — Wed, 18 Nov 2009 17:15:09 -0000

Great post, Nate.

Not only do you have really good customer service because you do it yourselves, but it's always really quick, too!

I definitely wish more companies did exactly what you (and JetBlue) do...

Re: Predicting Milestones with Inkling (ala Basecamp)

jedc_mercury — Tue, 06 Oct 2009 20:07:52 -0000

Does this ever time out? Or is it the case that once a user has made one prediction, they can never go back to update it?

It might be useful if users could update, perhaps on a weekly basis?

Re: A new competitor in prediction markets, and their brilliant case study

jedc_mercury — Tue, 29 Sep 2009 12:36:55 -0000

I just made a substantial comment over on Midas Oracle here: http://www.midasoracle.org/...

I'm copying it here since it seems relevant:

Hi, Paul.

I agree that CrowdClarity’s slides don’t get into the detail necessary to understand why they were successful. But that’s largely because those slides were published as a sales tool… a top-line attention grabber for potential sales leads.

The things I would want to understand to truly evaluate success would include: demographics of traders, the types of forecasts that already exist, what error rates are when sales volatility is more “normal”, what trader incentives were used, etc.

Regarding your point on number of traders, my own research showed that as long as you have more than 15 traders, the market generates a calibrated result. (The markets I ran were probabilistic.) So that’s actually quite a realistic number, even though it might intuitively seem low. But I would point out that because of the nature of the markets I ran, the likelihood that traders knew each other was low, so there was a natural diversity in the 15+ that I studied.

I would certainly consider the prediction market / case study CrowdClarity published a “success.” The key for me is turning that into a *business* success. You’re very right that if this was one of the first months that a PM was run, no one would have likely believed the results, even if they were the closest to the eventual truth.

But let’s say that a PM has been showing reasonable accuracy for several months, which I would define as similar or better accuracy than other forecasting methods. Then, like the case study, the prediction market shows a drastically different result than any other forecast. While a manager probably won’t “bet the farm” on a prediction market alone, that certainly would warrant re-thinking the forecast.

The reason is simply that the market aggregates the information more quickly than other methods. I would argue that GM and Edmunds.com used forecasting models that are quite good most of the time, but completely wrong when some of the basic assumptions collapse. Since the PM doesn’t rely on algorithms, collapsing assumptions won’t affect accuracy.

In many ways, I think this last point is the most important with prediction markets. 80-90% of the time a prediction market might generate a forecast with accuracy that’s on par with other methods… maybe a little better, maybe a little worse. But the other 10-20% of the time, when the forecasts diverge significantly, is where prediction markets can be *very* useful. When assumptions behind traditional models weaken or collapse, a prediction market can be the early warning signal, since it uses a different methodology to generate a forecast.

Maybe a company spends $3-4k a year on a prediction market that generally confirms or better brackets existing forecasts 11 months out of the year. But if the intelligence it generates that 12th month of the year helps the company save $20k, it strikes me as a wise investment.

Again, that’s not to say that a prediction market is going to solve a company’s problems. It needs to address problems where the cost of the error is worth the investment and where a prediction market can effectively address it. And that’s a sensitive balance.

Re: A review of idea and innovation software

jedc_mercury — Wed, 27 May 2009 15:21:40 -0000

Thanks, Jeffrey. Will definitely take a look at it and include it in an updated post.

Re: Measuring the prediction market industry – a proposal

jedc_mercury — Thu, 23 Apr 2009 12:14:45 -0000

Actually, I disagree a bit. While each vendor sells prediction market software, each has a different mix of software and services, so even I wouldn't be able to really figure out how well each of them are doing. Even if client engagements are flat year-over-year, if each engagement is of significantly more value (ie, a lot of consulting services), the overall value has increased.

Thanks for the comments; I hope to get this going soon!

Re: Measuring the prediction market industry – a proposal

jedc_mercury — Thu, 23 Apr 2009 08:45:40 -0000

Thanks, Paul.

I'm hopeful that they will release the data. I would hope that it's more important to show that the industry is growing/thriving (my intuition) than holding the info back.

However, only the *aggregated* data would be released, so there's no way to compare vendors against each other. I will also provide each vendor their own data compared to the aggregated data for their own benchmarking. (In other words, you won't be able to tell from this which vendors are growing, just how the industry as a whole is growing.)

Part of this is also due to the nature of clients that each vendor takes on. Inkling cater more (in general) to the self-service customers, while others (such as Newsfutures) have generally fewer clients but with a lot more strategic consulting around their engagements.

Re: A review of idea and innovation software

jedc_mercury — Sun, 19 Apr 2009 13:22:27 -0000

Thanks, Hutch. I appreciate the additional info.

Re: Approaching business problems differently

jedc_mercury — Fri, 03 Apr 2009 06:47:38 -0000

HI, John.

"Slow burn" isn't a diffusion method or metric, just a description of growth. Not dying, but also not "burning like wildfire." It's continued, steady growth.

I agree that PMs aren't an academic backwater; far from it! Yet they've also not been adopted widely. I have no idea where you see order-of-magnitude growth per year, and would love to see evidence of it. That leads to your last point...

I would love to set up an PM on diffusion and adoption, but am a bit stumped on what exactly to trade! Ideally it would be on some metric of number of clients and/or revenues of the PM software companies, broken out between software and consulting fees. But since everyone is private none of that information is available.

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:46:36 -0000

Hi, John.

While I certainly see growth, and even strong growth, I haven't seen evidence of order-of-magnitude growth, and don't believe it will happen this year. Last year many people said that 2008 would be the big year because of the US election and all of the press on prediction markets there... hasn't happened. You're saying 2009 will be the big year; I'm just saying that I think it's still going to take some time.

Each year for the past ten years companies have been saying that *that* year was going to be the big year for innovative uses on mobile phones; yet it was only once Apple came out with the iPhone (arguably, when they came out with the 3G iPhone) that the industry finally started to radically change for the better. I'm just wary that if I start saying that *this* is the year for prediction markets, that I'll look like a fool. I'm not going to say it until and unless I have really good reasons to believe it.

If you had more concrete evidence (sales growth, etc.) I would certainly be open to changing my mind, but I haven't seen anything like it.

Again, as I wrote in my post, prediction markets have the capability to explode at any given time; what's limiting their adoption is that they approach the problem completely differently from current solutions. It takes time for people (let alone organisations) to wrap their heads around new ideas.

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:24:55 -0000

Hi, John.

While I respect your opinion, I disagree. I think prediction markets will see continued growth for quite some time, but don't believe that 2009 will see "sharp growth." (This depends on what you mean by sharp growth; if we had a market on this question it would need to define this. I would use a metric of an order-of-magnitude growth in the industry in 12-18 months.)

As I wrote in the post, I think prediction markets are a kind of technology that will see a "slow burn" in growth for quite some time; it will be many years before we see any truly dramatic spike. Again, this doesn't mean that they're not useful, just that they're a different enough tool that it's tough for companies to adapt to use them.

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 06:25:46 -0000

Hi, Jacob.

This is partly what I was getting to in my post. Traditional forecasting (such as FMCG for hundreds/thousands of SKUs) will probably be best served by traditional techniques. If the product demand is relatively stable, I don't see how prediction markets would add a lot of value.

But I have seen some impressive case studies for prediction markets where a company is trying to forecast something that's not been forecasted before. Mat Fogarty (of Crowdcast) used prediction markets at EA to forecast quality scores for new games. The prediction market forecasts reduced the error rate in half!

There have also been some great examples of prediction markets being used in project management, specifically to assess the probability that a project will meet specific milestones. (I wrote a bit about that here: http://blog.mercury-rac.com... )

The general trend here is that prediction markets can be used to ask more strategic questions that are difficult to assess analytically.

In general I agree that the industry needs specific, successful examples of great prediction markets. Inkling has started posting some here (http://inklingmarkets.com/h... but even those don't have real specifics.

I'd be happy to help talk you through some ideas; you can e-mail me at jed (dot) christiansen (at) mercury-rac.com.

Re: Prediction Markets and March Madness

jedc_mercury — Tue, 17 Mar 2009 20:37:06 -0000

I'm not so sure about either your "locked up" or "best academic school"! And the second one's 'fightin' words'... :)

I know it wasn't easy for Michigan this year, but like I mentioned, they almost had to get in after the football teams' collapse.

Re: The value of prediction markets

jedc_mercury — Mon, 09 Mar 2009 15:25:35 -0000

Very interesting thoughts here.

I like that you distinguish between the PM software, the market itself, and how the market is used in decision making. I completely agree. Personally I believe you have to work backwards from how the market will be used to eventually get to the software you need to use.

I think public prediction markets can certainly almost always provide informational value to people unfamiliar with a topic; the value to people familiar with an industry/contract may be substantially less, though seeing immediate trends may still be useful.

Finally, I definitely agree with your last point on the corporate world missing opportunities. Fundamentally I think it comes down to the organisational tension: connecting the lower-level employees to the upper management threatens middle management, who like to be the conduit of information. I hope that more successful stories are published in the future about how PM's can in fact put numbers behind key areas of uncertainty in corporations.

Thanks again for your comment!

Re: A review of idea and innovation software

jedc_mercury — Wed, 04 Mar 2009 20:25:01 -0000

Hi, Jenny. Thanks for commenting.

I have to be honest; based on my understanding of Qualcomm's use of prediction markets, I'm really skeptical of how they're used. I understand that they're using PM's to rank new product/service ideas, which I think is an inappropriate use of PM's. What tends to happen is that people trade/buy those ideas that they think _other_ people will buy, and not necessarily the best ideas. I believe that doing so may find good ideas, but they'll be conventional ideas. The ground-breaking ideas won't get noticed.

That said, if they feel it works for them, that's fine. But I think there are other tools that are better suited for innovation. I'm doing a fair bit of work on this now, and hope to be able to talk about it more widely later this year.

Re: A (long) review of prediction market software

jedc_mercury — Mon, 16 Feb 2009 05:41:15 -0000

Hi, Robert. While I haven't updated this recently, most of the data still stands. I'll see if I can update it in the coming weeks/months.

Re: Peter Arnold: Newcastle Science City

jedc_mercury — Wed, 28 Jan 2009 09:32:33 -0000

Hi, and thanks for the links to my posts on my (Mercury Research & Consulting) blog. I've also put up a post on innovation & ideas markets, too. You can check it out here:

http://blog.mercury-rac.com...

Re: Hubdub is One Today!

jedc_mercury — Wed, 28 Jan 2009 07:18:47 -0000

Congratulations! With your additional funding, I'm really looking forward to seeing Hubdub develop in the coming year(s). Best wishes for 2009.

Re: Find 100 things to say

jedc_mercury — Mon, 26 Jan 2009 11:23:24 -0000

The quick and easy way to do it would be to compile a list of 100 recommendations, quotes, insights, etc. from your work on Enterprise 2.0. Then post them individually throughout one day.

Re: A new voice

jedc_mercury — Mon, 26 Jan 2009 08:14:12 -0000

Thanks for pointing this out.

I agree, Mike is a great guy and I look forward to reading more from him. While I'm not really his target customer, it will be interesting to see how he develops the site in the 2009.

Re: A review of idea and innovation software

jedc_mercury — Sun, 25 Jan 2009 04:57:47 -0000

Thanks for that, Mark. I wish you all the best as a public company in these crazy economic times.

Based on your comment, do you feel that some of your competitors play loosely with reality?

Re: Developing a business case for Prediction Markets

jedc_mercury — Wed, 21 Jan 2009 19:35:53 -0000

Hi, Danielle.

Offhand, I don't think there are any segments that are more or less suited than others for prediction markets. It really has more to do with information availability; do traders know and see the metrics they're judging regularly, and do they have enough information to form their own opinions. Please feel free to contact me for further detail.

Re: Prediction markets and innovation in 2009

jedc_mercury — Mon, 05 Jan 2009 10:27:43 -0000

What's life without a little recursion? :)

Re: A review of idea and innovation software

jedc_mercury — Sat, 27 Dec 2008 07:04:49 -0000

I'm glad that this was useful to you. Please feel free to contact me if you have any questions.

Re: Big news, film (trading) fans!

jedc_mercury — Tue, 09 Dec 2008 04:52:03 -0000

Hi, Alex. Thanks for the note.

Like I said above, I think this is a really exciting development. I hope there aren't any problems with the CFTC, but I'm guessing you wouldn't put the application in unless you thought there was a very high chance of approval.

Again, I look forward to seeing CandorExchange develop this!

Re: Evaluating prediction market success in the 2008 election (…or, why wrong is right)

jedc_mercury — Tue, 18 Nov 2008 13:41:19 -0000

Hi, Peter. Thanks for the note.

What I believe is that you can't say right now that either InTrade or
the polls were more accurate back in July/August/September. While you
can compare the relative dynamics between the two, there's no way to
say one was better.

I hope to post more about this soon...

Jed

On 18 Nov 2008, at 17:58, "Disqus"