Hey Nick, great question, and serendipitous - I was just putting together an post on Diigo.
My problem with Diigo is close to your initial concerns - and, although I generally don't let it stand in the way of giving it a try, I tend to think that services suffer from having to be downloaded and installed. It creates difficulty straight from the start, and lays a hurdle in the user's path.
Another problem I have with services like Diigo and others like Adaptive Blue's in-browser-service, is that I use multiple computers - and, I need for these things to be able to follow me wherever I go!!
Maybe I'm lazy, but I usually give up on these services before I get bored with them because they're, essentially, one more thing I'd have to do when I get online.