... but it also feels like it is in the making. Which is fine. However, I would like to point out that exclusively choosing two comments from the very first of 58 comment pages does not give great credibility to the claim that they read most of the 3,500 comments. I also would have loved a link back to all comments on the new page (maybe I overlooked it)...

I think it is important to discuss the rating system used, since as everywhere on the Net, visibility is important.

The ranking system favored user kllmt's comment, which was on top since almost the beginning. I do not say that his/her comment was not valid, it only got disproportional attention and therefore more positive ratings. On the same level, the next comment still got a rating of +38, but you would have read to the end of the page and move on to get there - so I suspect that this +38 comes from far less people than the +82 of the first comment.

One idea to get around this would be to make the ratings only a weight in a randomized process rather than having deterministic ranking of the comments.

However, I think it is great that they try this.

Christoph,

I noticed a similar effect recently on two other sites (both driven by UserVoice): ObamaCTO (http://ideas.obamacto.org/) and Rebuild the Party (http://ideas.rebuildtheparty.com/). Instead of truly bubbling up the best ideas, the system heavily favored those ideas that had been submitted early and had gained an initial lead in votes. Basically, the first 25 choices above the fold received all of the attention and hence only solidified their top position over time (for some number crunching, see here: http://www.flickr.com/photos/planspark/3028432142/).

While this may look like a perfectly negligible flaw for some use cases (e.g. user feedback management on UserVoice, threaded discussions on IntenseDebate), it poses a serious problem when used in large-scale, government-sponsored e-participation projects that have much lower tolerance for such favoritisms.

To solve this, the system should be exposing "top rated" content much more cautiously (relative to other views: it definitely shouldn't be the only or default view). Another way is to enforce a certain attention balance across all items (e.g. by requiring participants to review a number of low-attention items per every vote or comment on an item that is already popular).

Having said that, it's great to see Change.gov experiment so early and so openly.

--
Web: htp://www.plansphere.com
Twitter: http://twitter.com/planspark

It's feasible that all comments combined total somewhere around 430,000 words. That might take an average reader about 29 hours of uninterrupted reading (see http://www.flickr.com/photos/planspark/3060802471/). Just reading, not processing in any form...

--
Web: htp://www.plansphere.com
Twitter: http://twitter.com/planspark

This is interesting, and it's nice that they put together this video in response to the flood of comments. It's also nice to see there's a lot of dialogue (back and forth) in the comments, and that it's mostly civil.

On the other hand, this was also pretty scripted and staged. It's a performance-response. That's not unexpected, but the real hope is that we begin to find ways of moving beyond prepared remarks and into an actual dialogue w/the new administration.

That's likely not possible as long as the power dynamic of the conversation remains what it is (commenting 1000s vs a video that cherry-picks), but as a first step this is pretty cool, especially if they really are open to the ideas being generated by commenters.