Looking over the titles in today's (13 June) online special (all Early Release titles 50% off) reminded me of an irritant about reviews on the site: People who don't get the Early Release concept and give low star ratings to what are clearly works-in-progress. As a result, titles that could end up being high quality books when finished get saddled with a low rating that could be hard to come back from, and could hinder sales.
This is not O'Reilly's fault, but perhaps you could mitigate this by not allowing Early Release titles to be star-rated.
Still allow reviews to be left, of course, because Early Release readers often provide useful feedback, and that's the whole idea of making titles available via Early Release, right? Star ratings on unfinished titles are just pointless, and potentially needlessly damaging.
Help get this topic noticed by sharing it on Twitter, Facebook, or email.
As you point out, reviews are essential information for customers to make informed decisions, and they provide important feedback for our editors and authors. We have wrestled w/ bad reviews on some titles, but they're often an indication that the book was made available too soon. An Early Release needs to have enough for a reader to feel that they've gained some early insight about a technology and can start to apply it. We've learned that code examples have to be accurate. And we need to deliver on the promise of regular updates. If we follow those rules of thumb then we're usually fine.
Lastly, if an Early Release garners bad reviews that were resolved in later releases or the final edition we notify the reviewer, invite them to review the book again, and remove those reviews if they're no longer relevant.
I hope this helps. --Allen