As with many other things he’s written, Clay Shirky’s Against Well-designed Reputation Systems is well worth the read. Some choice bits (emphasis mine):

Constituting users’ relations as a set of bargains developed incrementally and post hoc is more predictive of eventual success than simply adopting any residue from previous successes…

Successful constitutions, which necessarily create clarity, are typically ratified only after a group has come to a degree of informal cohesion… The desire to participate in a system that constrains freedom of action in support of group goals typically requires that the participants have at least seen, and possibly lived through, the difficulties of unfettered systems, while at the same time building up their sense of membership or shared goals in the group as a whole…

Digg seems to have suffered more from both system gaming and public concern over its methods, possibly because the lack of organic growth of its methods prevented it from becoming legitimized over time in the eyes of its users…

Reputation systems create an astonishing perimeter defense problem. The number of possible threats you can imagine in advance is typically much larger than the number that manifest themselves in functioning communities. Even worse, however large the list of imagined threats, it will not be complete… As you will not know which of these ills you will face, the perimeter you will end up defending will be very large and, critically, hard to maintain.

To me, the fundamental take-away is that we shouldn’t treat successful online communities as a source of designs for building new social systems — we should use them as a source of insight into how online communities successfully grow into themselves. It’s the process, stupid.