Reputation system

Maintaining effective reputation systems

The main function of reputation systems is to build a sense of trust among users of online communities. As with brick and mortar stores, trust and reputation can be built through customer feedback. Paul Resnick from the Association for Computing Machinery describes three properties that are necessary for reputation systems to operate effectively.

1. Entities must have a long lifetime and create accurate expectations of future interactions.
2. They must capture and distribute feedback about prior interactions.
3. They must use feedback to guide trust.

These three properties are critically important in building reliable reputations, and all revolve around one important element: user feedback. User feedback in reputation systems, whether it be in the form of comments, ratings, or recommendations, is a valuable piece of information. Without user feedback, reputation systems cannot sustain an environment of trust.

Eliciting user feedback can have three related problems.

1. The first of these problems is the willingness of users to provide feedback when the option to do so is not required. If an online community has a large stream of interactions happening, but no feedback is gathered, the environment of trust and reputation cannot be formed.
2. The second of these problems is gaining negative feedback from users. Many factors contribute to users not wanting to give negative feedback, the most prominent being a fear of retaliation. When feedback is not anonymous, many users fear retaliation if negative feedback is given.
3. The final problem related to user feedback is eliciting honest feedback from users. Although there is no concrete method for ensuring the truthfulness of feedback, if a community of honest feedback is established, new users will be more likely to give honest feedback as well.

Other pitfalls to effective reputation systems described by A. Josang et al. include change of identities and discrimination. Again these ideas tie back to the idea of regulating user actions in order to gain accurate and consistent user feedback. When analyzing different types of reputation systems it is important to look at these specific features in order to determine the effectiveness of each system.

Standardization attempt

The IETF proposed a protocol to exchange reputation data. It was originally aimed at email applications, but it was subsequently developed as a general architecture for a reputation-based service, followed by an email-specific part. However, the workhorse of email reputation remains with DNSxL's, which do not follow that protocol. Those specification don't say how to collect feedback —in fact, the granularity of email sending entities makes it impractical to collect feedback directly from recipients— but are only concerned with reputation query/response methods.

Notable examples of practical applications

• Search: web (see PageRank)
• eCommerce: eBay, Epinions, Bizrate, Trustpilot
• Social news: Reddit, Digg, Imgur
• Programming communities: Advogato, freelance marketplaces, Stack Overflow
• Wikis: Increase contribution quantity and quality
• Internet Security: TrustedSource
• Question-and-Answer sites: Quora, Yahoo! Answers, Gutefrage.net, Stack Exchange
• Email: DNSBL and DNSWL provide global reputation about email senders
• Personal Reputation: CouchSurfing (for travelers),
• Non Governmental organizations (NGOs): GreatNonProfits.org, GlobalGiving
• Professional reputation of translators and translation outsourcers: BlueBoard at ProZ.com
• All purpose reputation system: Yelp, Inc.
• Academia: general bibliometric measures, e.g. the h-index of a researcher.

Reputation as a resource

High reputation capital often confers benefits upon the holder. For example, a wide range of studies have found a positive correlation between seller rating and selling price on eBay, indicating that high reputation can help users obtain more money for their items. High product reviews on online marketplaces can also help drive higher sales volumes.

Abstract reputation can be used as a kind of resource, to be traded away for short-term gains or built up by investing effort. For example, a company with a good reputation may sell lower-quality products for higher profit until their reputation falls, or they may sell higher-quality products to increase their reputation. Some reputation systems go further, making it explicitly possible to spend reputation within the system to derive a benefit. For example, on the Stack Overflow community, reputation points can be spent on question "bounties" to incentivize other users to answer the question.

Even without an explicit spending mechanism in place, reputation systems often make it easier for users to spend their reputation without harming it excessively. For example, a ridesharing company driver with a high ride acceptance score (a metric often used for driver reputation) may opt to be more selective about his or her clientele, decreasing the driver's acceptance score but improving his or her driving experience. With the explicit feedback provided by the service, drivers can carefully manage their selectivity to avoid being penalized too heavily.

Attacks and defense

Reputation systems are in general vulnerable to attacks, and many types of attacks are possible. As the reputation system tries to generate an accurate assessment based on various factors including but not limited to unpredictable user size and potential adversarial environments, the attacks and defense mechanisms play an important role in the reputation systems.

Attack classification of reputation system is based on identifying which system components and design choices are the targets of attacks. While the defense mechanisms are concluded based on existing reputation systems.

Attacker model

The capability of the attacker is determined by several characteristics, e.g., the location of the attacker related to the system (insider attacker vs. outsider attacker). An insider is an entity who has legitimate access to the system and can participate according to the system specifications, while an outsider is any unauthorized entity in the system who may or may not be identifiable.

As the outsider attack is much more similar to other attacks in a computer system environment, the insider attack gets more focus in the reputation system. Usually, there are some common assumptions: the attackers are motivated either by selfish or malicious intent and the attackers can either work alone or in coalitions.

Attack classification

Attacks against reputation systems are classified based on the goals and methods of the attacker.

• Self-promoting Attack. The attacker falsely increases their own reputation. A typical example is the so-called Sybil attack where an attacker subverts the reputation system by creating a large number of pseudonymous entities, and using them to gain a disproportionately large influence. A reputation system's vulnerability to a Sybil attack depends on how cheaply Sybils can be generated, the degree to which the reputation system accepts input from entities that do not have a chain of trust linking them to a trusted entity, and whether the reputation system treats all entities identically.
• Whitewashing Attack. The attacker uses some system vulnerability to update their reputation. This attack usually targets the reputation system's formulation that is used to calculate the reputation result. The whitewashing attack can be combined with other types of attacks to make each one more effective.
• Slandering Attack. The attacker reports false data to lower the reputation of the victim nodes. It can be achieved by a single attacker or a coalition of attackers.
• Orchestrated Attack. The attacker orchestrates their efforts and employs several of the above strategies. One famous example of an orchestrated attack is known as an oscillation attack.
• Denial of Service Attack. The attacker prevents the calculation and dissemination of reputation values in reputation systems by using Denial of Service method.

Defense strategies

Here are some strategies to prevent the above attacks.

• Preventing Multiple Identities
• Mitigating Generation of False Rumors
• Mitigating Spreading of False Rumors
• Preventing Short-Term Abuse of the System
• Mitigating Denial of Service Attacks