Token-Weighted Crowdsourcing

Crowdsourcing has proven to be a wildly effective method for generating and curating content -- consider Wikipedia which has almost 40 million pages of user-generated content [1].  On the commercial side, crowdsourcing product reviews and recommendations is extremely profitable and Yelp earned more than $1 Billion in revenue in 2019 [2], despite consistent evidence of reviewer fraud [3].  Fraudulent reviews plague crowdsourcing recommender systems, and analysis has shown that in the Amazon Marketplace, which maintains one of the largest crowdsourcing review platforms, in some categories more than 50% of the reviews appear to be fraudulent [4].  One of the challenges is that individuals have a strong incentive to fraudulently boost the reviews of their own products, while sabotaging the reviews of their competitors, and a vibrant market exists for buying and selling fraudulent reviews [5].

One of the challenges of designing a crowdsourced recommendation platform is that while the platform itself may be incentivized to make its recommendations honest so that users will continue to use the platform, individuals are incentivized to skew recommendations for their own purposes.  The rise of blockchain-based smart-contracts has created new opportunities to develop platforms that incentivize honest and accurate reviews and recommendations and align the incentives of the platform with its users.

Token-Curated Registries (TCRs) [6], were designed to crowdsource the creation of high-quality lists (e.g. a decentralized list of the top colleges [7]).  TCRs use a system of special-purpose cryptocurrency “tokens.”  Each token can be viewed as a share in the system, and the list is curated by token holders.  Candidates can propose their inclusion in the list by staking tokens, and inclusion is granted through a token-weighted vote from the shareholders.  If the applicant is rejected, the applicant’s stake is distributed among the token holders.  Inclusion in a high-quality list of this sort should provide value to its listees, and thus would create more demand from applicants (who need tokens to apply for entry into the list).  This increased demand would increase the price of the underlying token, and this increase in value would benefit the existing token-holders (who also serve as the curators).  In 2018 Adchain [8] and Dirt [9] developed TCRs.  But will the TCR incentive mechanism actually drive the creation of high-quality lists? 

Token-weighted voting has been deployed in other contexts as well.  Steem [10] and Sapien [11] use token-weighted voting to curate user-generated content.  Token-weighted voting has also been used for blockchain governance (e.g. Digix [12], 0x [12,13] and Tezos [14]).  In these contexts, token-weighting voting is used to decide on hard-forks and protocol upgrades.  CarbonVote [15] provides a platform for non-binding token-weighted polls on the Ethereum platform.  The intuition for token-weighted voting in this scenario is similar.  Token-holders have the most to gain if the platform succeeds.

Although token-weighted voting aligns the incentives of the voters and the platform, does it effectively tap into the wisdom of the crowd?

The recent paper, Token-Weighted Crowdsourcing [16], analyzes token-weighted crowdsourcing when participants have imperfect (and heterogeneous) information.  They consider a simple model, where token holders are asked to vote on the quality of a single product.

  1. There is a single product of unknown quality
  2. Each token holder gets a noisy “signal” about the true quality of the product
  3. Each token holder submits a vote to the platform
  4. The platform outputs a guess, which is the token-weighted average of the users’ votes
  5. The payoff to the platform (and hence the token holders) depends on the accuracy of the platforms’ guess

The first observation is that in order to optimally aggregate the users’ noisy signals, the signals should be weighted according to their precisions (votes from users with more accurate signals should receive higher weight), and not according to their token-holdings.

Strategic voters recognize that the platform’s payoff will be optimized by weighting votes according to the voter’s precision rather than their token holdings, and thus strategic voters will actually “shade” their votes, in order to counteract the platform’s token-weighting strategy and recover the optimal precision-weighting aggregation.  Thus, when all voters are strategic, the platform’s guess is optimal, i.e., corresponds to the maximum likelihood estimate of the true quality given all the voters’ signals.  On the other hand, this optimum is not achieved because of the token-weighting mechanism, but rather in spite of it.  In fact, when all voters are strategic, any weighting mechanism that assigns nonzero weights to all voters will lead to an optimal aggregation (because the strategic voters will shade their votes to unroll the platform’s weighting mechanism and instate the optimal, precision-weighted mechanism).

On the negative side, although the token-weighted aggregation mechanism achieves the optimal outcome for the platform when all voters are strategic, the presence of even a single “truthful” voter makes it impossible for the platform to recover optimality.  This sub-optimality in the presence of truthful voters holds even assuming strategic voters know the type (“strategic” or “truthful”) of all other voters.

In the real world, however, voters’ may actively seek to gather information to improve the precision of their vote, with the hope of improving the overall accuracy of the platform, and hence their personal returns.  To model this, Tsoukalas and Falk consider a two-stage game, where voters can engage in (costly) information-gathering to improve their precision, before submitting their votes to the platform.

In this setting, where voters can exert effort to improve their precisions, the voters’ information-gathering increases with token-holdings -- something that is often touted as a core feature of token-weighted aggregation schemes.  On the other hand, this increased information gathering is not driven by the token-weighted aggregation mechanism, which gives a larger weight to votes of larger stakeholders.  Instead it is driven by the fact that token holders have a larger stake in the system, and thus receive a larger payout from an increase in platform accuracy.  In fact, information-gathering increases with token holdings if the platform uses any weighting strategy that assigns nonzero weights to all voters.

When voters can exert effort to improve their signals, it is natural to ask whether the token-weighting mechanism induces an optimal level of effort -- in other words, could the overall accuracy of the platform (and thus the payout to all voters) by re-distributing effort among the voters?  In fact, the token-weighting mechanism does not induce an optimal effort allocation.

Tsoukalas and Falk show that it is not the token-weighted aggregation mechanism that incentivizes token holders to act for the community good, but instead the simple fact that token-holders share in the profit of the platform.  Indeed, when token-holders are strategic, almost any aggregation mechanism yields similar outcomes.

The core insight is that it’s not the token-weighted voting that drives user behavior, but simply that the token-holdings themselves align the incentives of individual voters with those of the platform.  For more details, see Token-Weighted Crowdsourcing [16].

1.    Milijic M. 21+ Wikipedia Statistics to Keep You Updated in 2020. 18 Oct 2019 [cited 26 Sep 2020]. Available: https://leftronic.com/wikipedia-statistics/

2.    Yelp Reports Fourth Quarter and Full Year 2019 Financial Results. 13 Feb 2020 [cited 26 Sep 2020]. Available: https://yelp-ir.com/news-releases/news-release-details/2020/Yelp-Reports-Fourth-Quarter-and-Full-Year-2019-Financial-Results/default.aspx

3.    Luca M, Zervas G. Fake it Till You Make it: Reputation, Competition, and Yelp Review Fraud. Management Science. 2016;62. doi:10.2139/ssrn.2293164

4.    Dwoskin E, Timberg C. How merchants use Facebook to flood Amazon with fake reviews. The Washington Post. 23 Apr 2018. Available: https://www.washingtonpost.com/business/economy/how-merchants-secretly-use-facebook-to-flood-amazon-with-fake-reviews/2018/04/23/5dad1e30-4392-11e8-8569-26fda6b404c7_story.html. Accessed 26 Sep 2020.

5.    Lee D, Murphy H. Facebook groups trading fake Amazon reviews remain rampant. Milwaukee Business Journal. 2020 [cited 26 Sep 2020]. Available: https://www.bizjournals.com/milwaukee/news/2020/08/13/facebook-fake-amazon-reviews.html

6.    Goldin M. Token-Curated Registries 1.0. ConsenSys; Available: https://docs.google.com/document/d/1BWWC__-Kmso9b7yCI_R7ysoGFIT9D_sfjH3axQsmB6E/edit

7.    2021 Best Colleges. In: US News and World Reports [Internet]. [cited 26 Sep 2020]. Available: https://www.usnews.com/best-colleges

8.    adChain – Medium. [cited 26 Sep 2020]. Available: https://medium.com/@AdChain

9.    Ha A. Dirt Protocol raises $3M for a decentralized, blockchain-based approach to information vetting. In: TechCrunch [Internet]. 11 Jul 2018 [cited 26 Sep 2020]. Available: http://techcrunch.com/2018/07/11/dirt-protocol/

10. Steem An incentivized, blockchain-based, public content platform. 2017 Aug. Available: https://steem.com/SteemWhitePaper.pdf

11. Bhatia A, Giometti R. Sapien: The Web3 Social Network of the Future. SapienNetwork; 2019 Feb. Available: https://github.com/SapienNetwork/Sapien-White-Paper

12. Digix. [cited 26 Sep 2020]. Available: https://digix.global/#/

13. 0x: Powering the decentralized exchange of tokens on Ethereum. [cited 26 Sep 2020]. Available: https://0x.org/

14. Tezos. [cited 26 Sep 2020]. Available: https://tezos.com/

15. CarbonVote. [cited 27 Sep 2020]. Available: http://carbonvote.com/

16. Tsoukalas G, Falk BH. Token-Weighted Crowdsourcing. Management Science. 2020;6. Available: https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2019.3515

Read the full article at https://doi.org/10.1287/mnsc.2019.3515.

Reference

Tsoukalas G, Falk BH (2020). Token-Weighted Crowdsourcing. Management Science 66(9): 3843-3859.

Comments