How Opinions are Received by Online Communities: A Case...

Problem SettingSocial Feedback Mechanisms

Empirical EvidenceConclusions

How Opinions are Received by Online Communities:A Case Study on Amazon.com Helpfulness Votes

Cristian Danescu-Niculescu-Mizil1

Gueorgi Kossinets2

Jon Kleinberg1

Lillian Lee1

1Department of Computer Science, Cornell University2Google Inc. (formerly at the Department of Sociology, Cornell University)

04-23-2009 WWW2009

C. Danescu-Niculescu-Mizil, G. Kossinets, J. Kleinberg, L. Lee How Opinions are Received by Online Communities

Opinion vs. Meta-Opinion

Opinion:“What did Gary think of the book?”

[survey by Pang and Lee, 2008; Wu and Huberman, 2008]

Meta-Opinion:“What did the users think about Gary’s opinion of the book?”

[Ghose et al., 2008; Kim et al., 2006; Liu et al., 2007]Widespread in everyday life (e.g., political polls, market research)

Opinion:“What did B think of A?”

[survey by Pang and Lee, 2008; Wu and Huberman, 2008]

Meta-Opinion:“What did C think about B’s opinion of A?”

[Ghose et al., 2008; Kim et al., 2006; Liu et al., 2007]

Widespread in everyday life (e.g., political polls, market research)

Meta-Opinion:“What did C think about B’s opinion of A?”

Widespread in everyday life (e.g., political polls, marketresearch) and in online communities (e.g., Amazon.com,Facebook).

In this work we analyze and model the factors influencingmeta-opinions.

Amazon.com as a Testbed for Meta-Opinion Analysis

Data4,000,000 reviews700,000 products

Factors

TextualNon-Textual

Factors

TextualNon-Textual

Factors

TextualNon-Textual

FactorsTextual

Non-Textual

FactorsTextualNon-Textual

Thought Exercise

A product has a average star rating of 3:

Suppose your aim is to write a helpful review for that product.You can only alter the star rating of the review.Which would be your star rating choice?

Social Psychology Hypotheses

Hypotheses:ConformityBrilliant-but-cruelIndividual-bias

Conformity HypothesisSocial psychology of conformity[Bond et al., Psych. Bulletin, ’96]

→ A review is evaluated as more helpfulwhen its star rating is closer to the averagestar rating for the product.

Briliant-but-cruel Hypothesis[Amabile, J. Exp. Social Psych. ’83]:“negative reviewers are perceived as moreintelligent, competent, and expert thanpositive reviewers."

→ A review is evaluated as more helpfulwhen its star rating is below to the averagestar rating for the product.

Individual-bias HypothesisWhen a user considers a review, he or shewill rate it more highly if it expresses anopinion that he or she agrees with.

→ A review is evaluated as more helpfulwhen its star rating reflects the evaluators’personal opinion about the product.

Deviation from the mean

absolute deviation = |star rating − avg. star rating|

1←0←

Further away from the average

signed deviation = star rating − avg. star rating

−1←

Above averageBelow average

Individual-bias Hypothesis

Individual-bias Hypothesis→ A review is considered more helpful whenits star rating reflects the evaluators’ personalopinion about the product.

Individual-bias vs. Conformity

Recall: Conformity Hypothesis→ A review is considered more helpful whenits star rating reflects the reviewers’ averageopinion about that product.

Individual-bias vs. Conformity

Recall: Conformity Hypothesis→ A review is considered more helpful whenits star rating reflects the reviewers’ averageopinion about that product.

Individual-bias = Conformity ?Undistinguishable if everybody agrees.

Variance

Individual-bias = Conformity ?Undistinguishable if everybody agrees.

Variance

→ “Go with the average."

→ “Go slightly above the avg."

→ “Avoid the average."

Variance

Mixture of Opinion Model

Empirical observation: Model generated:

What about the actual text?

Hypotheses:ConformityBrilliant-but-cruelIndividual-biasText-only

Text-only HypothesisHelpfulness is evaluated purely on thetextual content of the review.

The non-textual factors are simplycorrelates of textual quality.

Controlling for Text

Manual re-evaluation of thetext helpfulness

human effortsubjectivity

Manual re-evaluation of thetext helpfulnessUse Machine Learning to evaluatetext helpfulness

How to interpret the mismatchesbetween predicted and actualhelpfulness?ML alg. errors and non-textualinfluence are indistinguishable.

Manual re-evaluation of thetext helpfulnessUse Machine Learning to evaluatetext helpfulnessSeed reviews in different non-textualcontexts

wait for a few years...submit to WWW 2011

Plagiarism!

Controling for TextAbout 1% of the reviews are plagiarized![David and Pinch, 2006]

(MH statistical significance test)

Plagiarism!

Controling for TextAbout 1% of the reviews are plagiarized![David and Pinch, 2006]

: the plagiarized reviews with deviation 0are significantly more helpful than those withdeviation 3(MH statistical significance test)

Conclusions

We have analyzed how the helpfulness of a review depends onthe other star ratings of the respective product.

This dependence contrasts with theories from social psychology.

We have discovered the important effects of variance onmeta-opinions.

We proposed a simple mathematical model that can account forthis apparently complex dependence.

By taking advantage of plagiarism we proved that helpfulnessevaluation does not only depends on the text, but also dependsdirectly on non-textual factors.

Thank you!

How Opinions are Received by Online Communities: A Case...

Documents

Transcript of How Opinions are Received by Online Communities: A Case...

Received: Dopamina – nie tylko neuroprzekaźnik* Accepted ... · Dopamina – nie tylko neuroprzekaźnik* Dopamine: not just a neurotransmitter Jakub Drożak, Jadwiga Bryła Zakład

Structure and functioning of the benthic communities ...

Received: 2016.06.16 Metody oznaczania właściwości · Słowa kluczowe: reaktywne formy tlenu, przeciwutleniacze, komórkowe testy przeciwutleniające Received: 2016.06.16 ... RFA

Received: Zastosowanie niacyny w terapii Accepted ... · W produktach pochodzenia roślinnego NA jest ... wyjścia było dla niego opracowanie zwierzęcego modelu pelagry ... tłuszczy

SSTTRRAATTEEGGIIEESS FFOORR CCRREEAATTIINNGG ... · Receiving communities are the geographic and social entities, i.e. towns, neighborhoods, and schools, where immigrants and refugees

Startup Stage#3 - Communities - Marek Wierzbicki - Hive53

Received: 2012.04.10 Inhibitory deacetylaz histonów ...

Kodeks ekologiczny (The ecological code)zsslemien.edu.pl/download/prezentacja ekologiczna.pdf · Akcja ekologiczna dla lokalnych mieszkańców (An ecological action for local communities)

ДИВЕРЗИТЕТ MAPs У НЕКИМ ФИТОЦЕНОЗАМА СТАРЕ …€¦ · communities. The sites Javor and Prelesje, forest community Fagetum moesiacae montanum B. Jov.

Vincent Vancaeyzeele - Local Energy Communities Vancaeyzeele... · Title: Microsoft PowerPoint - Vincent Vancaeyzeele - Local Energy Communities Author: arian Created Date: 6/7/2018

MonteLucca-Sitemap - Century Communities · Title: MonteLucca-Sitemap Created Date: 7/12/2016 2:55:52 PM

Received: Accepted: 2012.06.22 Bakteryjne topoizomerazy ... · topoizomerazy IA • antybiotyki • topologia chromosomu • regulacja ekspresji genów ... Bakteryjne topoizomerazy

INSTRUKCJA UŻYTKOWNIKA - Plus.pl RSRQ (Reference Signal Received Quality) SINR (Signal to Interference plus Noise Ratio) RSSI (Received Signal Strength Indication)

Email:sales@hbenterprisesllc.com | …medicalequipment.moonfruit.com/communities/5/004/013/610/... · 2018-08-02 · • Para mascotas diabéticas, perros de caza, razas juguetonas

BENTHIC ALGAL AND SEAGRASS COMMUNITIES IN BAA ATOLL, MALDIVESrepository.usp.ac.fj/5289/1/Benthic_Algal_and_Seagrass... · 2012-09-16 · BENTHIC ALGAL AND SEAGRASS COMMUNITIES IN

Received: 2013.11.13 Metody genetyczne w diagnostyce ...

Emerging communities in networks a flow of tieskulakowski/zakopane2014.pdf · Emerging communities in networks – a flow of ties Małgorzata J. Krawczyk, Przemysław Gawroński,

Creating Value In Our Communities Since 1915

Startup stage#3 communities -basia madej

Received: 2016.05.14 Patofizjologiczne podstawy ...