Sentiment and Opinion

Sentiment and Opinion Sep18, 2012 Analysis of Social Media Seminar William Cohen

First assignment: due Friday • Go to http://malt/mw • Create an account for yourself • use andrew id • Go to your user page • Your real name & a link to your home page • Preferably a picture • Who you are and what you hope to get out of the class (Let me know if you’re just auditing) • Any special skills you have, research interests that you have, related projects you have been or might be working on, etc.

Outline • Announcements • Recap • With a little more on word senses • More discussion: what exactly is subjectivity, sentiment and polarity? • Annotating a corpus for subjectivity • Fine-grained sentiment for reviews • More distinctions: • Agreement and discourse

In our previous episode…

Motivations: sentiment common… Analysis : modeling & learning People Communication, Language Social Media Networks

…and important… • Product review mining:What features of the ThinkPad T43 do customers like and which do they dislike? • Review classification:Is a review positive or negative toward the movie? • Tracking sentiments toward topics over time:Is anger ratcheting up or cooling down? • Etc. [These are all ways to summarize one sort of content that is common on blogs, bboards, newsgroups, etc. –W]

…and non-trivial

What units do we attach sentiment to? • Individual words (“nice”, “comfortable”) • Phrases (“slow service”) • Sentences? • Documents? • … ?

Hatzivassiloglou & McKeown 1997 Build a graph of adjectives linked by the same or different semantic orientation (determined by conjunctions)… scenic nice terrible painful handsome fun expensive ICWSM 2008 9 comfortable

Hatzivassiloglou & McKeown 1997 …and a clustering algorithm partitions the adjectives into two subsets + slow scenic nice terrible handsome painful fun expensive ICWSM 2008 10 comfortable

Senses Word senses Jan - ICWSM 2008

Senses Is this polar? Jan - ICWSM 2008

Non-subjective senses of brilliant • Method for identifying brilliant material in paint - US Patent 7035464 • In a classic pasodoble, an opening section in the minor mode features a brilliant trumpet melody, while the second section in the relative major begins with the violins. Jan - ICWSM 2008

Subjective Sense Examples • His alarm grew Alarm, dismay, consternation – (fear resulting form the awareness of danger) • Fear, fearfulness, fright – (an emotion experiences in anticipation of some specific pain or danger (usually accompanied by a desire to flee or fight)) • He was boilingwith anger Seethe, boil – (be in an agitated emotional state; “The customer was seething with anger”) • Be – (have the quality of being; (copula, used with an adjective or a predicate noun); “John is rich”; “This is not a good answer”) S N S N ICWSM 2008

Objective Sense Examples • The alarmwent off Alarm, warning device, alarm system – (a device that signals the occurrence of some undesirable event) • Device – (an instrumentality invented for a particular purpose; “the device is small enough to wear on your wrist”; “a device intended to conserve water” • The water boiled Boil – (come to the boiling point and change from a liquid to vapor; “Water boils at 100 degrees Celsius”) • Change state, turn – (undergo a transformation or a change of position or action) ICWSM 2008

Objective Senses: Observation • We don’tnecessarily expect phrases/sentences containing objective senses to be objective • Will someone shut that darn alarm off? • Can’t you even boil water? • Subjective, but notdue toalarm and boil ICWSM 2008

Objective Sense Definition • When the sense is used in a text or conversation, we don’t expect it to express subjectivity and,if the phrase/sentence containing it issubjective, the subjectivity is due tosomething else. ICWSM 2008

Hatzivassiloglou & McKeown 1997 • Later/related work: • LIWC, General Inquirer, other hand-built lexicons • Turney & Littman, TOIS 2003: Similar performance with 100M word corpus and PMI – higher accuracy better if you allow abstention on 25% of the “hard” cases. • Kamps et al, LREC 04: Determine orientation by graph analysis of Wordnet (distance to “good”, “bad” in graph determined by synonymy relation) • SentiWordNet, Esuli and Sebastiani, LREC 06: Similar to Kamps et al, also using a BOW classifier and WordNet glosses (definitions).

Turney 2002 • Goal: classify reviews as “positive” or “negative”. • Epinions “[not] recommended” as given by authors. • Method: • Find (possibly) meaningful phrases from review (e.g., “bright display”, “inspiring lecture”, …), • based on POS patterns, like ADJ NOUN • Estimate “semantic orientation” of each candidate phrase • Based on pointwise mutual information: Altavista counts of phrase’s cooccurrence with “excellent”, “poor” • Assign overall orentation of review by averaging orentation of the phrases in the review

Pang et al EMNLP 2002

Pang & Lee EMNLP 2004

Methods: 2002 • Movie review classification as pos/neg. • Method one: count human-provided polar words (sort of like Turney): • Eg, “love, wonderful, best, great, superb, still, beautiful”vs “bad, worst, stupid, waste, boring, ?, !” gives69% accuracy on 700+/700- movie reviews • Method two: plain ‘ol text classification • Eg, Naïve Bayes bag of words: 78.7; SVM-lite “set of words”: 82.9 was best result • Adding bigrams and/or POS tags doesn’t change things much.

Pang & Lee EMNLP 2004 • Can you capture the discourse in the document? • Expect longish runs of subjective text and longish runs of objective text. • Can you tell which is which? • Idea: • Classify sentences as subjective/objective, based on two corpora: short biased reviews, and IMDB plot summaries. • Smooth classifications to promote longish homogeneous sections. • Classify polarity based on the K“most subjective” sentences

Outline • Announcements • Recap • With a little more on word senses • More discussion: what exactly is subjectivity, sentiment and polarity? • Annotating a corpus for subjectivity • Fine-grained sentiment for reviews • More distinctions: • Agreement and discourse

Manual and Automatic Subjectivity and Sentiment Analysis Jan Wiebe Josef Ruppenhofer Swapna Somasundaran University of Pittsburgh

Everyone knows that dragons don't exist. But while this simplistic formulation may satisfy the layman, it does not suffice for the scientific mind. The School of Higher Neantical Nillity is in fact wholly unconcerned with what does exist. Indeed, the banality of existence has been so amply demonstrated, there is no need for us to discuss it any further here. The brilliant Cerebron, attacking the problem analytically, discovered three distinct kinds of dragon: the mythical, the chimerical, and the purely hypothetical. They were all, one might say, nonexistent, but each nonexisted in an entirely different way... - Stanislaw Lem, “The Cyberiad”

Preliminaries • What do we mean by subjectivity? • The linguistic expression of somebody’s emotions, sentiments, evaluations, opinions, beliefs, speculations, etc. • Wow, this is my 4th Olympus camera. • Staley declared it to be “one hell of a collection”. • Most voters believethat he's not going to raise their taxes

Corpus AnnotationWiebe, Wilson, Cardie 2005Annotating Expressions of Opinions and Emotions in Language Leaving aside what’s possible, what sort of inferences about sentiment, opinion, etc would we like to be able to make?

Overview • Fine-grained: expression-level rather than sentence or document level • The photo quality was the best that I have seen in a camera. • The photo quality was the best that I have seen in a camera. • Annotate • expressions of opinions, evaluations, emotions • material attributed to a source, but presented objectively

Overview • Fine-grained: expression-level rather than sentence or document level • The photo quality was the best that I have seen in a camera. • The photo quality was the best that I have seen in a camera. • Annotate • expressions of opinions, evaluations, emotions, beliefs • material attributed to a source, but presented objectively

Overview • Opinions, evaluations, emotions, speculations are private states. • They areexpressed in language by subjective expressions. Private state: state that is not open to objective observation or verification. Quirk, Greenbaum, Leech, Svartvik (1985). A Comprehensive Grammar of the English Language.

Overview • Focus on three ways private states are expressed in language • Direct subjective expressions • Expressive subjective elements • Objective speech events

Direct Subjective Expressions • Direct mentions of private states The United Statesfearsa spill-over from the anti-terrorist campaign. • Private states expressed inspeech events “I fear electoral fraud,” Tsvangirai said. Fear is a private state Fear is a private state but not of the author

Direct Subjective Expressions • Direct mentions of private states The United Statesfearsa spill-over from the anti-terrorist campaign. • Private states expressed inspeech events “We foresaw electoral fraud but not daylight robbery,” Tsvangirai said. Fear is a private state This implies a private state, so it’s not direct..

Expressive Subjective Elements [Banfield 1982] • “We foresaw electoral fraud but not daylight robbery,” Tsvangirai said • The part of the US human rights report about China is full of absurdities and fabrications Understood as implying certain mental state • Compare: • “We foresaw difficulties with the electoral process but not to this extent”, Tsvangirai said. • The part of the US human rights report about China contains many statements that we were unable to verify.

Objective Speech Events • Material attributed to a source, but presented as objective fact The government, itadded, has amended the Pakistan Citizenship Act 10 of 1951 to enable women of Pakistani descent to claim Pakistani nationality for their children born to foreign husbands. [What does this have to do with opinion? You need it to sort out who has opinions about what… -W]

An example…

“The report is full of absurdities,’’ Xirao-Nima said the next day. Nested Sources (Writer)

“The report is full of absurdities,’’ Xirao-Nima said the next day. Nested Sources (Writer, Xirao-Nima)

“The report is full of absurdities,’’ Xirao-Nima said the next day. Nested Sources (Writer Xirao-Nima) (Writer Xirao-Nima)

“The report is full of absurdities,” Xirao-Nima said the next day. Objective speech event anchor:the entire sentence source: <writer> implicit: true Attributes: The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state. [Where to ‘hang’ the annotation’ -W] The source is the person to whom the private state is attributed. Note that this can be a chain of people. The target is the content of the private state or what the private state is about. Attitude type: If not specified, it is to be understood as neutral but can be set to positive or negative as required. Intensity records the intensity of “the private state as a whole.” Direct subjective anchor:said source: <writer, Xirao-Nima> intensity: high expression intensity: neutral attitude type:negative target:report Expressive subjective element anchor:full of absurdities source: <writer, Xirao-Nima> intensity: high attitude type: negative

Another example…

“The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities. ICWSM 2008

(Writer) “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities. ICWSM 2008

(writer, Xirao-Nima) “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities. ICWSM 2008

(writer, Xirao-Nima, US) “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities. ICWSM 2008

(Writer) (writer, Xirao-Nima, US) (writer, Xirao-Nima) “The US fears a spill-over’’, said Xirao-Nima, a professor of foreign affairs at the Central University for Nationalities. ICWSM 2008

Sentiment and Opinion