How Accurate is Review Sentiment Analysis?

by | June 7, 2016

Many marketing companies offer some sort of [itg-tooltip href=”http://www.travelmediagroup.com/hotel-marketing-glossary/#s” link-target=”_blank” tooltip-content=”<p>A report used to learn how customers feel about your hotel.<br /><em>Click to read more.</em></p>”]sentiment analysis[/itg-tooltip] for hotel reviews. Sentiment analysis is used in hotel reputation management to help add context to the star ratings a hotel receives. Sentiment analysis can be very simple (such as providing positive and negative sentiment word clouds) or quite comprehensive (such as providing advanced sentiment reports on 20-key factors).

Not all sentiment analysis solutions are created equal. When judging the success of your sentiment analysis reports hoteliers should answer the following:

  1. Is the data accurate?
  2. Is the data reliable?
  3. Is the data accessible?

Judging the Accuracy of Sentiment Analysis

Researchers recognize that when travelers leave reviews, their opinion can be offered in two different ways. A traveler can offer an explicit opinion, in which they directly share their sentiment (‰I loved the hotel room‰) or an implicit opinion, where the sentiment is implied (‰The hotel room was spacious‰). Most sentiment analysis products rely on a computer to ‰”scrape” the information from a review, and therefore can miss out on the true sentiment of an implicit statement.

Keeping this in mind, it‰’s important to ask questions about how the sentiment analysis data is generated. Many hotel marketing tools provide sentiment analysis by writing a computer program that finds keywords in your hotel reviews and then uses the overall review star rating to conclude if the sentiment is positive or negative. The problem with this system is that most traveler reviews are complex ‰- offering both positive and negative feedback within a review and assigning an overall score for their experience. Check out the example below from one of our clients to understand the flaw in a typical sentiment analysis system.

Hotel Review Example 1

Hotel Review Example

Although this review received a TripAdvisor score of 5/5, within the review text it mentions that a laundry machine was broken. A computer-generated sentiment analysis report would include ‰Positive or ‰5‰ for laundry, despite the fact that a human reading the report would recognize that this traveler would not rate the broken laundry machine 5/5.

Hotel Review Example 2

Hotel Review Example

In the second example, the overall score is 2/5, despite positive comments about the staff and the breakfast. Some sentiment analysis reports would assign a negative sentiment to ‰Staff and Breakfast‰ despite the positive feedback in this review.

It‰’s essential for hoteliers to understand how their sentiment analysis reports are generated to make sure they receive the most accurate information about guest experiences at their hotel. Our Respond & Resolve service includes a 20-point sentiment analysis report that is scored by our Reputation Analysts, ensuring that your reviews are analyzed by a human who understands the context of implicit statements. Your data is also completely accessible 24/7 in the TMG OneView™ platform.

How accurate is hotel review sentiment analysis?

0 Comments