Yelp restaurants analysis

Data Exploration on Yelp

Yutian Luo | University of Maryland, College Park

Introduction Background As the online community becomes more popular, more people are willing to share personal experiences in restaurants, evaluating food on the Internet, and using online information to help them decide. The goal of this project is to perform data analysis on the Yelp restaurant dataset, which consists of customer review texts, restaurant rankings, location information, etc. from a variety of restaurants.

At a high level, aiming to find out which attributes of the restaurants lead to higher restaurant ratings and making further inferences about the restaurant using text reviews. More specifically, our questions of interest consist of 3 perspectives: user, restaurant, and review. Study users’ ratings and reviewing patterns; find out the characteristics of high rating restaurants and basing the characteristics of reviews to deliver better recommendations for Yelp’s users, both customers, and restaurants.

By gathering insights about the customer base and developing strategic factors that would influence a customer’s decision to visit a particular restaurant, not only can help diners find the most desired restaurant by attributes such as quality of a restaurant or best cuisines but also can provide recommendations for restaurants to expand their business by attracting more customers and by improving clients experiences and targeting particular customers. Besides, for Yelp's perspective, we want to help Yelp to improve its recommendation system.

Data Description Chose the Yelp dataset from Kaggle because the data is feasible and has potential due to large volumes with 10 GB. Besides, since the information is gathered from the Yelp website, which is one of the most renowned review platforms which had a monthly average of 76.7 million unique visitors via its mobile website in 2019, it is authentic and able to develop practical insights.

The dataset includes 5.2 million user reviews of 174,000 businesses in 11 metropolitan areas in four countries. To avoid out-of-memory error, analysis focused on data of restaurants in Nevada, which is the state with the highest total amount of review counts across all U.S. states.

We acquired 7 datasets in csv format: yelp_business_attributes.csv, yelp_business.csv, yelp_business_hours.csv, yelp_checkin.csv, yelp_tip.csv, yelp_review.csv, yelp_user.csv.

Data Acquisition Dataset Preparation

Imported our Yelp data in CSV format using method pd.read_csv directly from Google Shared Drive, aiming at efficient group collaboration and code sharing. Applied data-type schema for faster read to solve large-volume data loading issues. Loaded data with chunk size 300000 to keep memory safety.

Data Selection Counts of Reviews in Different States To narrow down the research scope and avoid computer capacity issues, we created a bar chart of all review counts in different states. As the plot shows, Nevada has the highest number of review counts (only show top 10 data) and following analysis will focus on Nevada since its data is abundant.

Manually looking down to all 1294 categories and filtered categories related to food and beverages, such as bars and restaurants while omitted other categories such as sports and entertainment. Saved the selected categories as categories.txt on the shared drive. Filtered Nevada restaurants basing on the text and merged dataset. The shape of Nevada restaurants is 1185998 rows and 49 columns. Applied to combined data frame, which consisted of Yelp business and the respective business hour, user, and user review.

Data Cleaning To make sure there are no null values in columns of dataset. Ignoring column with null value like "neighborhood", "postal_code", and "name_usr" since they will not affect our analysis. For useful columns, all filled with value. Manually selected all kinds of cuisines.

Exploratory Data Analysis

“Which restaurant worth to go?” “How can restaurants attract more customers?” “Is this review fairness enough?” “As a restaurant, how to improve my rating?” “Can rating reflect customers’ real experience?”

“Can rating represents restaurant?” Aiming to answer these questions, separated analysis into three aspects: users, restaurants, and review.

Users Central User Impact Analysis Certain reviewers have more friends who are also active on the review platform, so we define them as “central users”. When people trying to decide which restaurants to go, many of them tend to believe reviews wrote by “central users”. We want to know how the ratings given by central users correlate to the corresponding restaurant average ratings. i.e. Are the values of these users’ ratings close to real average restaurant ratings? We will select the top 10% of the users who have the most fans on Yelp as central users. Deleted the first user whose fans is over 7000, which is too high to believe. After ignoring the outliers mentioned above, the values of rest statistics are very close to each other. The left side is restaurants’ rating overall status while right side is central users’ rating status.

Then plotted a dynamic multi-bar chart to show the comparison between ratings given by these central users and the corresponding restaurant average ratings. Blue bar indicates central users’ rantings and red bar indicates average ratings. The rating higher than 5 means this restaurant/bar has been reviewed by more than one central user.

The average ratings given by our central yelp users are close to the average ratings of the businesses in most cases, which consistent with the fact that their statistics values are very close and satisfy our prior expectations. This result can be understood in two aspects. First, central users can rate restaurants/bars very accurately and able to reflect truly quality of restaurants/bars. Second, the impact of ratings from central users on restaurants is very large. Besides, the ratings of central users are slightly higher than average business rating. Those people might overrate the restaurants, and since they have more fans, they have larger influences on potential customers.

Restaurants Major Attributes Impact Analysis What are the features that restaurants with high ratings have? Took a look at tags mentioned times by restaurants.

The most common attributes mentioned more than 500 times for a restaurant in NV are: "bike parking", "business parking valet", "wheelchair accessible", "business parking street", "happy hour" and "dogs allowed". Used these attributes to generate correlation heatmap.

Yelp allows users to search for restaurants based on tags, which is also an important indicator for yelp users to decide which restaurants to go to. To better analyze the impact of these tags, also known as the restaurants’ attributes, we only keep 6 attributes appear more than 500 times and put other 75 attributes in OtherAll. Then we create a correlation heatmap and found that “dogs allowed” have highest negative effect, “bike parking” have highest positive effect. And Correlation among attributes is low, so we can simply apply linear regression to build our model: y=3.47+0.13*x1+0.14*x2-0.10*x3-0.23*x4-0.11*x5-0.92*x6+0.23*x7

If we get new information about a restaurant, we can simply use this model to predict its average star. By comparing it to the actual star, we will know this restaurant is overrated or underrated.

Yelp can give new joining restaurant a new tag called “OVERRATED” or “FUTURE STAR” by comparing stars calculated by our model with the actual star to help our customer deciding. “Dogs allowed” have highest negative coefficient and “business parking valet” have highest positive coefficient, which consistent with our correlation heatmap. According to our model, simple attributes like “bike parking” and “business parking valet” are more likely to benefit restaurants star rating; special attributes like “wheelchair accessible” and “dogs allowed” tend to decrease the restaurants’ star rating. These special tags invisibly raise users’ expectations for restaurants, resulting in users tending rating lower. For those restaurants, Yelp can give them a virtual medal “GOOD OWNER” to encourage them and make up for the risk of adding these tags.

Famous Cuisine Analysis What are the top five frequent cuisines in Nevada? What are the top restaurants by star rating for each cuisine? Manually filtered all cuisine under categories and from the dataset, top five most popular cuisines are Thai, French, Korean, Hawaiian, and Vietnamese. Then, selected restaurants belong to these cuisines and sorted by star ratings. "Yanni's Bistro" and "Bonjour Bistro" have highest review ratings for French Food. "Hon Pocha" and "Island Style 2" have highest review ratings for Korean Food. "Poke Express" has highest review ratings for Hawaiian Food. "Bosa Boba Cafe" has highest review ratings for Vietnamese Food. Finally, "Mr. Bangkok" and "Andrada Muay Thai" have highest review ratings for Thai Food. Mapped these restaurants as following.

50% of restaurants are located in Las Vegas, and they are very spread out. There might be two reasons: First, Las Vegas is one of the most popular cities for traveling in America, so it has a lot of Foreign tourists coming in every day. Most foreign tourists tend to choose their home food, which results in the popularity of restaurants that provide foreign cuisines. Second, the population in Las Vegas is relatively large

compared to other cities, so the dining market is more competitive, so the possibility that Las Vegas has better restaurants is much higher because of the competition.

Operation Recommendation Analysis What is the relation between restaurants opening time and their review stars? Filtered out null values of the Monday operating hours and extracted the Monday opening hour then transformed it to date format.

Plotted the time series line chart to see the results.

From this graph, the relationship between average stars and restaurant open time fluctuates throughout the day. Generally speaking, the average star rating is lower when the restaurant opens at night after 8 pm and higher when it opens at lunchtime (around 12 pm) or dinner time (around 6 pm). There might be several reasons for this trend. First, the quality of the food may be higher at lunchtime or dinner time, as they are mostly freshly made. While at late night, the ingredients may not be fresh. Second, the atmosphere of the restaurants might be better during lunchtime or dinner time, as some restaurants offer special services such as live music during those times. Third, customers tend to get drunk at late night, so they may not have an accurate opinion about the restaurants. Fourth, at lunch or dinner time, people tend to be hungry and easy to be satisfied, so they are more generous to give higher stars.

Review Review text analysis What are some keywords that might correlate with a bad review? What common topics did customers talk about when they also give the restaurant a very high star rating? We would like to know if there is a difference in the types of keywords customers mention in their reviews when they rank the restaurant with 4 or 5 stars compared to 1 or 2 stars.

Removed non-restaurant review instances from the data frame containing restaurant star rating and table containing restaurant reviews. Observed the word cloud in two blocks below, and then refined stop words by removing common and uninformative words such as place, food in our word cloud. Generated two word-clouds as following, warm tones for high rating reviews, cold tones for low rating reviews.

Compared word-clouds, "service" is the most frequent words in low rating reviews; while in high-star reviews, a common word is “delicious”, describing the dishes. It appears that bad services might be a deciding factor for people to leave a low star rating. Also, several common topics people care in both high star reviews and low star reviews are service/experience, pricing, location, and atmosphere. Implement a more holistic star rating system where users can rate on these aspects separately might help Yelp generates a more fair rating.

Sentiment Analysis Are the customer ratings in line with their text reviews? i.e. Do customers contradict themselves when they rate the restaurant overall from 1-5 and when they write a text review? From common sense, most customer ratings are in line with their text reviews. To verify that, introduced a new numerical variable called “sentiment”, which is calculated by the positive and negative words found in review text. Apply sentiment analysis on each of the review text, should return average sentiment score with a range of -1 to 1. The correlation between sentiment and review stars is 61.52%, and relations are as follows.

Used Tensorflow tokenizer to remove punctuation, lower strings, and tokenized strings internally, then made text transformation to padded sequences and built Bidirectional LSTM model. Compiled and train our model. Fitted model and use it to predict multi-classification problem. From our accuracy of 68.78%, the customer ratings are mostly in line with their text reviews, but still, some customers contradict themselves when they rate the restaurant overall from 1-5 and when they write reviews. Introducing granular rating metrics: food quality, cleanliness, service, atmosphere so that users can express different opinions in different areas might increase sentiment prediction accuracy. Other than this, training sentiment scores by comparing it with restaurants’ stars and get rid of useless words in our word bags might improve our model as well.

Recommendations Users From the result, ratings of central users are slightly higher than average restaurant ratings, which indicates that people who have more fans tend to overrate the restaurants. Taking into consideration the fact that people tend to read central users’ reviews for suggestions, restaurants might pay central users to write better reviews. Yelp can give “MEDAL” to central users whose ratings are close to the real restaurant average ratings calculated one week after to avoid users giving out the same rating to get “MEDAL”. More “MEDAL” these central users have, more trustworthy reviews are.

Restaurants Yelp allows users to search for restaurants based on tags, which is also an important indicator for yelp users to decide which restaurants to go to. The average star rating is lower when the restaurant opens at

night after 8 pm and higher when it opens at lunchtime or dinner time. Restaurants open their business of the day around 12 pm or 6 pm and provide special services or dishes during those times are more profitable. Yelp can give a new tag like “OWL FRIENDLY”. From regression model, Yelp can give a new joining restaurant a new tag called “OVERRATED” or “FUTURE STAR” by comparing stars calculated by our model with the actual star to help customers choose. Since some special tags like “dogs allowed” tend to decrease rating, Yelp can give those restaurants with special attributes a virtual medal like “GOOD OWNER” to encourage them.

Review Word cloud of reviews informs us that service is a deciding factor leading to low star rating reviews. It also provides topic keywords. Currently, these valuable information is not available on Yelp at a glance.

Therefore, implement tags created from popular keywords in reviews for each Restaurant on the front end of Yelp might improve user experience. For example, tags like "good service", "fresh", "familyfriendly" can be updated in real-time. On the back end, provide review keyword graphs for each restaurant owner so that they can see what customers are saying about their restaurant and according to this to improve.

Credibility is the basis for yelp to become an excellent referencing website in the industry. Fair ratings help customers choose the ideal restaurant and attract more users, in turn, Yelp can also help restaurants to improve through user feedback, through which they together constitute abundant ecosphere. Whether users' ratings can truly reflect their dining experience is critical. Yelp can introduce granular rating metrics: food quality, service, atmosphere so that users can express different opinions focusing on different aspects. With new metrics, Yelp can get more fair ratings basing different priorities. With the sentiment scores of reviews and LSTM classification model, Yelp can apply model in star prediction, especially in giving all fair stars for those new restaurants with few star ratings. Besides, Yelp can use the model to filter out abnormal reviews, where average sentiment in the review contradicted the review star rating.