ProposalGroup008-Sp22.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COGS 118A- Project Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Description\n",
    "\n",
    "You will design and execute a machine learning project. There are a few constraints on the nature of the allowed project. \n",
    "- The problem addressed will not be a \"toy problem\" or \"common training students problem\" like mtcars, iris, palmer penguins etc.\n",
    "- The dataset will have >1k observations and >5 variables. I'd prefer more like >10k observations and >10 variables. A general rule is that if you have >100x more observations than variables, your solution will likely generalize a lot better. The goal of training a supervised machine learning model is to learn the underlying pattern in a dataset in order to generalize well to unseen data, so choosing a large dataset is very important.\n",
    "\n",
    "- The project will include a model selection and/or feature selection component where you will be looking for the best setup to maximize the performance of your ML system.\n",
    "- You will evaluate the performance of your ML system using more than one appropriate metric\n",
    "- You will be writing a report describing and discussing these accomplishments\n",
    "\n",
    "\n",
    "Feel free to delete this description section when you hand in your proposal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Peer Review\n",
    "\n",
    "You will all have an opportunity to look at the Project Proposals of other groups to fuel your creativity and get more ideas for how you can improve your own projects. \n",
    "\n",
    "Both the project proposal and project checkpoint will have peer review."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Names\n",
    "\n",
    "Hopefully your team is at least this good. Obviously you should replace these with your names.\n",
    "\n",
    "- Caitlin Connolly\n",
    "- Carolyn Yatco\n",
    "- Abby Koornwinder\n",
    "- Joshua Widjanarko\n",
    "- Caike Campana"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Abstract \n",
    "Our project focuses on attempting to predict both a hotel's average score based on their reviews. We will be utilizing “515K Hotel Reviews Data in Europe” from Kaggle. This dataset contains 515,000 reviews from 1493 different hotels and contains the review which separated by the positive and negative portions, the number word count of the positive and negative portion respectively, locations of hotel, days since the review. In addition, we will be doing sentiment analysis within the positive and negative portions to get the degree of the positivity and negativity.After the sentiment analysis we will be testing different models(OLS, K-Nearest Neighbor, etc) and determining which model has the highest accuracy. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Background\n",
    "Beyond location, price, and amenities, consumers judge their perception on a possible hotel to stay out based on the experience “or reviews” of previous guests. For example, according to TripAdvisor, a site for hotel and restaurant reviews and booking, roughly 81% of people always or frequently check a hotel’s reviews before booking <a name=\"lorenz\"></a>[<sup>[1]</sup>]. However, not all reviews and experiences are weighted equally. According to Sparks, B.A, and Browning, noticed that how information is framed within the review as well as the the focus of the review itself makes the biggest difference in how much a review would affect the overall view of a hotel  <a name=\"admonish\"></a>[<sup>[2]</sup>]. This begs the question on whether a computer can take a look at these reviews and determine the trust(in the form of ratings) that a hotel has in a similar way to how humans prioritze certain ideas, information, and looking at the overall sentiments.. Sentiment analysis, the ability of extracting emotion, feeling, and other subjective states, has been used on a wide arrangement of different reviews from movies on Netflix to restaurant reviews <a name=\"sotanote\"></a>[<sup>[3]</sup>]. However, most work on reviews ends with extracting the overall positivity or negativity of the review, not seeing if we can predict or gain insight on the overall view or rating of the hotel, movie, or other thing being reviewed. \n",
    "\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem Statement\n",
    "\n",
    "Our goal is to create a classifier for hotels in the European continent. We are going to use data from a review website its features, such as: Location; Negative and Positive reviews; user assigned tags; review age, and all the other features available. Thus, we combine the features into a classifier model and predict what will be the overall score of the hotel."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data\n",
    "\n",
    "We are going to use the dataset [515K Hotel Reviews Data in Europe\n",
    "](https://www.kaggle.com/datasets/jiashenliu/515k-hotel-reviews-data-in-europe) from the repository Kaggle and generated by the user Jianshen Liu. \n",
    "\n",
    "The dataset contains 515000 reviews of 1493 hotels throughout Europe, it was collected from the hotel booking website [booking](www.booking.com). It has 17 features, and some very useful are the average score, hotel name, geographical latitude,nationality of the  geographical longitude, days since the review, negative word counts, and positive word counts.\n",
    "\n",
    "Each single observation contains one customer review with all the features. Despite the data being clean and with no missing values,we need to process the review text such that we have a quantifiable metric.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Proposed Solution\n",
    "\n",
    "Our proposed solution to predicting a hotel’s average score and an individual’s score of a hotel will have multiple parts: Preprocessing, data cleaning, sentiment analysis of written reviews, determining the best classifier for predicting scores, and finally, training, testing, and evaluation. <br>\n",
    "\n",
    "To begin, here are the libraries we plan on using. <br>\n",
    "import numpy as np<br>\n",
    "import pandas as pd<br>\n",
    "from nltk.corpus import stopwords<br>\n",
    "from wordcloud import STOPWORDS<br>\n",
    "import matplotlib.pyplot as plt<br>\n",
    "import string<br>\n",
    "\n",
    "To preprocess and clean our data we have to remove any entries with missing values as well as remove those that correspond to reviewers who did not write out reviews and instead only gave numerical scores. Since we plan on using sentiment analysis to classify the written reviews, we need to remove the entries that did not provide any additional information aside from a score. We then need to remove any columns that correspond to features we do not need. <br>\n",
    "Thus, the variables we plan on using are: <br>\n",
    "the review date<br>\n",
    "the average hotel score<br>\n",
    "hotel name<br>\n",
    "reviewer nationality<br>\n",
    "negative review <br>\n",
    "positive review<br>\n",
    "the number of reviews the reviewer has given in the past<br>\n",
    "total number of valid reviews that the hotel has<br>\n",
    "tags the reviewer has given the hotel. <br>\n",
    "\n",
    "\n",
    "In order to perform sentiment analysis on the reviews we must perform some text cleaning. Namely, getting rid of stopwords, punctuation, and converting all letters to lowercase. We plan on performing a sentiment classification on the written reviews that will make the data quantitatively easier to put into our hotel rating prediction classifier. One way we can perform sentiment classification on the reviews is through using logistic regression to determine how positive or negative a review is. \n",
    "\n",
    "Next, we want to determine a classification model that would best predict the hotel ratings. The specific input we plan on using in this model are the nine variables listed above. Some potential models we could use include OLS and KNN. For instance, if we went with KNN we could use the ten closest neighbors and find the mean score of them.  We could also use a recurrent neural network (RNN) using Keras to perform sentiment analysis and classify the reviews.\n",
    "\n",
    "To split our data into train and test sets, we will use 80% for training and 20% for testing, choosing what entry goes into which set randomly. Once we train our model we will test it, run our evaluation metrics, and plot our results using Matplotlib. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation Metrics\n",
    "\n",
    "For our evaluation metrics, we have chosen to use the precision and recall scores to evaluate how many of our positive reviews are actually positive and how many of our negative reviews are actually negative. This will allow us to determine how good our model is at distinguishing positive from negative and therefore measure the accuracy of our model. \n",
    "\n",
    "\n",
    "To determine the errors we will use a confusion matrix to display the summary of our evaluations and calculate the errors made in our classification model. Through our confusion matrix, we will determine the sensitivity and specificity of our model, and our goal is to have high specificity and high sensitivity. As we tweak our model, we will compare the confusion matrices and re-train our model according to the results. \n",
    "\n",
    "\n",
    "A confusion matrix has two columns and two rows, where the columns are the true positive and negative conditions and the rows are the predicted positive and negative conditions. We have provided an example of what our confusion matrix might look like below.\n",
    "\n",
    "![img1](images/ex_confusion_matrix)\n",
    "\n",
    "For a confusion matrix we want to reduce the amount of errors our model makes and maximize our precision and recall scores while running our model on our testing data.\n",
    "\n",
    "The precision score (true positive rate) is calculated at the amount of true positives divided by the total amount of positives (true positives and false positives). \n",
    "\n",
    "        Precision = true positives / (true positives + false positives)\n",
    "\n",
    "The recall score (true negative rate) is calculated as the amount of true negatives divided by the total amount of negatives (true negatives and false negatives). \n",
    "\n",
    "        Recall = true negatives / (true negatives + false negatives)\n",
    "\n",
    "The below table from Lecture 12 slides (subsequently taken from https://en.wikipedia.org/wiki/Precision_and_recall)[<sup>[4]</sup>] contains the mathematical representations from which the precision and recall are derived as well as contains other evaluation metrics which may prove useful as we begin to evaluate our model.\n",
    "\n",
    "![img2](images/wiki_prec_recall_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ethics & Privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before beginning this project, our team has acknowledged potential ethics and privacy concerns that may arise from our data and implementation. Primarily, it is important to note that the dataset we plan on using is legally obtained as it is data from Booking.com that has been made publicly available. We discovered this specific collection of data from Kaggle.com, posted by Jiashen Liu who curated the dataset and made it available to the public domain to copy and modify as we intend to do. \n",
    "It is also important to acknowledge the potential biases in the dataset. Most notably, each data entry includes the nationality of the user which is one of the many various features provided by the dataset that we can use to predict the user’s score of the hotel. We take this into consideration as it can result in a potential bias in our predictions. \n",
    "Our team has also taken into consideration the privacy of the responders and has assured that the privacy of all reviewers is upheld as there are no identifying personal features other than the reviewer’s nationality. However, though there may be personal information in specific written reviews, all information was freely given by the individual. Thus, the anonymity of individuals is maintained. One way we can address these ethical and privacy concerns is through using an ethics checklist that addresses important ethical considerations in data collection, modeling, and analysis. A useful tool we can use to add such a checklist to our project is the command line tool, Deon. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Team Expectations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our team expectations strive to be in accordance with COGS 118A policies and guidelines. Each member has the responsibility to participate equally in all aspects of the project and communicate if any conflicts or difficulties arise. Meeting times will be planned in accordance with personal schedules and each member has a commitment to attend the meetings or make up any work if they are unable to attend. To keep things on a timely schedule each team member must be attentive to group conversations and keep in contact about any problems, ideas, or thoughts that could contribute to the project. In dividing any work among team members, each team member has the responsibility to contribute equally and finish their responsibilities at a timely manner. Additionally, each team member will be responsible for reviewing and communicating feedback as a whole before they are turned. If any conflicts arise, it is expected to be handled in a professional manner with consideration of all team members. Overall, it is expected that each team member will contribute equally and actively communicate to the rest of the team.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Timeline Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Replace this with something meaningful that is appropriate for your needs. It doesn't have to be something that fits this format.  It doesn't have to be set in stone... \"no battle plan survives contact with the enemy\". But you need a battle plan nonetheless, and you need to keep it updated so you understand what you are trying to accomplish, who's responsible for what, and what the expected due dates are for each item.\n",
    "\n",
    "| Meeting Date  | Meeting Time| Completed Before Meeting  | Discuss at Meeting |\n",
    "|---|---|---|---|\n",
    "| 4/21  |  7 PM |  Individual assigned for project proposal and write on shared document  | Review and turn in project proposal |\n",
    "| 4/25 |  3 PM |  Consider general project information, possible questions, and variables | Discuss peer review assignment and divide parts | \n",
    "| 4/29  | 3PM | Finish individual peer review assignments  | General review of peer review assignment and turn in |\n",
    "| 5/2  | 3PM |Consider specifics about the data and think about ideas to discuss with group; background research| Discuss how to clean data; assign work to do individually |\n",
    "| 5/4 | 7 PM  | Continue assigned work for datasets, import and wrangle data | Discuss EDA methods; begin programming|\n",
    "| 5/6  | 3 PM  | Assigned parts for EDA| Discuss next steps and possible features, algorithms, and metrics; look at EDA analysis and discuss |\n",
    "| 5/9  |3 PM | Work on possible pilot results | Touch base with work and continuing pilot results  | \n",
    "|5/11|7 PM|Work on check point 1|Discuss individual work for checkpoint 1 and assign parts for peer review| \n",
    "|5/13|3 PM|Assigned coding for each member; work check point 1|Review all code written and make edits if needed, finalize and turn in check point 1| \n",
    "|5/16|3 PM|Work on coding and testing different possible models|Discuss models, finalize project solutions and assign programming; assign check point  peer review work| \n",
    "|5/20|7 PM|Finished assigned parts for peer reviews|Finalize and turn in checkpoint peer reviews| \n",
    "|5/27|3 PM|Refine coding and algorithms|Discuss final details and assign coding work to be done before next meeting| \n",
    "|6/1|7 PM|Work on assigned parts and be prepared to show teammates|Discuss collected work and divide work for end of project| \n",
    "|6/8|3 PM|Complete team eval survey separately|Overall review of final project, finalize, and turn in|\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Footnotes\n",
    "<a name=\"lorenznote\"></a>1.[^](#lorenz):  TripAdvisor. (2019, July 16). Online reviews remain a trusted source of information when booking trips, reveals New Research. Retrieved April 24, 2022, from https://www.prnewswire.com/news-releases/online-reviews-remain-a-trusted-source-of-information-when-booking-trips-reveals-new-research-300885097.html \n",
    "<br> \n",
    "<a name=\"admonishnote\"></a>2.[^](#admonish): Sparks, B.A., & Browning, V. (2011). The impact of online reviews on hotel booking intentions and perception of trust. Tourism Management, 32, 1310-1323.\n",
    "<br>\n",
    "<a name=\"sotanote\"></a>3.[^](#sota):An example of one of these uses and how they approached sentiment analysis: https://towardsdatascience.com/customer-reviews-analysis-using-nlp-the-netflix-use-case-92b3645770e1\n",
    "<br>\n",
    "<a name=\"evalmetric\"></a>4.[^](#evalmetric):Wikimedia Foundation. (2022, March 31). Precision and recall. Wikipedia. Retrieved April 24, 2022, from https://en.wikipedia.org/wiki/Precision_and_recall "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}