Instructions

Instructions

Please read the following instructions and look over the screenshot of the interface that you will be using for the HIT. Once you have finished the instructions and feel ready to begin, click the Begin HIT button located at the bottom of this page.

For this HIT, you will be presented with a series of 10 examples taken from a dataset of Airbnb listings and the corresponding price per night of the listing. For each example, two different algorithms have been constructed using this dataset to predict the price per night of a listing using only the features of the listing (such as the number of bedrooms, the listing reviews and ratings, the amenities offered, etc.). A description of each feature can be found in the data dictionary.

Your task: For each example, try to determine to the best of your ability which of these two algorithms will perform more accurately in the "real world" on new listings which have not yet had a price set, and provide an estimate of how confident you are in your choice. Do not rush, but please try to select your choice promptly once you have decided on the more accurate algorithm. Each example presented will consist of 2 components:

  1. The two algorithms' predictions (and error) side-by-side for the same historic listing (which we know the price for).
  2. A chart of the features ranked that each algorithm considers important when making its prediction, and the corresponding weight of that importance. Importance is indicated by the magnitude (absolute value) of the weight: a large positive weight indicating positive correlation between the feature and price, and a large negative weight indicates a negative correlation with price.
NOTE: The algorithm with the lower error on the example presented will not necessarily always perform more accurately on listings in the "real world". You should use your intution about what features might be correlated with higher/lower Airbnb prices when determining your choice.

You are free to navigate to these instructions or the data dictionary using the buttons in the upper right of the interface as necessary to determine which algorithm you think will perform better in the real world. Also, each example may have a different number of important features shown (bars in the bar chart).

An example of the interface for a single example can be seen below. 👇


Interface

screen shot of the user interface for the experiment