What does it mean for Airbnb to eliminate positional bias in search results on its site? Follow the lead of users, according to a paper submitted to Cornell University by Airbnb engineers.

The paper outlines how the company describes its approach to search and the “most significant improvements in tackling inventory” when using a DNN, a web content management system and web application framework based on Microsoft.NET.

Inventory, or course, for Airbnb means property rental listings. The paper details the challenges and the changes made to improve the treatment of new listings on Airbnb’s platform and to help other companies that use ranking teams that are transitioning to deep learning.

One of the challenges points to “historically under-represented” listings or data that is not being ranked properly due to positional bias. Focusing on user problems, rather than what the company thought were the problems, helped to solve this challenge. 

And, yes, it’s a little geeky, but the engineers describe the launch of deep-learning technology for Airbnb’s two-sided marketplace that brings together owners of rental property and renters.

The process generated ideas from machine learning that eventually produced gains in their rental bookings. But it also led to challenges and a complete revision of Airbnb’s strategy on how to iterate on deep learning beyond its initial launch of the changes.

Deep learning added more layers to the site’s architecture, which complicated the situation. At first increasing layers did not deliver any gains in bookings from listings.

After testing the platform, developers moved to what they call a “users lead, model follows” strategy. This means to first quantify a user problem and then tweak the model in response.

As it turned out, the series of successful ranking model launches described were not only associated not only with an increase in bookings, but a reduction in the average listing price of search results, according to the paper.

“This indicated the model iterations were moving closer to the price preference of guests, which was lower than what the previous models had estimated,” according to the paper. “We suspected that even after the successive price reductions, there was likely a gap between the model’s choice of prices and what guests preferred.”

So to quantify this gap, engineers analyzed the distribution of the difference between the median price of search results seen by a guest and the price of the listing that the guest booked. The difference gets computed after taking a log of the prices. (You can read the entire process here.)

The process allowed Airbnb to improve the quality of search results on its site.

In conclusion, developers explains that deep learning continues to flourish in search ranking at Airbnb. The highlight of “our journey is the realization that to push the boundaries of our DNNs, the inspiration was not going to come from some external source. For that we had to follow the lead of our users.”