A new way to narrow down property search results using machine learning driven contextual categories.
Having joined a newly formed Machine Learning Acceleration team we had to demonstrate value of the team to the business by tackling bold ideas using machine learning.
We decided to tackle one our most high traffic and high value parts of the experience - narrowing down search. I ran a number of workshops, worked with stakeholder, inspired and led conversations with data scientist to create a number of features to help users narrow down their search.
Among a few other features, we created a new way to narrow down property results using machine learning driven contextual categories. All of the features we delivered in this spaced created substansial value to the business. Not only monetary, but also inspired other teams to adopt ML in their product areas.
Defining the problem space
To kick off the project I ran a workshop with senior stakeholders to decide which part of the user journey on our platform we should focus. Out of number of potential areas we decided to help users narrow down their search option. There were couple of reasons why - the high amount of traffic in this area would allow us to utilise ML best and there was an opportunity to solve a number of known pain points for our users.
Understanding user intent
Narrowing down search is a highly complex task for a user, they have to constantly weigh their trip objectives with what the property has to offer. So to really understand which area of narrowing down search we should tackle I looked through existing research and started drafting a framework that would later become a ‘Customer Intent Framework’.
Leading by design
The framework was well received and soon got some traction from senior stakeholder as an exciting proposition to tackle. However it has proven difficult to engage Data Scientist in something that was very conceptual. Because of this I organised and ran a workshop with the whole team to brainstorm some initial concepts. Showcasing some sketched out concepts made DS buy into the idea. We then worked with them to discuss algorithm options and tweaks, as well as any data capture needs.
Testing and prioritising concepts
After spending some time refining the concepts into design prototypes and sharing them for feedback. I then worked with the research team to run a lab study on a couple of prioritised concepts, we chose to use live prototypes with sample data to try and make the environment as close as possible to actual ML output. This proved to be a bit challenging and something I learned for the future, that it can take a substantial amount of time to integrate sample .json into live prototype.
Sharing the knowledge
However even with the prototypes being finished 1 hour before the study we managed to get some great findings. Not only we learned a lot about our features, we also learned a lot about user perceptions towards ML and personalisation from our interview part of the research. The findings presentation attracted great interest from other parts of the business which led me to arrange a ML best practises exchange between different teams.
Building contextual categories
After making some changes and tweaks following the research we flew to Bangalore in India to onboard the technical team on the upcoming features and get feedback on proposed designs. After running workshops we prioritised the first feature to be built - contextual categories. We kicked off development straight after. During the development we had to tweak our approach a few times, as I learned that with ML driven features there's more unpredictability in the outcome and fallback/edge case scenarios to consider, as well as, more dependancies with other services on the website.
Having built the feature we launched it as a MVT - the initial results was conversion to purchase negative, but had some positive indications for user behaviour metrics. I then worked with product, analytics and data science to rapidly iterate the feature. After 5 iterations we turned the feature conversion positive, increased CTR for data science and reduces some negative user behaviour metrics.
2020 © BENAS SKRIPKA