Cold Start To Jump Start
Get the first win right — everything else follows.
Mark Sage - 12 min read - 20/06/2024
During covid, Netflix became the go-to platform for lockdown households, increasing their subscriber base by 26m globally in the first half of 2020. Their popularity is, in part, due to the amazing content they both curate and create, but with an ever-increasing library of content, getting the right programme in front of the right viewer is critically important both for early onboarding and ongoing retention.
Netflix pioneered using recommendation engines to do this and it’s a core part of their success — even famously running a competition called the Netflix Prize for the best algorithm from 2006 to 2009. It clearly worked as they see 80% of stream time being achieved via their recommender systemand this then helps to create a user experience that improves customer retention — something post-covid they are increasingly in need of.
So for Netflix, the strength of the content shown to the viewer, underpinned by its selection through their data smarts, is both key to engagement and retention.
But what happens as a new user? This situation, known as a ‘cold-start’ is a common problem for recommendation systems that utilise user behavioural data to drive content decisions.
It was also a problem we had to solve for within yuu Rewards, the largest coalition loyalty programme in Hong Kong.
The yuu Rewards app is the main access into the loyalty programme and the key engagement driver within the app is the offer content shown on the home page like a ‘social feed’. As you’d expect within a social feed, the content needs to change regularly and have a sense of newness whenever the app is opened, and so getting this level of personalisation right was critical to the app and the programmes success.
To ensure this ‘offer feed’ content felt relevant to the member, we displayed and prioritised it based on their behaviour. However, at launch we had nothing — No members. No behaviours.
This was a real cold-start challenge for the yuu Rewards programme and something we had to solve for.
Netflix solve the problem in part by asking questions around interests during new user on-boarding and then utilise this to pull together the initial content suggestions. If however the user bypasses these questions, then they simply fall back to a set of popular titles.
This cold-start problem does have a fairly common solution too — using a second model or approach to ‘jump start’ the new users. This means there are essentially two models, one based on member behaviour (or stated interests) such as collaborative filtering and one based on meta data such as newest, most watched, best rated.
Within yuu Rewards, we decided to solve for the cold-start problem in the same way — leveraging two approaches to personalisation within a single unified offer delivery
As part of the yuu Rewards app design, we wanted it to be content heavy, bringing in offers and promotions from our banners and displaying these to members so that we could engage them out of store through our content.
Like Netflix, we also wanted this content to be immediately engaging to help with onboarding members such that they would be intrigued to browse and to dig further into the content and the yuu programme. This meant that the offers shown first needed to be those we considered most relevant for that member.
To enable this for launch, we developed our core personalisation design covering how we’d manage offer allocation and personalisation for our members. We initially had grand ambitions which included being able to take thousands of sales promotions already in-store and to personalise these for members at POS — however, this approach was rapidly scaled back due to the time available and the level of data and technical maturity.
Instead, we adopted a personalised offer strategy based on a mix of targeted, curated, loyalty offers, blended with a mix of prioritised and personalised mass sales promotion offers.
This would ensure that members would always have content available — even if we’re just highlighting in-store offers which are relevant to them — but through the one-to-one targeting, we could blend member specific offers to drive the ‘next best actions’.
So, the question now was how we’d actually make this work.
Facebook was a great place to start here. If anyone knew how to create engagement from content, arguably, at the time, Facebook were one of the masters.
Their post ranking algorithm famously used to use a system called EdgeRank which leveraged a ‘3T’s’ approach to content relevance scoring and was presented at the F8 Developer Conference in 2010. It showed how Facebook content was essentially scored based on its Type (e.g. status update, video, photo, etc.), it’s Tie or relationship strength with the user, and the Time, which looked at the aging of the post and hence relevance.
Although they have more recently updated their ranking algorithms to a more flexible machine learning based solution which reportedly looks at over 100k different attributes — for many years, the EdgeRank approach was what decided whether you saw stories from your favourite brands, your friends or your mum.
Within yuu Rewards we utilised a similar relevancy scoring approach to content personalisation but added an extra ‘T’ which looked at the customer ‘take’ or value returned.
Given we had a mixture of value across rewards, points and sales promotions we utilised a weighting mechanism to look at normalising the value of these different ‘currencies’ so that we could prioritise them against each other in terms of reward value relevance to the member.
This ended up giving us a 4Ts Loyalty Ranking Model for offer prioritisation which was based on:-
· Type refers to the business targeting/weighting of an offer such as mass, segmented, flash — what’s important to us
· Tie references the strength of relationship between the member and the offer. Initially measured by banner frequency this was expanded in complexity as the programme matured
· Time references the age of the offer such that it’s importance (and positioning) degrades over time
· Take is the value the member could get relative to the purchase and regardless of the reward (% discount, points, etc.)
It was this design then that we started to move forward on. Developing a recommendation system based on an offline scoring approach that would use overnight processing of transactional data to essentially score each offer (calculating the ‘tie’) for each member, individually. This was initially based on a banner propensity score — something we later updated post launch to consider category and product too.
When the member opened the app, we would pull through these offline generated offers and sort them on the fly in a more ‘online’ approach based on their value (the ‘take’) and newness (the ‘time’).
In this way, even though the offers typically lasted two weeks and were loaded daily across the banners, the feed for each member would change based on new and expired offers, how the offers were aging and their purchase behaviour.
When the offer feed was first requested by the app for a member or when it had been some time since the last request, then a new pull would be made and a new feed generated based on the mix of offline scoring and online personalisation. We then cached these results within the app itself so that subsequent calls would hit the cache first, increasing responsiveness and reducing overall system demands.
Another benefit of this caching was to support offline usage.
We knew that not all stores had WIFI or good data connections due to their location such as in a building basement, but we still wanted the app to be fully responsive even when offline. The use of a locally held offer cache within the app or WeChat Mini-Programme helped to ensure that for most customers, most of the time, being offline would be transparent to the customer experience.
The offer feed also needed to take account of user actions. We didn’t update the offer feed based on specific product purchases — so if you had an offer for toothpaste but had just bought it yesterday, you’d still see a toothpaste offer; that level of personalisation just wasn’t possible for launch. We did however want to ensure that where an offer had been used, either as a redemption in the app or as a single use offer in-store, that these offers would be removed from the feed to reduce confusion.
Leveraging offer usage behavioural data helped to ensure that the offer feed was also a real-time reflection of the member’s behaviour
With offers allocated individually to members offline and overnight, and these being personalised based on the member behaviours, we needed a solution for new members who had just installed the app. It also needed to be a solution that could scale because at launch, everyone was a new member and whilst we didn’t know it as this point, we were about to bring in over 1 million members in that first week alone.
This was our cold-start problem.
Given the complexity around our offer management and allocation, the cold-start situation was something we had to design into the overall solution to ensure that new members would see personalised offer content immediately and in a scalable way.
Whilst it wouldn’t be personalised in the sense that it used member behaviours, it was personalised to the extent that these offers were uniquely allocated to that member such that they could use them, save them, redeem them and scan them. It couldn’t be a generic set of offers — it had to be offers that belonged to the member.
To manage this cold-start we generated a list of offers for a member during the enrolment process such that a new member would be allocated offers as we were creating their account.
These offers would be limited to mass-offers only as there would be no segmentation or targeting at this point. The offers themselves were given a common baseline for the propensity score which meant that the online personalisation would kick-in to prioritise the offers based on just value and age. The offer ‘tie’ itself — the banner or category — would have no bearing at this stage.
This would then create the base set of offers to ensure the member had something to see following enrolment — which for most members was less than 30 seconds later!
At the same time, the enrolment process triggered a number of marketing automation journeys which would allocate a suite of enrolment offers that were unique to new members and would encourage initial trial of the programme and the banners.
Ideally these journeys would run in real-time such that the offers were visible when the enrolment journey was complete, but if not, they would show later that day once the cache was refreshed in the app.
Whilst these processes should ensure each new member had an allocated set of active offers and welcome offers, there was no guarantee that these processes would complete in time if processing loads were heavy.
New member offer allocation was essentially an asynchronous process by design as we didn’t want the enrolment itself to be waiting for activities to complete and risk the whole system grinding to a halt if demand was high — the first priority was always to get members enrolled and into the programme.
To mitigate this risk, we incorporated a fall-back design into the cold-start which meant we always had a generic set of mass offers maintained for ‘member zero’. This was a member account which would be kept up to date with the latest mass offers. If for any reason a call to retrieve a specific member’s offers failed to find any content, then the process would fall-back to retrieve offers for member zero.
Although it wouldn’t happen often, this approach ensured that we never had a situation post enrolment whereby a member didn’t have any content to view. The only restriction this placed on the member experience was that saving or redeeming those offers was restricted as they didn’t actually belong to that member. Instead, the member would need to perform an offer refresh to retrieve their own personalised offer set in order to have these capabilities enabled in the app.
Given we’d solved the challenge around offer display, personalisation and cold-start for new members, the final challenge was offer usage.
If we wanted to jump-start member engagement, then the offers themselves needed to be engaging — especially at launch!
In a sense, this was another ‘cold-start’ situation. We had to turn customers into members and highlight the benefits of the programme through actual experience with the offers and rewards.
It would be tempting at this stage to just go big, and utilise sales promotion offers with a view to drive maximum acquisition and trial. An offer like ‘download the app and get $50HKD to spend’ would buy behaviour, but it wouldn’t buy the right kind of behaviour.
Instead, we wanted the offers to onboard members and to lock in specific behaviours to maximise ongoing engagement.
Our initial focus was on earn activity as we wanted members to build points balances and to build these rapidly over the first few weeks. At a simple level this makes sense as it starts to show programme value to members as they can see their points growing.
There were however other reasons at play here too and it ties into how people actually behave within a loyalty programme.
We know that people like to continue with past behaviours — a tendency called the ‘commitment bias’ whereby people feel the need to follow through with something once they’ve invested time and/or money into it. From a loyalty programme perspective, if we can use high value initial earn offers to get people on-board and collecting, then it’s more likely they’ll continue with this behaviour as they’re now invested in it, even when overall earning rates start to normalise.
So these initial earn offers are not simply to drive balances, but also to drive commitment.
To support this, the initial offers around launch were mainly broad banner and category spend points multipliers and points bonuses. Offers such as “3x points at Wellcome on first spend “, “2x points on Beers, Wines and Spirits” or “200 bonus points on first spend at 7Eleven”.
These broad offers with deep value ensured that the majority of members could achieve them easily and that they got a large number of points on each purchase.
With the ability to stack offers too, it helped to ensure that new members would be over-rewarded to drive both programme trial and build up invested value that would lead to ongoing commitment.
We also put in place a new member journey which dropped high value offers over the first few weeks across popular and broad spend categories or banners. This meant that regardless of when the member joined, we’d be onboarding them with a great selection of habit-forming offers — personalised for them.
The second area we wanted to drive was redemption. It’s known that redemption drives increased activity both pre-redemption as people come closer to the point of actually redeeming and post-redemption, having achieved it and feeling good — something we term the ‘redemption lift’. The earlier we can get this behaviour into the programme the better.
It’s also interesting to note here that most people don’t use all of their points when redeeming. One study showed that, on average, members only redeemed around 26% of their points at any one time and only 3% redeemed 90% or more. This meant that as well as having achievable rewards available, we also needed to make sure that people had earned enough points to allow them to ‘safely’ use some of these to redeem whilst ensuring they had some held back in reserve.
There is a symbiotic relationship then between earn and burn and it was important that we had enough value flowing in initially to allow a little of it to then flow back out into redemptions.
To support this redemption behaviour, we made sure we had several achievable rewards from a points perspective as well as ‘flash reward’ offers to create more opportunities. We termed these ‘Hero Rewards’ because they were great value, heavily discounted and there to drive increased engagement.
A good example of this hero reward was the KFC Egg Tart which was available for just 400 points and achievable within just 2–3 average purchases. It was also a well-known and well-loved local item on the KFC menu so being able to get it for free through the reward programme was very attractive to many members.
The overall design of this offer solution then, with a blend of earn and burn offers across a combination of processes and planning helped us to really ‘jump-start’ our new members and ensure they landed well into the programme and that we locked in the behaviours we wanted to see.
It also meant that we gave away enough KFC Egg Tarts to reach every household in Hong Kong.