Predictive Studio - Score Definition

Creating a Predictive Score definition

Blueshift’s Predictive Studio allows you to define and customize a Predictive Score based on your Marketing objective. The underlying AI model learns a user’s likelihood (Predictive Score) to convert on:

  • The specific channel such as email, push, SMS, or in-app based on past behavior
  • The specific events in the future, based on past behaviors

 

Input for a channel engagement model

To create a model that predicts engagement on an event, click + Predictive Model on the index page of the Predictive Studio and then choose one of the following:

  • Email
  • Push
  • SMS
  • In-App

 

create_email_engagement_score.png

Next, specify what you would like for the email score:

model_params_channel.png

Goal Actions: Actions for which you’d like to create a Predictive Score. If a user performs the action on the message that we send, we count it as conversion. For example, you can add open and click as actions that should be counted as conversions for the email channel. At the moment, here are the actions that you can choose (based on the channel you select to create a score):

  • Email: Open, Click
  • Push: Click
  • SMS: Click
  • In-App: Open, Click

 

Goal Look-Forward: Predictions will be made and remain valid for this time period. For example, predict channel engagements in the next 7 days.

The default is 7 days. Shorter windows(a day or few days) could lead to more accurate predictions, while longer windows could introduce noise/irregularities, which leads to lower predictive accuracy.

Consider setting this value based on the historical conversion rate. For example, n number of conversions in the prediction window/number of active users in lookback days * 100 is at least 1%.

 

Activity Look Back: Audience for the Predictive Scores will include those users who had at least one action on the channel in the lookback period. The default value is 30 days (monthly active users).

Consider setting this value based on the historical conversion rate. For example, n number of conversions in the prediction window/number of active users in lookback days * 100) is at least 1%.

After you create the model, our platform generates a summary and visualization.  It is empty in the beginning since we refresh the model daily. So we recommend that you check back later. Once it's ready, you can take a look at it and here's how it looks:

email_score_model_summary.png

For more information, see Model Visualization. 

 

Input for an event engagement model

To create a model that predicts engagement on an event, click + Predictive Model on the index page of the Predictive Studio and then choose Event.

event_engagement_model_create.png

 

Next, specify what you would like for the event score:

event_model_parameters.png

Goal Events-  Events for which that you’d like to create a Predictive Score. Presence of at least one of these events will be counted as a conversion. For example:  purchase, subscription_renewal, etc. You can choose any combination of events that you are tracking with Blueshift.  You can read more on events here.

 

Goal Look-Forward - Predictions will be made and remain valid for this time period. For example: predict purchase intent in the next 7 days or subscription renewal in the next 30 days.

The default is 7 days. Shorter windows (a day or few days) could lead to more accurate predictions, while longer windows could introduce noise/irregularities, which leads to lower predictive accuracy.

Consider setting this value based on the historical conversion rate. For example, n number of conversions in the prediction window/number of active users in lookback days * 100 is at least 1%.

 

Activity Look-Back - Audience for the Predictive Scores will include those users who had at least one impression(event or campaign action) in the lookback period.  The default value is 30 days (monthly active users). This input should be equal to the length of the conversion funnel.

Consider setting this value based on the historical conversion rate. For example, n number of conversions in the prediction window/number of active users in lookback days * 100 is at least 1%.

In most use cases, the conversion is heavily correlated with the recent engagement, so 30 days(monthly) or 90 days (quarterly) usually suffices.

E-commerce funnels are usually short, so 7 to 14 days work for most e-commerce purchase intent use cases. Subscription businesses can set this input based on their billing cycles to predict subscription churns. You can add funnel events using the Advanced Options button.

 

[Optional] User attributes - You can choose to include user demographics (i.e. age, location, etc), preferences(attributes of interest from the catalog), memberships (subscription type) and lifetime attributes (lifetime orders, revenues, etc). We use the latest value of the user attributes.  Warning: Be careful and don’t pick a user attribute which is updated as a result of the conversion.  Otherwise this would cause leakage(overfitting) and the model wouldn’t generalize well on unseen data. An example of one such user attribute is “total_orders” for predicting purchase conversions. If we increment total_orders after conversion, for one-time purchasers, the model could confidently learn a simple rule that users who convert must have total_orders>0, which is not what we want.

Click here for more details on user attributes. See this article for more details on Blueshift’s user attributes. You can see the summary on the right side. 

model_summary.png

After you create the model, our platform generates a summary and visualization.  It is empty in the beginning since we refresh the model daily. So we recommend that you check back later. Once it's ready, you can take a look at it and here's how it looks:

event_engagement_score.png

 For more information, see Model Visualization.

 

Data requirements

The most important thing while setting up a Predictive Score is ensuring that the historical conversion rate, ( i.e number of conversions in prediction window/size of the modeled audience * 100), is at least 1% and at least 1,000+ users have converted in the past prediction window. The AI model learns to make predictions based on historical conversions. Too few conversions or very low conversion rates may lead to low performance or stats resulting in an insignificant model. 

If data requirements are not met, Predictive Scores will fail and there will be an alert on the studio dashboard. Consider tuning the prediction window, goal lookback days, user attribute filters to make it work.

Once data requirements are met, an AI model will be created. AI Model summary details are talked about in the Model Viz section.

Output

A Predictive Score between 0-100 is generated for every user in the scoring audience. The Predictive Scores are available on customer profiles, which can be used for lookups, segmentation, syndications and exports.

FAQs

Can I choose a goal based on an event attribute, like product sku?

Yes, the studio doesn’t support this option yet, but we can customize the goal inputs on the backend. Currently, we support goal event filters on product attributes(i.e category, brand, sku) if product sku is available on the event.

Can I filter my audience based on event counts and event attributes?

Yes, the studio doesn’t support this option yet, but we can customize filters on the backend. Currently, we support event filters based on event count(e.g number of purchase > 0), event recency(e.g last purchase < 7 days) and product attributes(e.g category, brand, sku) if product sku is available on the event.

Was this article helpful?
0 out of 0 found this helpful