Should I use STO for delivering time-sensitive emails?
In most cases, no. For time-sensitive emails, such as limited special offers or discounts, you should consider the duration of the offer before using this feature. If an offer is only available for a 24-hour period, for example, you will not be giving all your customers the same advantage if STO is used.
Can I exclude specific hours in order to avoid contacts receiving emails at those times?
Currently, no. Our algorithm analyzes contact events in order to determine the best possible send times. Excluding hours manually would decrease the efficiency and the performance of the algorithm.
Can I manually define the period over which the campaign is staggered?
No. Scheduled campaigns are launched from 00:00 through to 22:00 on the calendar day of the launch (in the timezone of the account), and manually launched campaigns are staggered over 24 hours from the time the calculations are finished. No other control over the timescale is possible.
The technology behind STO
Is the algorithm proactive, or merely reactive?
Both. We cannot know for sure which is the best hour for any contact, so it is important to experiment with send times we have minimal or no information about. On the other hand, we obviously want to exploit the knowledge we already have and send at times which perform well with high probability. The algorithm finds the best trade-off between exploration and exploitation. The more historical data you have the better results you can expect as STO needs less exploration and can do more exploitation.
How does STO determine the send time for a new contact with no past events?
If you are an existing customer, we use your account’s aggregated historical data on open times to assign the most likely send time to your new contacts. If you are a new customer with no historical data, new contacts are randomly distributed among the launches if STO is turned on. That is why it is highly recommended to wait at least a month before activating STO.
What happens if a contact moves to a different time zone?
The algorithm automatically adapts to consumer preference changes: for example, if the client moves to a country in a different timezone or their daily routine changes, the algorithm will gradually adjust the sending time as we receive this feedback.
How can the algorithm learn the optimal send time if both 6:00 AM and 8:00 AM are good for my client but all the other slots are bad?
If there are two, or more, similarly performing send times the algorithm will send to these with similar frequency; it is not a goal to pick one of them. This ensures that we have the possibility to find out if one of them is actually better in the long run, even if both produce very similar results for a period.
What if I switch STO off for a while, will it continue to learn?
Partially, yes. It still collects response data on an account level, so the underlying statistical model will be better when you next switch it on, benefiting future contacts.
How do you define a timeslot as successful for a contact?
We count any open as a success, no matter how long after the launch it comes, so a contact will continue to receive emails at that time, as long as they continue to respond. No open response for a campaign will cause the algorithm to reevaluate the time for that contact, based on its current statistical model.
How much time does it take for STO to deliver results?
It usually needs at least 10-20 launches to see the first stable improvements, until then, results may fluctuate, even deteriorate, so be patient. You will see the results in time.
How long does an STO campaign take to start after manual launch?
The calculations begin as soon as Launch now is clicked, and control group email will be sent as soon as they are finished, that is, within a maximum of two hours. The first real STO launch will happen on the following even hour.
Therefore a delay of up to three hours after clicking the Launch now button is, in theory, possible.
How often is the send time database updated?
Each contact’s profile is updated every 2 days with the latest engagement information in order that we have the most recent data at our disposal.
What kind of algorithm is used?
STO applies a method called a multi-armed Bayesian bandit algorithm, if you must know.
The effects of STO
How do you calculate the added value of STO?
For any given week, we compute the aggregated open rate for both STO and CONTROL launches, then calculate the difference as a percentage of the CONTROL open rate: (STO open rate - CONTROL open rate) / CONTROL open rate as %.
- If the two groups performed identically, e.g. both have a 15% open rate, then the added value of STO will be 0%.
- If the control group launches have a 15% open rate and the STO launches 16.5%, then the added value of STO will be +10%, since the +1.5% difference is 10% of control group's 15% open rate.
Does STO have any side effect on unsubscribe rates?
No. We have found no significant difference during the pilot phase in unsubscribe rates between STO and non-STO campaigns.
How does the Frequency Cap work with STO campaigns?
If the Frequency Cap is active, it will apply to each launch at the moment of launch. For example, if an STO campaign is scheduled for a day when contacts whose Frequency Cap is set to 1 have already received a message, the STO child campaigns launched on that calendar day will not be sent to them. But they will receive the launches sent on the next calendar day.
Is it possible that some of the 12 staggered launches will include no emails at all due to STO?
Yes. In such cases the given launch will not be created at all, which means that there will be fewer than 12 launches in the campaign. But if you have a large contact list containing thousands of email addresses, and especially if these are geographically dispersed, this is highly unlikely to happen. You can always check the statistics breakdown for each STO campaign by going to Analysis > Emails and clicking the plus sign to see the results of the individual launches.
About the control group
How are the members of the control group selected?
Completely randomly. For each campaign, the members of the control group are picked at random so when we compare the results the only deliberate difference between the two groups is STO.
Why are 10% of the recipients selected?
We need a large enough control group in order to reliably measure results. On the other hand, we do not want to adversely impact your engagement results. We think that 10% is a good trade-off.
Can I turn the control group off?
No. The control group is not optional. It is the basis of all monitoring and reporting, and it also saves you the time and effort you would need to invest in A/B testing.