The Automation Center is a robust and flexible tool, and you can build programs in many different ways. However, the way you build a program path can affect its performance.
Here are some issues that you should always take into consideration before you start building programs, to ensure that their performance is optimal.
Please note that Emarsys is continuously monitoring the Automation Center and in certain circumstances we may put programs into fail-safe mode if we find that they pose a risk to the overall performance of your account. In such circumstances we will inform you and will help you fix the issues that caused the program to be stopped.
Batches are better
A good general rule is that batch processes are more efficient than single ones. There are many occasions where transactional entry points can just as easily be replaced by recurring batch ones; the trick is to consider how important the timing of a program action really is.
For example, if you want to create an Abandoned Cart program, instead of using a transactional program based on an event triggered by the cart abandonment, you can set up a recurring filter that checks daily for carts abandoned in the preceding 24 hours.
As a general rule:
- If your entry node is Target segment or Recurring filter:
AC will batch your users, and the batch is processed as a whole, resulting in significant performance improvements.
- If your entry node is New contact or Data change:
Even though these are transactional entry nodes, they can receive large batches of contacts at a time, so you should bear in mind your other registration methods or contact data update processes (see: Pay attention to imports, below).
- For all other entry nodes:
Processing is performed on each contact individually, resulting in a heavier load on the system.
The Automation Center can process batches of up to 250,000 contacts at a time without problems. Above this figure, you might run into performance issues and should consider splitting up your target segment or import into smaller chunks and only processing each one when the previous one has finished.
If you want to send different content to contacts depending on filter criteria, use the personalization options available in the email rather than creating separate emails on separate program paths. This will also make the email reporting more coherent.
If you use block targeting to personalize content, please remember that this relies on filters, which greatly increases the overall performance requirements of an email. If you use emails with block targeting, make sure to schedule them for different times in all your programs.
There are many cases where a Recurring filter entry node will have the same result as an External event node, but will perform far better. Ask yourself how critical it is that the email is triggered instantly, before you opt for a transactional email.
For example, you can trigger a shipping confirmation email using the External event node, but you can just as easily set a daily filter to return all contacts whose items have been shipped the previous day, and send a batch email.
If you set the Wait node to wait for 24 hours, each contact going through it will be processed individually. This means that for a busy program, this node could be running almost continuously, causing unnecessary load.
Where possible, set it to wait for a specific time, e.g. "wait until tomorrow 10 am", so the node can process all the contacts together.
If you have multiple Wait nodes in a program, try to set each of them to a different time of day to spread the load. If you want to ensure that all the messages following these nodes are sent at the same time, use additional Wait nodes to compensate. In the example below, the filter Didn't respond to email will run at different times, but both of the reminder emails will be sent at 09:00.
Pay attention to imports
Imports can inadvertently add contacts to programs that start with a New contact or Data change entry point, if they contain new registrations or a field defined in a Data change node.
If you import more than 250,000 contacts and you have programs with these entry nodes:
- On auto-import
- New contact
- Data change
…plus a Send email node, the program might cause performance issues and trigger a fail-safe.
If you wish to send more than 250,000 contacts through the Automation Center (for example: importing 1M contacts on new contact creation) then split the volume up to 250,000 chunks, and let each chunk be processed fully before starting the other.
Small is beautiful
It is tempting to create one huge, complex program to cover an entire marketing strategy, but this really brings no benefits. If you create a number of smaller, less complex programs instead you can more easily identify where your strategy can be optimized and make your changes without affecting the other programs. Contacts can easily be passed from one program to another.
A simple way to see if your program is getting too complex is to watch how long your browser takes to open it. The more assets it has, the longer this will take, and a delay of more than a few seconds is a sign that you should consider splitting it up.
One contact, one path
Contacts should only ever proceed along one path of a program. If you split paths, it is up to you to ensure that the filter logic excludes the possibility of a contact proceeding along more than one path.
If you rejoin paths after a split, we will not check for duplicates when the contacts reach the junction. It is therefore possible for a contact to appear twice on the same path, and receive the same messages twice.
Performance considerations for single user events
If the contacts are handled one by one you need to keep in mind that each type of node has a maximum throughput per program.
- Most nodes allow 500 contacts/minute/program to be processed.
- Response nodes and Action nodes such as Segment, Exclude segment, Filter switch and Quick filter allow 300 contacts/minute/program.
Please make sure to read Automation Center - Before You Start Building Programs as this explains how to keep the performance of your automated programs optimal.
Simple new contact program
If the new contact entry is triggered by an import, even a million contacts will get their emails almost instantly. On the other hand, if contacts are created one by one (for example through the API) then the throughput of this program is 500 contacts/minute.
External event with filter
Here the filter is the bottleneck, and the program has a throughput of 300 contacts/minute, regardless of the external event.
External event with multiple filters
Now since we have two filters, and they have to share the same computational resources, the throughput drops to 150 contacts/minute. This is the same if the filters are in sequence or in parallel.
If you set your Wait node to delay each contact for a certain number of hours, your throughput will be the standard 500 contacts/minute, and as long as the incoming contacts do not greatly exceed this, your program should perform as expected.
If you set your Wait node to hold contacts back a number of days, or wait for a specific day, you also have to define the time of day to process them. This means that your throughput will be 500 contacts/minute from that moment on.
For example, if your program collects 30,000 contacts a day, and your Wait node is set to ‘Wait one day and send at 08:00’, those 30,000 contacts will be processed at a rate of 500 per minute and it will take an hour to send all the messages.
And naturally, if your timer is followed by filters that will also affect sending speed.
Including exit criteria in your program can also affect performance. If you specify a segment as an exit criterion, that will essentially insert a segment node after every entry point and every timer. So when you have exit criteria defined you should only count on 300 contacts/minute to pass through the entry nodes and timer nodes.
If your program contains a campaign with a heavy section target, delivery can be delayed or even fail.
For example, an email used in 5 program variations, each with a 40M section target segment, killed the dedicated environment because the campaigns were scheduled at the same time.
- If the section target segment itself runs for more than 10 mins, you should be prepared for delays.
Avoid running campaigns at the same time if they have heavy section target. Consider instead replacing the section target with an exclude segment in AC.
There are some assets (forms, fields, messages, external events, etc.) that you can use in multiple live programs at the same time, and some that you can only use in one. These assets are checked for exclusivity only when the program using them is launched.
If you try to launch a program that uses an asset that is already selected in a node in another program, you will be informed of this and must de-select the asset in the nodes of the other program before you can launch the first program.
Here is an overview of the rules concerning exclusive assets.
- Forms can only be used in one active program at a time.
- Emails can only be used in one active program, ever.
- New contact can be used as entry point for only one active program at a time.
- Fields may only be referenced by one Data change entry point at a time. You may have many programs starting with this node, but each must reference a different field.
- External events may be used in as many active programs as you like.
- All channels other than email (e.g. SMS, Mobile Engage, CRM Ads, Web Channel) support using the same message in more than one active program.
Ending a program
When you end a program, the contacts still in it are allowed to continue through the program until they exit it. Therefore all the exclusive assets in the program remain exclusive and cannot be used in other programs.
Aborting a program
When you abort a program, all contacts in it are automatically discarded and exit the program. All exclusive assets (except email campaigns) will be free up and made available for selection in other programs.
Ignoring opt-in for certain transactional emails
As a rule, all emails sent via the Automation Center will respect the opt-in status of the recipient. However, an exception has been made for the following transactional entry points:
- External event
If you create a program that starts with one of these entry points, the first email in the program will be sent to all participating contacts regardless of their opt-in status (i.e. opt-in can be empty or FALSE).
This is to enable you to set up double opt-in programs, and critical transactional programs such as support requests, order and shipping confirmation, etc.
The Ignore opt-in feature does not work in Test mode. So, if you would like to test programs starting with the Form or External event transactional entry points, then you can send campaigns only to contacts whose opt-in status is
- These emails may not contain any marketing content that is not directly related to the transaction that triggered them.
- Only the first email in the path will ignore the opt-in. All others will behave as normal emails.
Turning Send Time Optimization (STO) on
You can only enable STO in a program if its state is In design. You cannot turn STO on or off once the program's state has been changed from In design to another one.
To enable or disable STO in an active program, you have to pause or end it, then copy it to create a new program. Now you can turn STO on or off in the new program.
Consider your account activity as a whole
Remember that an automated program must use the same account resources as your other segments and emails, as well as the other programs.
- Avoid scheduling Wait nodes for high traffic hours when you are sending large email campaigns.
- Avoid scheduling Wait nodes for the same fixed time across multiple programs.
- Avoid scheduling many Recurring filters for the same fixed time.
- If possible, consider breaking your complex program into 2-3 smaller programs (see: Connecting programs).
The settings you choose will have an important effect on the performance of your program.
- Contacts can enter the program anytime even if they are already in it is the least performance-heavy setting. You should use this whenever possible, and exclude the contacts you don't want to enter the program via a filter (based on whatever conversion criteria you are using).
- Contacts can enter this program only once ever has a high performance cost because every contact entering it must be checked to see if they have entered it before. You should use this only if it is absolutely necessary because for a program with a large number of contacts, referencing the program participation table could slow down any other programs using it.
In most cases batches are extremely fast. The only exception is when the advanced participation settings are used. These are:
- Contacts can enter this program immediately after exiting it
- Contacts can enter this program again X days and Y hours after exiting it
These settings mean that we have to keep track of each contact individually to know when they leave the program. For this reason, the Advanced participation settings have a high performance cost and should only be used when absolutely necessary.
Exit criteria place a segment node after the entry point and all Wait nodes. Depending on the complexity of your program, this could cause heavy load. In such cases you could consider using a filter switch before the channel nodes to exclude contacts.
The use of attachments in emails is not supported by the Automation Center. However, since they are supported by Triggered Email, you can send the attachment using that feature and then use the same external event to add the contact to a program to follow it up.
An issue to be aware of with all programs that spawn duplicate campaign names differentiated only by # numbers.
Campaigns with the same name can be found under analysis, because the naming convention creates a time stamp for campaigns without a second counter, and the system needs to differentiate with a hashtag and number. The resulting ambiguity can cause user confusion.
Solve this by placing a Wait node on one of the paths causing the naming differentiation.
- All filter nodes reference your entire contact database, even if they are filtering for a single contact, so use them efficiently.
- You should never run multiple filters sequentially; replace them with a single combined segment.
- Never place a filter immediately after a transactional entry node. Content should be personalized within the email where possible (see above).
- Avoid having the same segment run more than once at the same time. For example, if you select a segment for the Exit criteria, this will run after every Wait node in the program, so make sure these are set for different times of the day.
- A segment can only be run by one program at a time, so if the same segment is referenced by multiple programs at the same time, the segment will be queued and processed program by program. This also applies to Exit criteria. You can get around this by making copies of the segment and referencing a different copy in each program.
- Avoid running many Segment nodes in parallel, since all contacts must be processed by each one. Use a cascading set of Filter switches instead.
Parallel filters - bad:
Cascading filter switches - good:
Using the same segment in multiple programs
Each individual segment can only be active in one program at any one time. If you have several programs with similar design, using exactly the same segment simultaneously, only one segment instance will be processed and the others will be queued. The same applies to exit conditions as well; if multiple programs are running with the same exit condition they can block each other.
There are several options to avoid segment runs being queued:
- If your use case allows, launch the several similar programs at different times; the segments will be accessed at staggered intervals.
- Put a short waiting node before the segments, with a different time in each.
- Make as many copies of the segment you have programs, so that they will be running different segments but with the same segment logic.
Since the program reporting shows the number of contacts returned by the filter nodes, you may be tempted to add additional filters at the end of paths just to see the number of contacts that end up there.
Not only does this cause unnecessary load on the program, but each filter reflects only one specific outcome and should not be relied upon as an accurate measure of your marketing strategy.
You will get much more accurate and relevant reporting when you tie your customer engagement programs to your Smart Insight customer lifecycle reports.
For more information, see Automation Center - Program Reporting.