Here you can find all the information you need to set up and work with the Emarsys for Magento 2 integration. Some steps are only relevant to users of Smart Insight or Predict and are clearly marked.
- Installing the integration
- Contact data
- Product data
- Order data
- Configuring your transactional messages
- Testing the integration
- Maintaining the integration
1. Magento 2 requirements
From Magento you will need:
- A Magento store of version 2.1.x or higher.
- PHP versions 5.6.0 and 7.0.17.
- php5-curl must be installed on your Magento server.
- PHP SOAP extension must be enabled.
- TCP port 21, 32000-35000 must be open (these are required for file transfer over FTPES).
- Set your Magento crons to run every 1 minute.
- Make sure your server’s PHP memory limit is set to a minimum of 512 MB. This is because some extension jobs are large and require increased PHP memory.
- AOE Scheduler (or similar) is not mandatory but highly recommended as this exposes the scheduled tasks visually and will help us to support you.
2. Emarsys requirements
From Emarsys you will need:
- A fully set up and working Emarsys Marketing Platform account.
If you have more than one Magento site, you can map them to the same Emarsys account or request a separate account for each one.
Multiple Magento stores within the same site can only be mapped to the same Emarsys account.
- Your Emarsys merchant ID.
- Your Emarsys WebDAV folder and credentials (for the initial contact data upload and bulk exports).
- Your Emarsys API credentials (for ongoing contact data synchronization).
- Your Emarsys SFTP folder and credentials (Smart Insight customers only).
All of the above should already have been set up as part of your standard Emarsys onboarding. If you are missing any of them, please contact Emarsys Support.
- The installation package (available from https://github.com/emartech/3rd-party-integrations-magento-M2, or as a .zip file from Emarsys Support).
In addition to these, you will need to create the following in your Emarsys account:
- An external event and mail stream for each one of your mapped Magento events (see below).
- All the custom data fields that you need to map to Magento Customer fields.
- Custom fields of the type Short Text field (max. 60 characters) with the following names:
- Magento Customer ID
- Magento Customer Unique ID
- Magento Subscriber ID
You can create these custom fields in Emarsys, or wait until the installation is complete and then create them from the Installation Checklist page in Magento. Note that the names are case-sensitive. For example, Magento customer id will not be accepted by the integration.
- Whitelist the IP address 184.108.40.206 and the range 220.127.116.11/24 (this are required for opt-in synchronization back from Emarsys to Magento).
Installing the integration
1. Install the Emarsys module
In order to install the Emarsys module in Magento 2, proceed as follows:
- in Github (or in the unzipped installation package), navigate to emarsys/app/code/Emarsys and then copy the Emarsys folder into your app/code directory in Magento.
You may have to create the appropriate subfolder on your M2 server, and at the end the path may look like this: /var/www/html/magento2/app/code/Emarsys/Emarsys.
At this stage you should also make sure that all the folders inside the Magento project root are set to be able to write, and they must be owned by both the Magento user and the apache2 usergroup, for example: magento:www-data.
If they are not owned by the apache2 usergroup, you will encounter errors.
After installation, you should re-check your folder permissions, as often some new folders or new files are written as the Magento primary user-group rather than the apache2 www-data group, resulting in errors. If so, you can manually fix this by assigning these to the correct group again. Make sure to pay close attention to the error-logs if this happens.
2. Check the installation
Once you have run the commands as described above you can check the status of your Magento system before you proceed. You can do this via the new Emarsys menu available in your Magento backend.
Go to the Emarsys menu, Operations, Installation Checklist. Here you can see the status of your installation.
- System Requirements - Checks that the Magento prerequisites have been met.
- Extension Settings - At first these will show the default values set by the installation package. You will configure those to your own specifications later in the installation workflow.
- Mandatory Emarsys Customer Fields - Shows Yes if the fields have been found and mapped by the extension during the installation. If not, you will be able to create them in Emarsys directly from the checklist.
3. Enable web behavior data collection
For Predict users, these scripts are the basis for the product recommendations.
For Smart Insight users, they provide the revenue and attribution data.
For all Emarsys users, they provide the values for the standard set of behavior fields.
To enable these scripts, go to the Emarsys menu, Settings, Web Extend.
- Merchant ID - Your Emarsys merchant ID.
4. Enable product recommendations
If you are a Predict customer, you can also define which widgets will be used to display the product recommendations on your various web pages.
To do this, go to the Emarsys menu, Settings, Predict.
For every page type, select the widget you want installed there.
For more information, see:
You will need to ensure that your Magento contacts (customers and subscribers) are correctly synchronized with your Emarsys contact database, and that you have the right fields mapped for your segmentation requirements. This is done in three steps.
Preparing your contact data
The initial contact data load is performed via your Emarsys WebDAV folder. After that, contact data is kept synchronized via the API, with the WebDAV used for bulk exports and as a failsafe backup.
1. Enable the integration and configure the API settings
In the Magento backend, go to the Emarsys menu, Settings, Connection to bring up the Emarsys Settings screen.
- Select Yes in the Enabled field to enable the integration.
- Enter your Emarsys API credentials.
- Click Test Connection to verify everything is working. Magento will display a message to confirm whether it is working or not.
- After testing, you may have to re-enter your credentials.
- Click Save Config when you are done.
2. Configure the WebDAV connection
In the same screen, configure your WebDAV settings.
- Enter your WebDAV credentials.
- Click Test Connection to verify everything is working. If the connection is working, the button turns green, otherwise it will turn red.
- After testing, you may have to re-enter your credentials.
- Click Save Config again when you are done.
3. Customer Mapping
Magento 2.0 no longer requires both customers and subscribers to have their data fields mapped separately. Instead, you first map all the contact data fields and then you map the values for the single- and multi-choice fields. The mapping that you define here will be used for both the initial data load and all subsequent synchronization, for both types of contact. Proceed as follows:
- First, go to the Emarsys menu and select Mappings, Customer.
- On the Emarsys - Customer Mapping page open the Select Mapping drop-down and select Customer.
- Click Update Schema to load the Emarsys fields to the Emarsys Customer Attributes column.
- In the Magento Customer Attribute table, select the appropriate Emarsys field for each entry.
Make sure that you map Emarsys fields of the same type. If the field type is not appropriate for the data, you will run into problems when creating segments in Emarsys based on these fields.
- When you are done, save your changes and click Update Schema to display the new mapping.
- Select Customer-Field from the Select Mapping drop-down.
- All the single-and multi-choice fields that have been mapped are shown and you must now map the values for each one.
In both mappings, you can save time by using the Recommended Mapping button. This will try to identify and associate matches based on field name similarity. If you use this, you should still verify every single mapping to make sure that they are correctly associated.
If you add new fields in Emarsys, you can make these available in Magento by clicking the Update Schema.
4. Prepare the Emarsys auto-import
Once you have mapped all your fields, you will need to set up two auto-imports in your Emarsys account, one for Magento customers and one for subscribers. You must set this up as follows:
Source File Settings (the first page of the wizard)
- Where to look for the import file - Select Emarsys WebDAV and enter the path.
- Import file settings / File name - The Magento initial load files will be named customers_<timestamp>_default.csv and subscribers_<timestamp>_default.csv, so you should enter customers_*_default.csv and subscribers_*_default.csv for your two auto-import profiles.
Sample File (second page)
- You will need to upload a sample file containing the header and at least one line of the file you will use for your initial load, with all your mapped fields. You may have this from your staging system, otherwise you can make an initial export to the WebDAV and use that file.
Match Fields (third page)
- Check that the fields are correctly mapped and select your unique identifier.
When you have finished, make sure that the status is set to Enabled on the Auto-imports on the overview page!
The same auto-import will also be used for the contact synchronization failsafe mechanism (see below). After the initial load has been performed, you may want to return to your Emarsys auto-import and change the name to differentiate later imports from the initial one.
The initial contact data load
Once you have prepared your customer data as described above, you can perform the initial data load to import all your Magento customers and subscribers to Emarsys.
1. Define the opt-in settings
Before you perform the initial customer data load, you must define how the opt-in settings will be defined in Emarsys. Proceed as follows:
- Go to the Emarsys menu, Settings, Contact Synchronization.
- In the Initial DB Load section and select the opt-in status to apply, choosing from:
- Set opt-in status for all users to true - Enables Emarsys to send to all imported contacts straight away.
- Set opt-in status for all users to empty - You cannot send to any contacts from Emarsys until their opt-in has been confirmed by another method (i.e. double opt-in).
- Set opt-in status true for all users depending on attribute - The opt-in value is contained for each contact in a mapped database field.
Once the initial data load is complete and the integration is synchronizing the data across both systems regularly, opt-in status is handled on a per-contact basis as they sign up, opt-in or opt out of content.
This is the only data that is synchronized both ways, with the latest timestamp taking precedence.
2. Perform the initial data load
When you have set your opt-in status, and click Export on the Initial DB Load page to trigger the initial data load.
Do not save any Contact Synchronization Settings before you make the initial upload, as this will immediately begin to synchronize data according to your schedule, and you may end up overwriting data when the initial load is performed.
Synchronizing contact data
Once the initial data load has been performed, you will need to define the settings for the ongoing synchronization of new data.
If you make any changes to the field mapping, the new fields will be synchronized for new contacts only. To update the new fields for existing contacts, you will have to run another bulk export (see Maintaining the integration below).
To configure your synchronization settings, go the Emarsys menu, Settings, Contact Synchronization.
There are two methods for keeping contact data synchronized, Realtime-failsafe and Background.
- Realtime-failsafe (recommended)
This maintains contact synchronization in real time, sending every update to Emarsys as it occurs in Magento. This option is suitable for use cases where Emarsys needs the data immediately (e.g. for automated engagement programs). In case of unsuccessful updates (e.g. due to network downtime or excessive volume), they are placed in a failsafe queue and are processed once a day at the specified Background Runtime (see below).
To enable this option, set Realtime-failsafe to Yes.
- Background only
This collects all contact updates that have occurred over 24 hours and synchronizes them together at the specified Background Runtime. This option is suitable when there is no urgency for the data to be transferred to Emarsys, and there is no limit to the volume of data being synchronized.
To enable this option, set Realtime-failsafe to No.
- Background Runtime
Here you enter the time of day (in HH:MM:SS) when your failsafe export will run (if you are using the Realtime-failsafe option), or when your regular Background export.
- Background Frequency
The options are Daily and Hourly. If you select hourly, the sync will run at the minutes and seconds past each hour as defined above.
- Unique Field
This is the unique identifier that will ensure the data is synchronized to the right contact in Emarsys. You can choose from the email address, the Magento ID or a combination of email address, website ID and store ID.
Email#WebsiteID#StoreId is our recommended option, but bear in mind that if you have multiple stores, this will create separate Emarsys contacts per store, even if it is the same person making the purchase.
Once you have set the Unique Field and synchronized your first contact, this setting cannot be changed.
- Notification secret key
Here you can enter any password or secret you like. It is purely an internal security code to protect your data files from external interference.
What Data is synchronized?
- From Magento to Emarsys
The Magento contact database is considered the master database for contact details, and all values synchronized for the mapped fields will overwrite the existing values in the Emarsys application.
New contacts will be exported to Emarsys with the same fields as per the initial export.
- From Emarsys to Magento
Once a day, at the specified Background runtime, Emarsys also checks if the Opt-in field of any Magento contacts has been modified. If so, this change is synchronized back to Magento.
Whenever the opt-in field is synchronized, the integration checks the last modified timestamp in both systems and the latest timestamp always takes priority. This is to prevent an unsubscribe action in Magento being overwritten by the former value in Emarsys.
If you are using Smart Insight and/or Predict, you will need to ensure that your Magento product catalog is correctly loaded to Emarsys and kept synchronized.
Unlike contact and order data, the entire product catalog is always exported rather than simply the delta. Therefore, the initial load and ongoing synchronization are actually the same process.
Preparing your product data
1. Configure the FTP connection
Your product and order data is synchronized with Emarsys via an FTP server. To configure this connection, go to the Emarsys menu, Connection to open the FTP Settings page.
- If you are a Smart Insight customer, enter the Hostname, Port, Username and Password as provided by Emarsys Support.
- If you are a Predict customer who does not use Smart Insight, you can enter your own server details here instead.
- Bulk Export Directory - If you have created a sub-directory on the server to separate the Bulk (manual) exports from the regular scheduled exports, enter that folder name here (preceded by /).
- Use FTP over SSL(FTPS) - Set to Yes (recommended).
- Use passive mode - Set to Yes or No as per your IT requirements.
- Click Test Connection to verify everything is working. Magento will display a message to confirm whether it is working or not.
- After testing, you may have to re-enter your credentials.
- Click Save Config when you are done.
2. Product Mapping
First, go to the Emarsys menu, Mappings, Product. Here you map your Magento product attributes fields to the Emarsys product catalog fields, ensuring that your product catalog has the correct columns and field types to conform to our product catalog standards.
If you want to include custom fields, you must create them in Emarsys using the c_* naming convention. For more information, please see:
Proceed as follows:
- Click Update Schema to make sure that all your custom Emarsys fields are available in the Emarsys Attributes column.
- Map each appropriate Magento attribute to an Emarsys field.
- If you leave a field as Please Select it will be ignored and not loaded to Emarsys.
You must map at least the mandatory fields:
3. Configure the Feed Export
When you have mapped your fields, go to the Emarsys menu, Settings, Predict to configure the catalog export.
- Enable nightly product feed export - Set this to Yes. It does not matter if a scheduled export occurs before you have made your initial load as the entire catalog is always overwritten every time.
- Include Bundle Product - Defines whether bundles are included or not, see below.
- Exclude Categories - Here you can exclude entire categories or sub-categories of products from the export if you so wish (for instance, if they are irrelevant to your Emarsys marketing activities).
- Execute and Export at - Sets the time of day for the catalog export ( in HH:MM:SS).
- Frequency - Set to Hourly or Daily as you wish.
When you click Save Config the product catalog will be exported from Magento to Emarsys according to your defined schedule.
If you set the field Include Bundle Product to No, your export file will contain only the product data:
If you set the field to Yes, then the Bundles will be included as an extra line in addition to the product data:
The initial product catalog load
Once you have configured your catalog fields, go to the Emarsys menu, Operations, Bulk Export to make the first manual export of your catalog.
- Select Product as Export Entity Type.
- Define whether to Include Bundle Products or not.
- Exclude any categories you want (make sure that these match any categories excluded in the Feed Export settings above.
- Click Export.
The entire product catalog will then be exported to Emarsys.
Synchronizing your product catalog
As long as you have enabled your catalog Feed Export as described above, Magento will push your entire product catalog to Emarsys according to the schedule you have defined.
With every export the entire catalog will be overwritten with the new one.
If you are using Smart Insight, you will need to ensure that your Magento order (sales) data is correctly loaded to the Emarsys Smart Insight database and that you have the right attributes mapped for your Smart Insight segmentation requirements.
As with contact data, this is done in the same three steps.
Preparing your order data
First, go to the Emarsys menu, Mappings, Order. Here you give the Magento order columns the names that will appear in the Smart Insight screens as Purchase attributes.
There is a fixed default set of columns already mapped - these are named in the Emarsys Order Attribute column and cannot be changed.
You can add any other columns to the export by entering an attribute name. If you leave this field empty, that column will not be included in the export file.
If you have not already configured your Emarsys SFTP connection as described above, you should do so now.
The initial order data load
When you have named all the columns in the export file, you should make a bulk export of your historical order data. Proceed as follows:
- Go to the Emarsys menu, Operations, Bulk Export.
- Export Entity Type - Select Order.
- From and To fields - Select the time range for the order data.
- Export To - Select CSV.
There is no limit to the data you can export, but we recommend to include at least the last two years if you can.
- Click Export. The order data will now be loaded to the Smart Insight database and will be available for Smart Insight segmentation in Emarsys.
Do not save any Smart Insight Settings before you make the initial upload, as this will immediately begin to synchronize data according to your schedule, and you may end up with some order data being loaded twice when the initial load is performed.
Synchronizing order data
This step is required by Emarsys customers who are using Smart Insight.
After your initial order data load, you must configure the ongoing load of new order data. Proceed as follows:
Go to the Emarsys menu, Settings, Smart Insight.
- Smart Insight Enabled - Set this to Yes.
- Export using email as identifier - If set to Yes, the email address of the Magento contact will be added to the export file and used to map the order data to an Emarsys contact, instead of the Magento Customer ID. This is important if you are including Magento Subscribers in your marketing campaigns, as you do not have a Magento Customer ID for them.
- Export guest checkout orders - Defines whether or not guest (i.e. anonymous) orders will be exported. These cannot be used for Smart Insight segmentation but they will help to improve the product affinity model.
- Frequency - Hourly or Daily. We recommend to set this to hourly as this will keep the export files smaller.
- Order Export for Statuses - This lists all the status values that an order can have. Orders with a selected status will be exported. Multiple selection is possible.
When you click Save Config, Magento will export all the orders which have transitioned to one of the selected statuses since the last export, at the appointed time.
Configuring your transactional messages
Magento 2 has a number of customer-facing events, some of which can be used as triggers for transactional campaigns in Emarsys. You will need to map these events to the external events in Emarsys which will trigger the campaign. You can also map Magento variables to placeholders in Emarsys and personalizes your message content. Proceed as follows:
1. Prepare your Emarsys campaigns
For every Magento event you want to map, you will need to create the following in your Emarsys account:
- One external event per Magento event.
- One campaign per external event. These should be created directly in the campaign creation interface for that channel (Triggered Email, SMS or Mobile Engage) rather than using an Automation Center program (programs should only be used if you intend to follow up the initial message with further actions, which is typically not the case for transactional messages).
- A list of personalization placeholders that you can map to the Magento variables for each event (see below).
Personalizing your message content
Since your Magento contacts are synchronized with Emarsys, you can of course include in your message content all the regular personalization options available in Emarsys, including conditional content and block targeting. However, there may be important information about the event that is only available in Magento (for example, the URL of the confirmation link in a confirmation email).
To add this Magento content to your Emarsys message, you must add special Magento placeholders to the content. These are written with a double % either side (for example, the variable first_name should be written in the message content as %%first_name%%).
You will need to map these for each event so you should carefully track which placeholder you have included in which transactional message.
These placeholders can be any text and are only used to make sure the right values in the JSON object are added to the right place in the email. They do not reference the Emarsys database and do not have to be the string IDs of Emarsys fields. We recommend that you name them as closely as possible to the Magento variables you will match them to.
2. Define your opt-in strategy
This integration allows you to use Emarsys to trigger confirmation messages for newsletter subscriptions. To do this, go to the Emarsys menu, Settings, Opt-in.
- Enable Opt-in = No - There are no automatic triggers from Magento to Emarsys (you can manage the confirmation from Magento, and there are still options to trigger emails from within Emarsys based on the change in value of a database field).
Enable Opt-in = Yes - Magento can automatically trigger transactional messages from Emarsys confirming the opt-in for the following subscription points:
- Customer Registration Page - Triggered by activating the checkbox during the registration process.
- Checkout process - Triggered by activating the checkbox during the purchase process (a code snippet is provided for this checkbox).
- Newsletter Box on every page - Triggered by entering and email address in the subscription field in the footer of every page.
If you do enable this, you will then need to define which opt-in strategy to use for each subscription method.
It is strongly recommended to manage the opt-in status as far as possible in Emarsys. If you change the status in Magento, it could take up to 24 hours to reach Emarsys in the case of a Background or failsafe sync, during which time the contact could possibly receive further emails.
2.1. Single opt-in
With this strategy, a subscription event in Magento will create or update a contact in Emarsys with their opt-in status set to
TRUE, and you can optionally map an Emarsys event to trigger a confirmation message.
Since anyone could, in theory, use any email address to subscribe on a public website, we do not recommend this method as you cannot guarantee that the person who owns the email address is the same as the one who subscribed.
2.2. Double opt-in
This is our recommended strategy as it ensures that the person who subscribes is the same as the person who owns the email address used.
The initial subscription will create or update a contact in Emarsys, but their opt-in status will be
FALSE until they click the confirmation link.
You can then select up to two Emarsys events:
- External Event id for Double-Opt-in Event - Mandatory; this message will be triggered after the initial subscription and will contain the confirmation link.
- External Event id After Opt-in confirmation - Optional; this message will be triggered after the confirmation link has been clicked.
3. Map the events
When you have prepared your campaigns in Emarsys, go to the Emarsys menu, Mappings, Events.
- For every available Magento event, select the appropriate external event in Emarsys.
- If you create a new external event in Emarsys, click Update Scheme to make that available in Magento.
Available Magento events
The following Magento events are currently available for mapping:
- Contact Form
- Credit Memo Update
- Credit Memo Update for Guest
- Forgot Admin Password
- Forgot Password
- Gift Registry Sharing
- Gift Registry Update
- Invoice Update
- Invoice Update for Guest
- New Credit Memo
- New Credit Memo for Guest
- New Invoice
- New Invoice for Guest
- New Order
- New Order for Guest
- New RMA
- New RMA for Guest
- New Shipment
- New Shipment for Guest
- New Account
- New Account Confirmation Key
- New Account Confirmed
- Newsletter Subscription Confirmation
- Newsletter Subscription Success
- Newsletter Unsubscribe Success
- Order Update
- Order Update For Guest
- RMA Admin Comments
- RMA Admin Comments for Guest
- RMA Authorization
- RMA Authorization for Guest
- RMA Customer Comments
- Remind Password
- Rewards Points Balance Update
- Rewards Points Expiry Warning
- Send Product to a Friend
- Share Wish List
- Shipment Update
- Shipment Update for Guest
- Store Credit Update
4. Map the placeholders
When you have mapped your events, you can map Magento variables with the placeholders you added to your Emarsys campaign content. For each event, click Placeholders.
Now write the Emarsys placeholder, without the double %, in the field next to each Magento variable you want to include in the message content.
5. Enable transactional emails
When you have finished your mapping, go to the Emarsys menu, Settings, Transaction Email and enable this functionality.
- Enable Emarsys Transaction Emails - Set this to Yes.
From this point on, all the mapped events in Magento will trigger the corresponding campaign in Emarsys. This is not restricted to email - you can also map events to SMS or Mobile Engage campaigns.
6. Check the JSON
It is a good idea to check the JSON payload that will be transmitted to Emarsys, to make sure all the placeholders have been correctly mapped. To do this, click JSON Request next to each event.
This will show only the Magento JSON request that is sent to Emarsys, and will not include any content or personalization that you have included in the Emarsys message content.
Testing the integration
To test that the contact data synchronization is working, proceed as follows:
- Create a new dummy account in your Magento store.
- Log in to the shop with the dummy account.
- Make a test order.
- Update the user profile.
- Monitor the Emarsys database to track the data changes.
To test that the e-commerce data synchronization is working, proceed as follows:
- Add a new product to your Magento store.
- Wait for the product data to be synchronized (as per your configuration).
- Query the data in Emarsys to see that the new product has been imported correctly.
Maintaining the integration
Data monitoring in Emarsys
Once you have made your initial data loads and set up the schedule for regular synchronization of new data, you can take advantage of the data monitoring tools that Emarsys provides to make sure that the data flow is working properly.
These tools validate every data import into Emarsys. If an issue is identified, this is highlighted and a suggested solution offered.
The tools are located in your Emarsys application on the Data Sources page.
Keeping your Magento schema updated
If you create new assets (fields, external events, etc.) in Emarsys, you can update your Magento interface at any time by clicking Update Schema.
You can also set up a regular check to run in the background to see if any new assets have been created. To do this, go to the Emarsys menu, Settings, Transactional Email and set the time and schedule for the check.
If the check finds a new asset, you will receive a notification prompting your to update your schema again.
Making a bulk export
Under the Emarsys menu, Operations, Bulk Export you can make a one-off manual export of your Magento data at any time.
Use cases for this include:
Customers and Subscribers - Select these if you want to update your Magento contacts in Emarsys, for example after you have added a new Magento field or changed the field mapping.
- For Customers, you will need to enter a time range. All customers registered during this time range will be exported.
- For Subscribers, you do not define any time range. All subscribers in your Magento account will be exported.
If you do map new fields for customers or subscribers in Magento, make sure to update the corresponding Emarsys auto-import profiles.
- Order - Use this option to upload offline sales data.
- Product - Since the entire product catalog is uploaded every time, this is only useful if you want to make an immediate export of your product catalog and not wait for the scheduled sync.
Under Emarsys, Operations, Logs you can find a list of all the integration events between Magento and Emarsys. This will help you debug the integration during installation, and monitor it afterwards.
For more details on an individual event, click View.
Under Emarsys, Settings, Logs you can define how long to store the logs for, clean up your log files, set up a schedule for log reports and download the current log files.