The Data Import page provides functionality for making ad hoc, manual imports and for scheduling regular auto-imports of your contact data into the Emarsys Marketing Platform.
User guides
Further reading
As a quick reminder, here is the onboarding video about Contact Data Import&Export:
What is Data Import?
Data Import provides an easy-to-use wizard for importing external data files to Emarsys and making sure that all the fields are correctly matched.
There are two types of imports, manual imports and auto-imports. Manual imports allow one-time uploads, while auto-imports can be set for regular synchronization events.
This is the method that you will most likely use for the initial upload of your contacts during the onboarding phase (this upload can be either a manual or an auto-import, depending on the size of your database).
You can also use manual import to upload all of your offline registrations collected from flyers, trade fairs, etc.
Data Import is designed for users with little technical know-how. If you have the technical resources to use the Emarsys API, we recommend that method for higher performance and real-time synchronization.
How does it work?
Manual imports
In a manual import, you upload a CSV file from a local source, match the fields and values with your Emarsys fields, and import it. The import is processed in the background and you will receive a notification when it is finished.
The import requires a unique identifier field which is used to check if a contact exists or not. New contacts are created in your Emarsys database, existing contacts are updated. For all fields, the new values will overwrite the old ones, with the exception of email opt-in.
For more information, see Matching the fields and values of the upload file.
Auto-imports
With auto-imports, you define a location where Emarsys will regularly check for files that follow a certain naming convention, and then import those files.
All contacts imported via the auto-import are saved in the Emarsys application as a contact list.
In order to configure an auto-import, you will need a sample file. This is used to match the fields and values and to check that the text and field delimiters are correct. It is then discarded (no contacts in the sample file are imported).
A sample file needs to have only one or two actual contacts but these must have:
- All the fields that will be included in the actual import files, in the same order as the import files.
- All the possible values for all the multiple- and single-choice fields in the import files.
Pre-processing
Automated imports also support so called pre-processing, which is where encrypted files can be automatically uncompressed and imported. The pre-processing can be configured by Emarsys as part of the Auto-import setup, and relies on the password for each file remaining the same. Both PGP (Pretty Good Privacy) and GPG (GNU Privacy Guard) are supported.
Emarsys supports the following standard compression formats: .zip, .rar, .arj, .tar, .tgz, .7z.
For more information, please contact Emarsys Support.
Opt-in management
The email opt-in field is managed by imports as follows:
For new contacts
You are asked to define the opt-in status in the import settings.
For existing contacts
Duplication handling manages the opt-in field as follows:
- If the opt-in field in the import file and in your account have the same value, the merged contact will keep this value, and the field history will be updated with the most recent timestamp of the two versions.
- If the import file has a value of
FALSEand your account hasTRUE, the merged contact will have the valueFALSE. - If the import file has a value of
NULL, and your account hasFALSE, the merged contact will have the valueFALSE. - If the import file has a value of
NULL, and your account hasTRUE, the merged contact will have the valueFALSE. - If the import file has a value of
TRUE, and your account hasFALSE, the merged contact will have the valueFALSE. - If the import file has a value of
TRUE, and your account hasNULL, the merged contact will have the valueTRUE.
The opt-in field should never have an empty string as the value as this will be treated as an invalid value. Use NULL instead.
Why should I be using Data Import?
The main use case for Data Import is when you are using Emarsys as your primary contact database and have scenarios where using the Emarsys API is not appropriate, for example uploading lists of contacts collected offline.
It is also the fall-back option if you do not have the technical resources to manage your contact data transfer via the API.
Finally, you can use auto-imports to trigger automated programs.
Supported functionality
You can do the following on the Data Import page:
- Check your current imports
- Check your import history
- Initiate a manual import
- Set up an auto-import
- Edit an import
- Abort an import
- Match the field and values of the upload
- Change the field matching of an existing auto-import
Getting started with Data Import
Before you start with Data Import
In the case of manual imports, the following have to be considered:
- Only CSV files can be uploaded.
- When you are initiating a manual import, please note that files should not be larger than 64 MB.
If you have requirements that are different to this, you should consider setting up an auto-import.
In the case of auto-imports, the following have to be considered:
- You will need a place to store the file that is to be uploaded. The following options are available for this:
- When setting up an auto-import, please consider that although uploading single files larger than 1GB is possible with WebDAV or SFTP, it is highly recommended to keep the size of the uploaded files below 1GB. Repeated uploads larger than 1GB can quickly use up the temporary space for processing, resulting in failed uploads.
Emarsys checks the WebDAV for new files every minute, the SFTP or FTPS servers every hour.
Contact lists created with importing contact data will be archived after a fix period if they are not used in a segments, Automation Center program or any campaigns. The archival date depends on customer and account specific settings. Please contact Emarsys Support to get to know the exact period.
File naming conventions
In general, try to avoid using the space character in your filenames. Use underscore instead, for example. Using space does not result in an error message but may cause malfunction.
Contact import will be unsuccessful if the part of the file name that the wildcard stand for begins with a space character. The file won't be found or recognized.
Example
filename: EDW_Contact_Feed 2021-06-30_115346.csv
filename with wildcard: EDW_Contact_Feed*.csv
In the above scenario, the file is not recognized even though it should be according to wildcard logic. The reason for this is that the initial character is a space. One workaround could be to use the wildcard in the following way:
filename: EDW_Contact_Fee*.csv
Data Import FAQ
No, you can only create new lists, but there are two workarounds:
- Include a unique field in the import file (e.g. named after the auto-import name) and create a segment that filters for contacts with the appropriate value for this field.
- Use the Automation Center On auto-import entry node and then follow that by a Save contact list node, with the setting Add new contacts to existing list enabled.
You cannot import more than one file manually via the import wizard – for this you will have to set up an auto-import. Auto-imports will pick up all the files that are correctly uploaded to the storage folder.
SFTP or FTPS servers are the most secure transfer protocols available and are our recommended option if you have the technical resources available. Emarsys checks these external sources once every hour and successfully imported files are deleted from the source.
A WebDAV is an SSL-secured source hosted and maintained by Emarsys. This is convenient for many clients because Emarsys Support will set it up for you (free of charge) and all you need to do is use a suitable client application such as Bitkinex or Cyberduck to access the folder, or program automatic updates directly from your CRM.
Emarsys checks the WebDAV of each account every minute and processes any new files it finds there. Processed files are saved back on the WebDAV with the following naming convention:
- filename.csv.done – for successful imports.
- filename.csv.error – for unsuccessful imports. A notification with details of the error will be sent to the Notification Center.
- filename.csv.ignore – for files whose names do not match the naming convention defined for any of the enabled auto-imports.
Note: if you use a compression file, the import file will be extracted to the WebDAV and renamed as above, and the compression file deleted.
We recommend including both static and dynamic parts in the file names. The static part should be clear and easily understandable, ideally relating to some aspect of your customer lifecycle strategy, while the dynamic part (e.g. a timestamp) will differentiate individual files. The dynamic part is represented by a wildcard asterisk * in the naming convention field.
Examples
- If you will only ever need one auto-import configuration, you can use a simple naming convention such as .csv. This will mean that every .csv file placed on the source folder will be processed.
- If you use more than one auto-import, then we recommend to match the static part to the auto-import name, and keep the dynamic part for a timestamp. In this way you could define the naming convention for the converted leads auto-import as: converted-leads-*.csv, and then name each individual file as: converted-leads-2015-02-12.csv
Please note that file names are case sensitive.
A sample file is necessary so that Emarsys knows which fields to expect in the import files, and how to map them and their values to existing fields in the Emarsys database. If you make any changes to the import files, however minor, you should upload a new sample file based on the new import file.
A sample file must contain all the expected fields, in the correct order and with all the possible values for any multiple- and single-choice fields.
No. Emarsys is only interested in the file settings and the fields and values.
For most Emarsys accounts, only UTF-8 encoding is supported. The only exception are the accounts on the environment www.emarsys.net, which support only Latin-1.
Make sure to upload your files containing only Latin-1 characters but with UTF-8 encoding to avoid any issues.
Please note that Emarsys does not verify the encoding used; if you use any other encoding, the contact data you import may be corrupted and unusable.
The import file itself must be a CSV file. However, you can compress this file. The following compression formats are supported: .zip, .rar, .arj, .tar, .tgz, .7z.
If you do want to use a compressed file format, make sure you include the naming convention for the compression file and any password required to open it.
This populates the field Registration language. This is purely in case you want to segment your contacts via language, for example when sending newsletters in multiple languages.
Changing the language setting of the import file can affect two things:
-
The field names and values
If a file language is defined, Emarsys will use that language to match the fields and their values in the import file. Please note that this requires you to provide the relevant translations for any custom fields included, in the respective language(s). -
The date format
Certain date formats are associated with different languages. Emarsys will check the sample file for the date format and if this does not match the language, an error will be shown in the auto-import wizard. The supported date formats are:- YY-MM-DD
- DD.MM.YY
- MM/DD/YY
- DD/MM/YY
- This is a precaution that we take to ensure that the same file cannot be imported twice.
Auto-import configurations which were set up prior to the release of Emarsys version 8.2 (February 2015) can still have settings which are either not yet supported by the new import feature, or which have since been superseded by improved functionality. Examples of such features are ‘Newsletter opt-in’, ‘on-import campaign’ or ‘no duplication handling’. If you want to edit these auto-imports, please contact Emarsys Support.