Here you can find instructions on how to work with Data Import. This can be found under the Contacts menu, Data Import.
As a quick reminder, here is the onboarding video about Contacts Data Import&Export:
Checking the ongoing imports
The ongoing imports, both manual and auto-imports, can be checked on the Current Imports tab of the Data Import overview page.
This gives you a full description of the content of the import file, including how many entries will be created as new contacts and how many will be merged with existing contacts.
If there are no active imports, you can start a manual import.
Click Import File, you can set up a manual import or an auto-import.
Checking the import history
You can get an overview of all your previous imports on the History tab.
You can also initiate the workflow for creating a manual or auto-import by clicking Import Data.
A Field Delimiter is called a Delimiter in the Advanced Settings menu. A Text qualifier is called a Separator.
Data consistency
The consistency of the data you want to import plays a crucial role in the success of the manual data import. If you import a file with duplicates, there is no telling which value ends up as the final one for the contact on the Emarsys platform. The reason for this is that our import algorithm does not perform a linear import job, it consists of multiple simultaneous import processes. It is not a traditional row-by-row or top-to-bottom process.
Let's say that you have three different values for the same field scattered throughout your file to import and the right value for that field is at the bottom. That value has the same chance of being taken as the final one as the other two values before it. So, if you end up with the intended value, it is simply sheer luck.
As an example, consider the following situation, where the Age value for Contact 1111 could be any of these three:
Line 1 -> Contact 1111 -> Age = 11
Line 200 -> Contact 1111 -> Age = 22
Line 3000 -> Contact 1111 -> Age= 33
Cleaning the file to import, getting rid of the duplicate data, must be done before performing the manual import, otherwise data quality cannot be guaranteed.
Initiating a manual import
The Match Fields and the Match Values steps of the wizards for manual import and auto-imports are identical and they are discussed in a separate chapter, below. In this chapter we focus on steps unique to the manual import process.
To initiate a manual import, click Import File on the Current Imports or on the History tab of the Data Import overview page and then select Import File Manually in the pop-up window.
Drag and drop a file inside the dotted lines or click Select a file to browse for one locally.
Please note the limitations concerning format and size:
- Import files must be in .csv format.
- When you are initiating a manual import, please note that files should not be larger than 64 MB.
For the Match Fields and Match Values steps of the wizard, see chapter Matching the fields and values of the upload file.
In the Finish step, you can set a name for the contact list that will be created from the imported file, and trigger the import.
Setting up an auto-import
The Match Fields and the Match Values steps of the wizards for manual import and auto-imports are identical and they are discussed in a separate chapter, below. In this chapter we focus on steps unique to the auto-import process.
For auto-import configuration related question, see Auto-import FAQ.
To initiate an auto-import, click Import File on the Current Imports or on the History tab of the Data Import overview page and then select Create Auto-Import in the pop-up window.
When you are setting up an auto-import, please consider that although uploading single files larger than 1GB is possible with WebDAV or SFTP, it is highly recommended to keep the size of the uploaded files below 1GB. Repeated uploads larger than 1GB can quickly use up the temporary space for processing, resulting in failed uploads.
Source File Settings
In this step of the wizard you first have to enter a name for your auto-import.
You then define the name of the contact list that will be created after every successful import. There are two options:
- Based on the Auto-import name - with a timestamp added, e.g.: auto-import-name YYYY-MM-DD HH:MM
- Based on the uploaded file name
Next, you have to define Where to look for the import file.
This can be:
- Emarsys WebDAV
- Remote source
For the latter you have to define a path for the remote source as well as the credentials for accessing it.
If you select sftp as your source and Key as your Authentication type, you can select your previously created key from the drop-down menu.
Please note that if you are using password-based identification on the remote host, the password cannot contain the following characters: "/", "\" and "#".
Emarsys will verify these credentials on completing the wizard. If there is an error, a message will appear in the Notification Center.
Now you need to define the Import file settings.
You have to specify a File name. This is a crucial step because you may configure many auto-imports in your account, and Emarsys must know exactly which naming convention applies to which import configuration. You can use the wildcard "*" to manage the dynamic parts of the file name, such as timestamps.
- Example 1: "*.csv" will import all files with a .csv extension.
- Example 2: "newsletter*.csv" will only import CSV files starting with "newsletter".
Important:
- Please note that file names are case sensitive.
- Please use only the following characters when specifying the file name:
A-Z,a-z,0-9and_. Special characters are not supported.
Finally, specify the naming convention and password for the compression file, if one is used.
The Platform will only extract the files defined in the Compression file name field. These will not be automatically uploaded.
Example:
You have a folder with multiple files following the naming convention:
-
_data.csv(You want to upload these.) -
_info.csv(You want to upload these.) -
_compressed.csvThese may contain_data.csvfiles. -
_overhead.csv(You do not want to upload these.)
You will need two auto-imports to upload all files ending with _data.csv and _info.csv.
Auto-import 1:
- File name:
*_data.csv - Compression file name:
*_compressed.csv
Auto-import 2:
- File name:
*_info.csv
These two auto-imports will upload _data.csv and _info.csv files even if they are packaged inside _compressed.csv files. However, _overhead.csv files will not be uploaded.
Sample File
Now you must upload a sample file.
The sample file should contain just one or two contacts but must contain all the fields you will use in your import files, in the same order and with all the options for single- and multi-choice fields.
A sample file is necessary so that Emarsys knows which fields to expect in the import files, and how to map them and their values to existing fields in the Emarsys database.
If you make any changes to the import files, however minor, you should upload a new sample file based on the new import file.
No contacts are imported from the sample file.
For single- and multi-choice fields, both the field and all the available values must already exist in Emarsys.
When the file is uploaded you will be automatically taken to the next step: Matching the fields and values of the upload file.
Matching the fields and values of the upload file
Matching fields and values correctly is critical to ensuring that your contact data is stored properly and can be used for segmentation and personalization.
You should make sure that your data is stored in the appropriate field type before importing it, and that these fields are matched with corresponding fields of the same type in Emarsys. Storing data in the wrong type of field will make segmentation harder and can result in errors.
Match fields
In this step of the wizard you have to match the fields for your imports - this step is identical for manual and auto-imports.
Settings
In the Settings section select the Unique identifier, the Opt-in declarations and the Contact language to use for the imported file.
The Unique identifier has to be defined to be able to handle duplicate email addresses.
The Opt-in declarations setting is compulsory, the options available are:
-
Opt-in everyone in this import – All imported contacts are marked as Opt-in =
True. - In file – The opt-in setting is included in the file. You must select which field this is.
- Status undefined – No action is taken with regard to the opt-in.
-
Unsubscribe everyone in this import – All imported contacts are marked as Opt-in =
False.
The Contact language drop-down will populate the field Registration language for all contacts in the import.
Advanced Settings
Show the Advanced Settings if you want to:
- Change the delimiter or separator used in the import file.
- Change the File language (this will affect how Emarsys recognizes the field names in the import file and matches them to fields in the database).
- Change the Date format used in the file. The supported date formats are:
- DD.MM.YYYY
- DD/MM/YYYY
- MM/DD/YYYY
- YYYY-MM-DD
Note that these date formats are only supported for the CSV contact data import. If you use the contact API, the only date format supported is YYYY-MM-DD
For more information, see: CSV guidelines: Date format, Contact System Fields.
- Ignore empty values - this will ensure that you do not overwrite any existing values with an empty string.
- Update only - This will ignore any contacts which do not already exist in your Emarsys database.
Delimiters are used to separate fields in CSV files. Separators are used to identify fields when a field may also contain a delimiter character in its text.
Example:
- Delimiter: ,
- Separator: ;
Super co.,Mega co.,;Smith, Son and Grandson co.;,
A delimiter is called a Field Delimiter in the Import Details. A Separator is called a Text Qualifier.
Super co. |
Mega co. | Smith, Son and Grandson co. |
Field matching
In the Field matching section, which is an identical step for both manual and auto-imports, match the default fields in the drop-down menus to the fields used in the imported CSV file.
If you have fields you do not want to include in your import, click Do not import (or select it from the menu). These fields will then display the Do not import flag. To reverse this decision, simply match the field again to an Emarsys field.
In the case of single- and multi-choice fields, match all the possible values to the parent field. If the name of the values in the import file is the same as the name of the values in Emarsys, they will be automatically matched. Otherwise you will have to map them manually in the Match Values step.
If you select Internal_ID as a unique identifier here, all previously selected fields will be deselected and you cannot add any other unique identifiers. If you want to use another field, you have to deselect the Internal_ID field first.
Click Next Step when you have finished matching the fields.
Match Values
If you have single- or multi-choice fields in your sample file, you need to match their values to the Emarsys fields. If the names of the values are identical in Emarsys and the import file (including capitalization), they will be matched automatically, otherwise you match them manually here.
Select the relevant values and click Next step when finished.
Summary
When all settings have been applied, a summary of the auto-import is displayed.
Click Save to activate the auto-import.
If the import fails, for example due to missing unique identifier or mismatched field count (the number of fields in the example file does not match the number of fields in the import file), you will receive a message in the Notification Center giving the reasons and asking you to fix the errors and try again.
Changing the existing field matching of an auto-import
If you need to change the field matching for an auto-import, click the Edit icon for the desired element in the list of auto-imports.
When you click Change Field Matching, you will be prompted to upload a new sample file, which should contain the all the field and values that you wish to match. From then on the process is the same as described above.
Auto-import FAQ
A WebDAV is an SSL-secured source hosted and maintained by Emarsys. This is convenient for many clients because Emarsys Support will set it up for you (free of charge) and all you need to do is use a suitable client application such as Bitkinex or Cyberduck to access the folder, or program automatic updates directly from your CRM. Emarsys checks the WebDAV of each account every minute and processes any new files it finds there. Processed files are saved back on the WebDAV with the following naming convention:
- filename.csv.done - for successful imports
- filename.csv.error - for unsuccessful imports. A notification with details of the error will be sent to the Notification Center.
- filename.csv.ignore - for files whose names do not match the naming convention defined for any of the enabled auto-imports.
- for files previously successfully uploaded with the same name. Avoid reuploading the same files repeatedly.
Note: if you use a compression file, the import file will be extracted to the WebDAV and renamed as above, and the compression file deleted.
SFTP or FTPS servers are the most secure transfer protocols available and should be used if you have the technical resources available. Emarsys checks these external sources once every hour and successfully imported files are deleted from the source.
We recommend including both static and dynamic parts in the file names. The static part should be clear and easily understandable, ideally relating to some aspect of your customer lifecycle strategy, while the dynamic part (e.g. a timestamp) will differentiate individual files. The dynamic part is represented by a wildcard asterisk * in the naming convention field.
Examples
- If you will only ever need one auto-import configuration, you can use a simple naming convention such as .csv. This will mean that every .csv file placed on the source folder will be processed.
- If you use more than one auto-import, then we recommend to match the static part to the auto-import name, and keep the dynamic part for a timestamp. In this way you could define the naming convention for the converted leads auto-import as: converted-leads-*.csv, and then name each individual file as: converted-leads-2015-02-12.csv
Note: File names are case sensitive.
A sample file is necessary so that Emarsys knows which fields to expect in the import files, and how to map them and their values to existing fields in the Emarsys database. If you make any changes to the import files, however minor, you should upload a new sample file based on the new import file.
A sample file must contain all the expected fields, in the correct order and with all the possible values for any multiple- and single-choice fields.
No. Emarsys is only interested in the file settings and the fields and values.
For most Emarsys accounts, only UTF-8 encoding is supported. The only exceptions are the accounts on the environments www.emarsys.net and www1.emarsys.net, which support only Latin1. Please note that Emarsys does not verify the encoding used; if you use any other encoding, the contact data you import may be corrupted and unusable.
The import file itself must be a .CSV file. However, you can compress this file. The following compression formats are supported: .ZIP, .ARG, .GZIP, .TAR, .TGZ, .BZIP2, .7Z. If you do want to use a compressed file format, make sure you include the naming convention for the compression file and any password required to open it.
None. This is purely in case you want to segment your contacts via language, for example when sending newsletters in multiple languages.
Changing the language setting of the import file can affect two things:
- The field names and values. If a file language is defined, Emarsys will use that language to match the fields and their values in the import file. Please note that this requires you to provide the relevant translations for any custom fields included, in the respective language(s).
- The date format. Certain date formats are associated with different languages. Emarsys will check the sample file for the date format and if this does not match the language, an error will be shown in the auto-import wizard. The supported date formats are:
- YY-MM-DD
- DD.MM.YY
- MM/DD/YY
- DD/MM/YY
Please, note that you must map all your expected fields in the uploaded file to the fields in the Emarsys database at auto-import setup. If your subsequent files contain unmapped fields, the following can happen:
- In case of new contacts, they are created with empty fields where the unmapped values should be
- In case of existing contacts, the fields, where the new unmapped values should be, will be empty