You can check your data loads, error details and potential data quality issues on the Management > Sales Data Batch Loads page.
The Sales Data Loads page consists of the following tabs:
Load Overview
Here you can check each sales data file uploads in the last 60 days. The Load Overview tab consists of the following widgets:
To change the selected time period, click the Date Range field.
Successful Order Loads
You can check your orders that were successfully loaded into Emarsys during the selected time period in this widget.
This widget comes in handy when you need to:
- Search for days when no orders were loaded into Emarsys.
- Check irregularities in order loads: files are uploaded to Emarsys but they contain less orders than expected.
- Investigate when data load irregularities began.
- Answer questions such as: When was the last purchase date? Are there any days with missing purchase data?
The numbers displayed in this widget also include orders with data quality warnings.
To display more details for a specific day, hover your mouse over the chart.
Here you can check the following metrics:
- Number of Loaded Orders - The aggregated number of orders that were successfully loaded during the selected time period. Here you can also see the number of files containing the orders that were either successfully or partially uploaded.
- Last Missing Load - The last date when all file uploads failed or no upload was initiated.
You can switch between the following views:
- Load Date - It displays orders based on the date when the sales data files containing the orders were loaded.
- Order Date - It displays orders based on the date when they were made.
File Load History
You can check your sales data loads in this widget along with the following information:
-
Load Status - The following statuses are available:
- Full Load
- In Progress
- Partial Load
- Failed Load
For more information, see Load statuses and error types in CSV loads.
- Load Date
- Successful Order Loads - The number of orders in the sales data file that were successfully loaded.
- Failed Order Loads - The number of orders in the sales data file that couldn’t be loaded.
- Total Orders - The total number of orders in the sales data file.
You can filter the results by using the Search field or by selecting a Load Status.
Checking the Sales Data Loads details
To check the details related to a sales data upload, click the View Details icon.
Here you can check the following additional information:
- Error Details - Specific details describing why the sales data file upload failed or was partially loaded. Below the error message, a maximum of 10 examples are provided for each error type from the sales data file. You can also see the affected fields. To check the rest of the error messages, export the details by clicking Export Data.
- Load Duration - The time it took to load the sales data file.
- Total Lines - The number of lines in the sales data file.
- File Size
Order details are not available in the case of some failed data loads because Emarsys couldn’t process the sales data file.
To export the complete file or the orders containing errors, click Export Data. You can export the sales data file.
Notes:
- You can export the sales data file for 30 days after the upload.
- You cannot export the sales data file if the upload failed due to exceeding the size
Load statuses and error types in CSV loads
Full Load
The sales data file was uploaded successfully and all the orders are loaded into Emarsys.
Partial Load
The sales data file contains a single or multiple orders that were not loaded into Emarsys due to the reasons listed below. The rest of the orders are loaded and successfully processed by Emarsys.
Error types:
- Invalid data type: a single or multiple values do not fulfill the specified data type defined in your configuration.
- Invalid data format: The sales file contains a single or multiple lines that are not formatted correctly.
- Unaccepted value: The sales file contains a single or multiple lines with values that do not fulfill the requirements.
- Mandatory value missing: a mandatory value is missing from the order.
If either the transactionType
or transactionId
is missing, the file upload will fail.
- Extra value: A single or multiple delimiters were not escaped and as a result, there are more columns in the affected lines than specified in your configuration.
- Non-unique external_id: An external ID appears multiple times within a transaction.
- Refunds value error: The
transactionType
is set torefund
but theamount
is not a negative number. - Refunds without reference ID: The
transactionType
is set torefund
but it does not contain areferenceOrderId
.
The referenceOrderId
cannot be empty. Ideally, it has to match an existing order ID.
- Top level field error: A top level field is different within a transaction.
- Future order: The
transactionCreatedDateTime
ortransactionLastChangedDateTime
contains a date that is later than the date contained byreceived_at
. - Invalid account_currency: A field contains a currency different from the one specified in your configuration.
- Duplicated orders: The
transactionId
already exists for thetransactionType
in the last 30 days.
Failed Load
None of the orders in the sales data file could be loaded to Emarsys due to the following reasons.
Error types:
- All orders in the sales data file contain errors.
- The file size exceeds the limit (100 GB).
The file or the errors cannot be exported in this case.
- The format is invalid. Only CSV files can be uploaded.
- A required column is missing from the file or is misplaced.
- A mandatory value is missing: The value
transactionId
ortransactionType
is missing from the order. - A single or multiple columns contain more characters than allowed.
- The sales data file contains a quotation mark that has not been closed.
- A quotation mark is not escaped.
- The sales data file contains invalid encoding. UTF-8 encoding is required.
- Values are missing from a required column.
- The sales data file contains an unknown error type.
Fixing Failed and Partial Loads
In the case of Failed and Partial Loads, proceed as follows:
- You can access orders or sales data files containing errors by exporting them. To do so, click Export Data.
- After resolving the issues, you can upload the corrected orders or sales data files again.
We suggest uploading only the orders or sales data files containing errors in this case. Uploading orders successfully multiple times within 30 days will result in duplicate order error messages.
- After a successful upload, orders appear correctly in the database.
The status of the sales data file you have just uploaded appears in a new row with a new upload date in the Sales Data Load History widget. The status of the original Failed or Partial Loads and the number of errors they contain remain unchanged.
Data Quality
Here you can check the quality of the sales data successfully uploaded to Emarsys. You can decide how to handle orders that might contain errors depending on your business model and data load practices.
You can check the following metrics on the Data Quality tab:
By default, the metrics are calculated based on the sales data loaded into Emarsys in the last 7 days. To change the selected time interval, click the Date Range field.
Each widget displays the numbers and percentage of the affected orders along with the total number of orders for the selected time period on the Data Quality tab. The widgets consist of two parts:
- Chart: Here you can see the number of impacted orders based on the date when they were loaded. This is useful when you need to investigate when a specific issue has started.
- Details: Here you can check the definition of the specific metric and its implications.
Managing Data Quality Issues
To check how your data quality issues are handled, click Manage Data Quality Issues.
Depending on your business model or data load practices, you can ignore specific metrics as follows:
- Select the Data Quality Issue to be ignored.
- Enter a comment why you would like to ignore the specific issue. You can edit this comment anytime. Comments are visible to all users connected to the specific account. For example, you can help your colleagues with such comments explaining to them why a decision was made to ignore the specific issue.
When selecting data quality issues to be ignored, please note that:
- The chosen metrics will not be shown on the Data Quality screen.
- Notifications for the selected metrics will not be displayed.
Only one notification is displayed at the top of the Data Quality tab indicating that the specific issue is ignored.
Missing Product Catalog Entry
Definition:
The orders in your sales data file contain products that cannot be found in the product catalog (i.e. the productId
does not match any product in the product catalog).
Impact:
This issue affects Audience Reporting and the Product Affinity Reports.
Anonymous Contacts
Definition:
The sales data file contains orders made by contacts without an external contact ID (i.e. their externalContactId
is NULL
).
Impact:
- Anonymous contacts are missing from segments and Automation programs and they will not receive your campaigns.
- Reports based on lifecycle calculation are affected.
- All reports based on attributed revenue are affected.
Unidentified Contacts
Definition:
The sales data file contains orders made by contacts who cannot be associated with a contact in the Emarsys contact database (i.e. their contact_id
is NULL
).
Impact:
- Unidentified contacts are missing from segments and Automation programs and they will not receive your campaigns.
- All reports based on attributed revenue are affected.
Emarsys updates contact data on a daily basis, so it might happen that this issue is resolved without any intervention if contacts related to the affected orders are loaded into Emarsys later on.
Duplicated Item Purchases
Definition:
The same transactionId
and transactionType
contains the same productId
with quantity
and amount_in_account_currency
multiple times.
Impact:
This issue might affect lifecycle analytics, buyer status calculations and Revenue numbers on all reporting screens and the Average Order Value.
Fixing data quality issues
Data quality issues can be fixed only after deleting the corresponding sales data.
Currently, only the complete sales data can be deleted. For more information, see Deleting Sales Data.