Loading the Sales Items File
Is Web Extend required for loading the sales data file?
No, Web Extend is not required. It is Smart Insight that collects your sales data from external sources.
What happens if there is no order ID in the sales data file?
The loading fails. This is a mandatory field. The file must be corrected and sent again.
What happens if the order date is missing from the sales data file?
The loading fails. This is a mandatory field. The file must be corrected and sent again.
What happens if the customer ID in the sales file is missing from the contact database?
We ‘create’ a pseudo contact for this order (basically all such orders are attached to one pseudo contact). This contact would appear as ‘generated’ on the Smart Insight screens and could not be matched with any contact in the Emarsys application.
Could the above issue be fixed by re-sending the same order with the customer ID added?
No. This would duplicate the order, one would be loaded for the ‘pseudo’ contact and the other to the ‘real’ contact. The file must be corrected by you and a new complete reload is done by us. You are intended to communicate the problematic file load to Support and we do the reload.
What happens if the customer ID is missing from the sales data file? (customer ID = unique customer identifier field that we use to match orders with Emarsys contacts)
An order with a customer ID missing from the Emarsys contact database will be loaded to the data warehouse and the related contact will be generated with only an internal Smart Insight customer ID (that you cannot use later for identification) and flagged as generated
in Customer Lifecycle. The value of the number_of_purchase
field of contacts flagged as generated
is set to 1. Although a single contact might make multiple purchases, these contacts cannot be merged into a single entry in the database. By using the generated
filter and setting the value of the number_of_purchase
field to 1, this contact can be separated from the ones where the connection could be made between the orders file and contact DB.
Could the above issue be fixed by populating the missing customer IDs in the Emarsys application?
Yes. If the unique customer identifier is added to a contact who previously was loaded as ‘generated’, they will turn to a ‘valid’ contact by the next data load.
What columns are mandatory in the sales data file?
- order - The order ID of the order that this item was purchased in.
- timestamp - The date of the purchase.
- customer - The ID of the customer who made the purchase. This must be contact data from the Emarsys database.
- item - The ID of the product that was purchased.
- price - The total price of the item.
- quantity - The number of items bought.
What date formats are accepted?
YYYY-MM-DD is preferred. If HH:MM:(SS) is also sent, time will not be taken into account.
Can we have additional columns?
It is possible to add custom fields to the sales file as long as the field name starts with prefix “c_” (e.g. c_brand, c_webshop). Please note that the order of the columns matters. Columns can be sent in any order, but it must always be the same. End of columns must not contain empty rows.
Is it possible to have the unit item price and the number of purchased items (quantity) in the files and would SI calculate the total sales amount based on these?
No, the files we load contain the total price for items (e.g. 2 items cost 5 EUR altogether). No unit price is displayed and we don’t support such calculations in SI.
If there is an error in the sales file, will the good records still be loaded and the bad ones omitted from the load?
No, the load process stops and nothing from the sales file will be loaded. The error has to be fixed by you and the file needs to be reloaded after the fix by us.
What happens with the contacts who have purchase(s) which are negative (have only refunded items) or zero (have only free items)?
They will be loaded as leads (contacts who have not bought anything yet). If they are not identifiable (cannot be matched with any Emarsys contacts) they will be ‘generated leads’ with ‘Unknown’ lead source and ‘New lead’ lifecycle status.
What happens if there is no header in the sales (or product) data file?
If there is no header in the file, the first row will be lost as this will count as a header. Header is required.
What happens if we get a purchase without a product ID in the sales data file? Do we load that as an unknown item?
No, in this case the loading would fail. We only create ‘unknown products’ when the product ID for an order is missing from the product catalog. The file must be corrected and sent again.
Why is it important to upload the historical sales data and product data file at same the time?
To ensure that Smart Insight calculations and analytics work properly, the product catalog has to include the items the sales data file is referring to. Otherwise, Smart Insight reports will contain unknown products and this issue cannot be resolved even by uploading your historical product data file again.
When do new purchases appear in Smart Insight lifecycle segments and reports?
New purchases are displayed in Smart Insight lifecycle segments and reports only after a data load that takes place on the following day (new purchases will not be registered even if you perform multiple sales data loads on the same day when the data loads are made).
The filter Total spent on selection is an exception. This Smart Insight filter is based on real-time purchase data as opposed to the Total lifetime spend filter which is calculated after the data load that takes place the following day. For this reason, we recommend combining the Total spent on selection filter with other purchase attributes. Combining it with contact attributes such as Customer lifecycle stage may lead to inappropriate results. In this case, we recommend combining the filter Customer lifecycle stage with Total lifetime spend.
How do I need to format my sales data file if I’m using the Batch Sales Data Load solution?
If you are using our Batch Sales Data Load solution, then you need to format your sales data file according to the Standard Sales Data CSV file requirements.
Where can I check my sales data loads and data quality if I'm using the Batch Sales Data Load solution?
If you are using our Batch Sales Data Load solution, then you can check your sales data loads and data quality notifications on the Management > Sales Data Loads page.
Loading the Products File
How should the product data file be sent?
Either:
- initial product catalog containing all products + daily deltas with the changes, or
- full product catalog each day, or
- daily feed only with the products that were ordered that day?
You should upload your full product catalog every time, to ensure consistency.
What columns are mandatory in the product data file?
These columns are needed for SI, only the itemID is mandatory.
- item (string) – The unique ID used in your system to identify an item. This is the ID that all data-collection and rendering scripts will refer to.
- title (string) – The name of the product.
- category (list of hierarchy levels) – Describes the category that a given item belongs to (e.g. Electronics>Phones>iPhone|Telecommunication>Smart Phones>iPhone). A hierarchy can consist of maximum 5 categories, but any number or hierarchies can be included.
What happens if there is no item (productID) in the product data file?
The loading fails. The file must be corrected and sent again.
What happens if the item is not unique within a single file?
The loading fails. The file must be corrected and sent again.
What happens if multiple products files are loaded at once and the same item can be found in several files?
The files are loaded one by one (in alphabetic order) thus the last occurrence of the item will overwrite all the previous ones.
What happens if there is no name for the product in the product data file?
The item will be loaded and displayed as Unknown product ‘product ID’ on the second graph of the Past Purchases screen. In this case, a product is created by us in the Emarsys database with only a product ID, no other data is provided for it. Note that if an item is not included in the product catalog, it will also be an unknown product.
What happens if there is no category in the product data file?
These items will be loaded, but they will not appear under any of the categories on the Past Purchases screen.
What happens if a product comes with an item (unique ID) already loaded to SI?
If an item that was loaded earlier comes again in a new file, the old item will be overwritten by the new one (e.g. modified product category or correction in item name).
Which category levels can be displayed on the first chart of the Past Purchases, Top Products, Average Order Value and the Dashboard screen?
By default, it is the first (top) level but any of the category hierarchy levels can be chosen for visualization. However, please note that the specified level cannot change from item to item (if it is set for category level 2, it applies to each and every item).
Refunds Handling
Are refunds figured in the total number of orders?
Refunds themselves are not added to the total number of orders, but full refunds affect this metric as fully refunded orders are not included in the total number of orders. As a rule of thumb, any order or order part with a positive sales amount is added to the total number of orders, while those having a negative or 0 sales amount are not.
How should the refunds be included in the sale data file?
When submitting returned items or orders, always use the order, customer and item IDs of the original order and provide a negative value in both the c_sales_amount
and the quantity
fields. Please make sure to populate the fields in your sales data file this way, otherwise refunds will not be properly represented in your reports.
Example
This is how the original order is uploaded in the sales file:
Order ID | Date | Customer ID | Item ID | c_sales_amount | quantity |
---|---|---|---|---|---|
OR12345 |
15.10.2015 |
CU54343 |
IT8798 |
45 | 1 |
OR12345 |
15.10.2015 |
CU54343 |
IT2235 |
30 | 1 |
OR12345 |
15.10.2015 |
CU54343 |
IT8840 |
25 | 1 |
A week later the customer decided to return 2 items, which should be reported in the sales file as follows:
Order ID | Date | Customer ID | Item ID | c_sales_amount | quantity |
---|---|---|---|---|---|
OR12345 |
22.10.2015 |
CU54343 | IT2235 |
-30 |
-1 |
OR12345 |
22.10.2015 |
CU54343 |
IT8840 |
-25 | -1 |
These two order parts will then be aggregated into a single order with the total value of 45, while the order date will remain unchanged (i.e. we keep the date of the original order).
How do refunds affect the revenue impact screen?
On the Revenue Impact screen the orders are not aggregated. Each order contributes to the revenue of the day when it was made. If we take the above example, the 100 Euros will be added to the total revenue of 15 October, while the refund worth -55 Euros will be reported for 22 October.
Can we send the refunds in a separate file?
Sure. Just make sure that you follow the sales file naming convention, i.e. you name the file using the pattern sales_items*.csv
. In other words, the file name must begin with the string sales_item
and its extension must be .csv, while you can replace the asterisk with a number of characters of your choice.
How do returned orders affect customer lifecycle scoring?
As far as lifecycle scoring is concerned, any positive order counts as an activity, even if it is returned later. For example, if you upload an order with a positive sales amount for any of your First-time buyers, they will become and remain Active buyers even if the order will be refunded later, either fully or partially. Please note, however, that orders having a negative or zero value do not affect your customers’ lifecycle scoring.
The Refunds affect lifecycle calculations option and orders with 0 sales amount:
- Regardless of whether the Refunds affect lifecycle calculation option is turned on or off, orders with 0 sales amount will not affect the Customer Lifecycle calculations.
- If the Refunds affect lifecycle calculation option is turned on, then refunds will affect the number of orders, except for orders with 0 sales amount.
Are refunds added to the contacts’ purchase statistics?
Returns and refunds are not taken into account when calculating the number of purchases or the last purchase date. They do not count as an engagement, either, so they are not considered when last engagement is calculated.
In general, refunds only affect the total turnover of the contact and thus their buyer status.
How do refunds affect the average order value shown on the reporting page?
Average order value is calculated by dividing the total of all sales amounts by the number of orders in the given period. Refunds are taken into account when calculating the total sales amount.
How about the total lifetime spend filter?
When calculating Total lifetime spend and the buyer status, all orders are taken into account, irrespective of whether their aggregated value is positive or negative.
How do refunds affect the Monetary score, which is always calculated for a given period?
The Monetary score is calculated by summing purchases over a defined time period. For returns, it is the date of the original order that matters. If that falls outside the time range, the order will not be taken into account when calculating the Monetary score (as well as total spend and buyer status), irrespective of whether the return transaction is within the given period.
Loading Contacts
What happens if the custom registration date field is empty or doesn’t exist in the Emarsys application, and the customer has older order dates than his/her first registration date in the Emarsys application?
The registration date of these contacts will be set to the same as the first order date in SI by us. This won’t affect the Emarsys side since we don’t write back to Emarsys from SI, so the custom registration date field will remain empty in the Emarsys application.
What happens if the custom registration date field is newer than the Created date/Date of first registration?
The registration date of these contacts will be the date defined by Created date/Date of first registration. The custom registration field will not be applied in the last engagement in this case.
What happens if a contact has only click data but any other data of this contact is missing?
The contact was deleted but its click data remained. This contact will appear on the Lead lifecycle page, its lifecycle status will depend on the date of click and the lead source will be Unknown.
Filters
What are ‘Total Amount spent’ and ‘Number of purchases’ on the Contacts/Segmentation page?
These filters can be used to create segments like ‘total spend or number of purchases in a given time period and /or to a given product.’ These filters differ somehow from the two filters with similar names on Customer Lifecycle (Total lifetime spend and Total lifetime purchases) because while those work from contact data, these two work from purchase data. Those contacts who register and purchase on the same day are regarded as leads for the first day and would turn to customer only the second day.
However, their orders would be loaded to the data warehouse on the first day already, while they are leads from the system’s point of view. This can result in slightly different numbers between the segments created with these similar filters.
Duplication
What happens if there are duplicate orders in my sales data?
Due to incorrect sales data (i.e. duplicate orders), your results or Smart Insight screens will be distorted. In this case, we recommend reloading your Smart Insight data. For more information, see Reloading data into Smart Insight.