Once your product data file is ready, upload it to Emarsys, check if it meets our format requirements and then schedule regular updates.
To perform these steps, go to Management > Predict Data Sources and in the Product Data box click Manage product data. This takes you to the Product Data page, where you can manage everything related to your product data.
Your first upload
The initial product data upload is used to validate the format of your data file. You have two options for uploading:
Uploading from the URL where your product data file is hosted
You can use this option to upload and validate both GPF (Google Product Feed) and CSV product data files.
- In the Settings box, select the file format (Google Product Feed or Emarsys CSV):
-
Google Product Feed:
- Enter all your URLs and format settings per locale.
- If your country/region and language are not listed in the locale drop-down, select United States. If you want us to add a country/region or language to the list of supported locales, please contact Emarsys Support.
- In case of adding multiple locales, you can set the default one.
- Optionally, the default Image optimization function can be turned off when your use case requires disabling optimization, or your image size parameters cannot be fixed the other way.
- You can enable or disable the Parent/Child Catalog feature that allows you to handle two-tier product structure, suitable for product catalogs containing product variants. For more information, see Parent/Child catalog feature.
- Enter all your URLs and format settings per locale.
-
Emarsys CSV
- Paste the full URL of the product data file into the URL for scheduled updates field.
-
Google Product Feed:
You can use a wide range of protocols for hosting your Product Data file.
For uploading your data file you can use the following protocols:
- SFTP (SSH File Transfer Protocol)
- FTPS (FTP over SSL/TLS) These must start with ftps://.
For scheduling the Product Catalog using the URL method it is necessary to allowlist the ScarabBot/1.0 user agent.
If you upload your Product Data file to a password-protected storage, enter the file URL in the following format:
https://username:password@mywebshop.com/catalog.csv
or
sftp://username:password@mywebshop.com/catalog.csv
For a full list of supported protocols, see the documentation of curl, the tool we use to fetch the data file from the URL you specify.
- You can Schedule how often the Product data will be updated.
- Click Save and Update from URL now.
For more information, see How long does it take for Predict to process my product catalog?
Optimizing GPF product data image size
Image optimization function ensures that your uploaded GPF product image is displayed and large-sized images (wider than 800 px) do not slow down the loading of the webpage. By default, image optimization is enabled. Optionally, you may decide to turn it off.
You must be aware of the following:
- Enabled: Product data image size is automatically optimized to have the best dimensions for displaying it, however the image may get distorted.
- Disabled: Product data image with incorrect size (wider than 800 px) may not appear. You can reduce the size of the image by Resizing images with the Emarsys image proxy.
Uploading your CSV product data file from your computer
You can only use this option to upload and validate CSV product data files smaller than 100 MB in size.
- In the Manual update and validation section, click Upload file to validate.
- Browse for your CSV product data file.
Validating and publishing product data
Once you have uploaded your product data file, we run our validation checks and display the results.
If issues are detected, they are listed with a short description and the affected fields highlighted. In such cases, click Discard this update, fix the issues and submit your product data file again.
As you can always discard a product file update without affecting existing product data, using the Update from URL or Update file to validate options is a safe way to test a new product feed. So feel free to experiment with your product data file!
If no issues are detected, click Activate update. It takes up to a few minutes for your product data to be published. You can follow the status of the upload in the Current Product Data box.
You can activate a product data file even if it passes the validations with warnings. But this is not recommended as the items which have issues will be missing from your Emarsys product database. In other words, you will not be able to use them in your campaigns and they will not be shown in any reports.
Checking published product data
Once the validated product data file is uploaded, the Product Search box becomes active, allowing you to search for products in your Emarsys product database.
Experiment with searching for items. Clicking a search result will take you to the Product Info page, which lists all the fields found in the Emarsys product database for the given item.
Notes
- The Product search is case sensitive.
- If your product data file contains more than 2 million items, you will not be able to use product search.
- Items marked as unavailable (available =
false) will not show up in the search results. - Be aware that in comparison to the Predict product search, the Visual CMS Product Search add-on will display unavailable products as well. This is possible to prepare emails with currently unavailable products in advance.
Here you can check if everything went well with your product data import.
This is something you should certainly do when you submit a GPF product data file, for example to check which GPF fields were kept during the conversion and how they are represented in the Emarsys product database.
Once you have also implemented our Web Extend JavaScript API, we will generate a product category tree on the basis of the category classification you provide for your products. You will be able to explore this category tree in the Category Explorer, which you can open by clicking the link in the upper right corner of the Current Product Data box.
Scheduling product data updates
A regular update schedule is needed so that Emarsys always has your most recent product data. For this, you will need to publish your latest product data file on a server from where we can fetch it.
- If you have not done it yet during the validation, paste the full URL of your product data file into the URL for scheduled updates field.
- Under the heading Schedule, specify how frequently you want us to update your product data.
- Select your timezone and save your settings.
We will update your product data at least once a day at about 22:00 in your local timezone by default. In the Schedule box, you can select a different hour or set up more frequent updates by selecting multiple hours at which the updates should occur.
Set up a reasonable update schedule that matches the dynamics of your web store. You may typically want us to update product data multiple times a day if some product attributes, such as availability or price, change frequently.
The actual updates will not necessarily happen every time at exactly the hour(s) you specify, as all scheduled updates are queued. For more information, see How long does it take for Predict to process my product catalog?
Past product data imports and their status are listed in the Update History table at the bottom of the page.
Forcing product data updates
Sometimes you may want to update your product data manually, for example when the latest scheduled update failed due to connection problems or when a scheduled upload is rejected.
In such cases, you can force a manual update by clicking Update from URL now and then clicking Activate update. This will instantly update your existing product data.
Product data updates are often rejected because the number of items in the new product data file is markedly lower than before. Before forcing an update in such cases, always make sure that the file’s contents are correct.
Remember that your Emarsys product database contains only the contents of the last update and is not cumulative. If you upload a file containing fewer items, only these will be known to Emarsys.
A forced manual update does not affect your upload schedule, and Emarsys will continue to try to update your product data as scheduled.
FAQ
How long does it take for Predict to process my product catalog?
Processing may involve several stages:
- Downloading the catalog from your server: this depends on the connection speed between your and the Emarsys servers.
- If the product catalog passes validations, it will be synchronized to our serving infrastructure. Depending on the time of day, there might be a processing queue, which means that it has to wait for other requests to be processed first, but generally, synchronization takes a few minutes. At this point, changes are reflected in all Web and Email recommendations.
- Additional data analysis takes place in batches, typically daily, that may affect product relationships in the recommendation models.
Important:
- In general, your product catalog file should not be larger than 10 GB. If you plan to upload a product catalog larger than 10 GB, please contact Emarsys Support.
- Emarsys supports .GZ file compression for CSV catalogs which also can save some bandwidth.
Troubleshooting
Some of my scheduled product data updates are not listed in the Update History table. Why?
Normally, every product data update attempt should be listed here, even those that fail. If an update is missing, it was probably skipped due to temporary network overload. We are continuously improving our infrastructure to prevent such scenarios, but if you run into them regularly, please contact our support team.