Logo Help Portal
  • Getting Started

    Introduction

    • Onboarding overview
    • Project Initiation
    • Email compliance
    • Account Administration

    Data

    • Data exchange
    • Data onboarding
    • Web data collection

    Channels

    • Email onboarding
    • Smart Insight
    • Predict
  • Training
    • Online Self Learning
    • Onboarding Videos
    • Instructor Led Training
    • Webinars
    • Seminars
  • User Guides

    Strategy

    • Personalization
    • Automation
    • Data Monitoring

    Channels

    • Email
    • Mobile Apps
    • Web
    • Ads

    Add-ons

    • Smart Insight
    • Predict
    • Incentive Recommendation
    • AIM
  • News & Support

    News

    • What’s New in the Emarsys Platform
    • Pilot features
    • Articles on Data Protection and GDPR
    • Release notes - archive

    Support

    • Help and Support at Emarsys
    • Managing your user profile
  • |
  • Developers
  • System Status
Need help? Submit a request
English Deutsch Español Français Русский Türkçe 简体中文
Sign in
  • Getting Started
  • Preparing for Integration
  • 0 Getting Started Preparing for Integration

In this section:

  • Data exchange resources
  • Strategic Platform Integrations
  • Preparing your product data file
  • Preparing your sales data file
  • Using the Google Product Feed format
  • Migrating your first email campaigns
  • Preparing your account assets
  • Migrating your brand assets
  • Introduction to the Emarsys API
  • in Emarsys42

    Expand all

    Preparing your product data file

    Updated: February 01, 2019 15:31

    Keeping your product data consistent between Emarsys and your webshop is essential to ensure that the content of your messages and their recommendations matches what is actually available online.

    Please read this article before you set up your regular data synchronization schedule, and make sure that your product data file is formatted according to the specifications and guidelines described below.

    Important: Your product data file always replaces the entire product catalog each time it is imported. All data is overwritten, rather than updated, and if products are left out of the file, they will no longer be available in the Emarsys product database.

    Contents

    • Before you start
      • What is product data?
      • Why is product data important?
      • How much effort do I need to invest in this?
      • Who do I need to involve from my side?
      • What should I include in my product data file?
      • How often do I need to upload product data?
    • General rules for your data file
    • Product categories
    • Standard product data fields
    • Other predefined fields
    • Custom fields
    • Field localization
    • Disambiguation
    • Resizing images with the Emarsys image proxy
    • Examples

    Before you start

    What is product data?

    Product data is all the information on all the products you sell, and have ever sold, on your website. It is usually kept in one file (also known as the product catalog) on a server hosted by you and synchronized daily with Emarsys.

    Your product data file can be monitored and modified on the Admin > Data Sources > Product Data page. 

    This is how the elements of a product page would translate into Emarsys product data fields.

    Why is product data important?

    Your product data is required by most of the Emarsys products and features, including Predict and Incentive Recommendations, and Smart Insight.

    Accurate product data enables you to target specific customer segments on the basis of which products they bought, viewed or left in their shopping carts, or send them dynamic, tailor-made product recommendations.

    How much effort do I need to invest in this?

    This depends on the product data file format you are going to use. We support two formats: 

    • Emarsys CSV format
    • Google Product Feed (GPF)

    If you use the Emarsys CSV format you will need to generate an output file that conforms to our specifications, described below and in our CSV guidelines. Depending on the size and complexity of your product catalog, it may take up to a few hours for your data specialist to deliver this file. Then, if our validation engine detects some issues, you will need to fix these first, which may take another couple of hours of your data specialist’s time.

    Here is an estimate of how much time you may require to set up your product data file:

    Action
    Role Max effort
    Generating product data Data specialist Up to 6 hours
    Validating your product data file Data specialist 5 mins
    Fixing product file, if required Data specialist Depends on the number and severity of issues, but usually no more than 2 hours
    Scheduling regular updates Marketer/Data specialist 5 mins

    If you already have a Google Product Feed (GPF), you may be able to skip these steps as your GFP could be readily usable in Emarsys. However, make sure that your GPF contains all the required fields. If you are unsure, just contact Emarsys Support and we will check your file.

    Who do I need to involve from my side?

    • If you use the Emarsys CSV format, you will need a data specialist to provide the product data and, optionally, a product owner to oversee the project.
    • If you reuse your Google Product Feed, all you may need is a link to your GPF file, which is something your data specialist should know and be able to provide.

    What should I include in my product data file?

    We strongly recommend to include all the products that have ever been sold on your website, even if they are no longer available (simply set the corresponding value in the available field).

    Including past products and categories will help to build a more accurate and robust affinity model from your web behavior, and will reduce the risk of your product data becoming corrupted or inconsistent.

    Orders also typically contain a number of service items such as delivery costs and method, packaging, free gifts or promotions, etc. Emarsys will not be able to distinguish between these and actual products in the order, so you should make sure that they are all listed in your product data file as unavailable products. This will prevent them from ever being offered as recommendations.

    How often do I need to upload product data?

    We recommend to upload your product data at least once a day. You may want to consider doing it more frequently, if important fields such as price or availability are likely to change during the day.

    General rules for your data file

    • Your product data file should be a simple .csv table, with each row containing all the attributes of a given product, such as the item ID, the URL of the product page, a link to a product image, etc.
    • Use a CSV export library that is compatible with the standard CSV format.
    • Use a comma to separate fields in a row.
    • In fields containing multiple values, use a pipe (|) to separate the values. Example: …,John Lennon|Paul McCartney|George Harrison|Ringo Starr,...
    • When a field does not have any value, leave it empty. Example: value1,,value3,value4
    • Do not use newline characters, either unquoted or quoted, in the fields.
    • Use UTF-8 encoding.
    • Include the field names in the first row (header) and make sure that all the subsequent rows contain the values corresponding to the fields listed in the header.
    • Do not use more than 255 columns.
    • Do not use field values longer than 64 KB.
    • Redundant (starting and trailing) white spaces are automatically trimmed unless they are quoted (e.g. 1, 2, 3 is the same as 1,2,3).
    • Even though your product data file normally contains information that is publicly available on your website, you may want to add a level of security by password-protecting it. Emarsys fully supports uploading password-protected product data files over a secure connection.

    Need an example? Click here to see how a product data file looks like in the Emarsys CSV format or download a sample file.

    File size

    • If your product data file contains more than a million items, please contact Emarsys Support before importing it.
    • You can upload gzip compressed product data files to Emarsys. If your file runs to several hundred megabytes in size, you may want to consider compressing it.

    Product categories

    In the category field, you submit the product category path of your items according to the product categorization system of your online store. 

    Please pay particular attention to the consistency of product categories in your web shop and in your product data file. 

    A common cause of errors (and therefore delays) when onboarding product data is that the product data file does not have the same understanding of categories as the Web Extend data collection script category command.

    A category path describes the location of each product in the category hierarchy. It is a >-separated list of subcategories that starts with the top-level category.

    For example, a small bicycle store may have the following category hierarchy:

    Bikes
       Road Bikes
       Mountain Bikes
     Parts
       Road
         Tyres
         Chains
       MTB
         Tyres
         Chains

    In this structure, Bikes and Bikes > Road Bikes are both valid category paths, while Road Bikes in itself is not, since it is not a top-level category.

    The category path for bike1, an awesome road bike, is Bikes > Road Bikes, which can be submitted in your product data file like this:

    item, title, category,...
    bike1, Awesome Road Bike, Bikes > Road Bikes, …

    Emarsys Predict supports an unlimited number of category levels, while Smart Insight can handle a maximum of five levels. If there are more than 5 levels, Smart Insight keeps the top 5 and ignores the rest.

    Items with multiple categories

    Sometimes you may want to categorize products into multiple categories. For example, our bike shop sells pedal1, a universal hybrid pedal, which works on both road bikes and mountain bikes. As such, it belongs to both Parts > Road > Pedals and Parts > MTB > Pedals, so pedal1 belongs to a list of categories. The elements of lists should be separated by a | (pipe) within the category field.

    item, title, category, ...
    pedal1, Universal Hybrid Pedal, Parts > Road > Pedals | Parts > MTB > Pedals,...

    Standard product data fields

    Field
    Required Type Example Description
    item
    Yes String BOOK0012 - The unique ID used in your web store to identify the product.
    - Do not include SKU-level product variants in the product data file. A good rule of thumb is to consider an item unique if it has a unique URL in your online store.
    title
    Yes String The Hitchhikers guide - The name of the product.
    - Do not include any promotional text.
    link
    Yes URL http://www.buy.me/hitch/ - The URL of the items product page.
    - Start with the protocol, such as http or https.
    - Do not encode the URL.
    image
    No URL http://www.buy.me/hitch_thumb.jpg - The URL for the product image.
    - Start with the protocol, such as http or https.
    - Our best practice recommendation is to use a 800-pixel wide image to ensure that it looks good on all displays.
    - Do not use images larger than 1 MB or 1280 x 1280 px.
    - For high-resolution product images, use the zoom_image field.
    - If you only have high-res product images, we can create the thumbnails for you automatically. Learn more.
    zoom_image
    No URL http://www.buy.me/hitch_fullres.jpg - The URL for the high-resolution product image.
    - Start with the protocol, such as http or https.
    category*
    No Click here for details Fiction > Sci-Fi | English literature > Sci-Fi The list of category paths in which the given item belongs.
    available
    No Boolean true - The current availability of the product in your store.
    - Use either lower case (true) or all-caps (TRUE).
    - Make sure to keep this field up to date.
    - If possible, include all the items you have ever offered on your website. This will improve the analysis of historical data and makes it possible for Emarsys to recommend available alternatives for items no longer for sale.
    - Submit products currently unavailable as false.
    description
    No String Douglas Adams' seminal work of fiction. A brief description of the product.
    price
    No Float 13.99 - The current product price you display to your customers in your web store.
    - Do not include the currency.
    - Always use the period (.) as the decimal mark.
    msrp
    No Float 17.99 - The original price of the product (i.e. the price before discounts).
    - msrp must be higher than price. If msrp is equal to it or lower, it will not be shown in email recommendations.
    - Do not include the currency.
    - Always use the period (.) as the decimal mark.

    * In this field, you can use multiple values.

    Other predefined fields

    Field
    Required Type Example Description
    album*
    No String Let it be The title of the album(s) on which the product was released.
    actor*
    No String Brad Pitt The actor(s) associated with the product.
    artist*
    No String Janis Joplin The artist(s) associated with the product.
    author*
    No String Douglas Adams The author(s) associated with the product.
    brand*
    No String Nike The brand name(s) of the product.
    year
    No Integer 1984 The publication year of the product.

    * In these fields, you can use multiple values.

    Custom fields

    Besides the standard and predefined product data fields, you can also add other product information to your data file by using custom fields. The names of custom fields has to begin with the c_* prefix (e.g. c_color or c_size).

    When deciding on custom fields, think about the kind of segmentation you would like to have available in your Emarsys account, then work back to identify the fields that you need in order to achieve those goals. 

    This is particularly relevant if you are using Smart Insight, as these fields will enable you to create segments based on product attributes.

    Make sure not to include any of the predefined field names in any part of the names of custom fields. An example of a custom field name to avoid is c_available_colors as available is a standard field.

    Custom fields are not validated by Emarsys.

    Field localization

    If you run a web store in multiple markets or languages, it may be necessary to also include the localized versions of standard product data fields in your product data file. Name your localized fields using the pattern [field name]_[language identifier], that is, the name of the standard product data field must be followed by an underscore and the language identifier (for example, title_it for the Italian title of a book).

    All standard and predefined product data fields can be localized, except for item.

    To learn more about localization, see our article on Localized Websites.

    Disambiguation

    Since several product attributes, such as categories, authors and actors, can be used to build statistical models, it is important to use them canonically. For example, the author J. K. Rowling should always be spelled the exact same way in the product data file. When there are more identical strings representing different values of a given attribute, they should be disambiguated consistently, for example J. Doe and J. Doe(2).

    Resizing images with the Emarsys image proxy

    Certain Emarsys apps, such as product recommendations, work properly with smaller images. If you only have high-resolution product images, Emarsys can create their thumbnail versions for you. To use this automatic image resizing service, simply replace the image URL with the following URL pattern in the image field:

    https://imgfly.scarabresearch.com/w_<thumbnail image width in pixels>/<image URL>

    Make sure to start the URL of the original image with the protocol, which can be either http or https, and to use the standard URL syntax.

    Our best practice recommendation is to use 800-pixel wide images to ensure that they look good on all displays.

    Examples

    This is how a product data file looks like in the Emarsys CSV format. Click here to download a sample file.

    item,link,title,image,category,price,msrp,available,brand,description,zoom_image
    103,http://www.lifestylelabels.com/steven-by-steve-madden-pryme-pump.html,"elegant red pump",http://www.lifestylelabels.com/media/catalog/product/cache/1/small_image/210x/9df78eab33525d08d6e5fb8d27136e95/s/t/steven-by-steve-madden-pryme-pump.jpg,"Shoes > Pump",299,0,true,"Steve Madden","Nothing will turn his head faster than you wearing the sexy Pryme pump from Steven by Steve Madden. This daring pump has a pretty patent leather upper with light shirring, a double stitch detail surrounding the collar, and a vampy almond shaped toe.",http://www.lifestylelabels.com/media/catalog/product/s/t/steven-by-steve-madden-pryme-pump.jpg
    108,http://www.lifestylelabels.com/nine-west-women-s-lucero-pump.html,"glittering red pump",http://www.lifestylelabels.com/media/catalog/product/cache/1/small_image/210x/9df78eab33525d08d6e5fb8d27136e95/n/i/nine-west-women-s-lucero-pump.jpg,"Shoes > Pump",289.99,0,true,"Nine West","The Lucero pump from Nine West may just leave him at a loss for words. This flirty pump has a leather upper, a pretty almond-shaped toe with a slight V-cut vamp, leather linings, and a cushioned insole for long-wearing comfort.",http://www.lifestylelabels.com/media/catalog/product/n/i/nine-west-women-s-lucero-pump.jpg
    112,http://www.lifestylelabels.com/womens-golf-shoes.html,"womens golf shoes",http://www.lifestylelabels.com/media/catalog/product/e/c/ecco-womens-golf-flexor-golf-shoe.jpg,"For Women > Shoes > Golf",159,0,false,"Ecco","Sporting with style, this is a durable and super-comfortable golf shoe built for performance.",http://www.lifestylelabels.com/media/catalog/product/e/c/ecco-womens-golf-flexor-golf-shoe.jpg

    The above file without white spaces: 

    item,link,title,image,category,price,msrp,available,brand,description,zoom_image
    103,http://www.lifestylelabels.com/steven-by-steve-madden-pryme-pump.html,"elegant red pump",http://www.lifestylelabels.com/media/catalog/product/cache/1/small_image/210x/9df78eab33525d08d6e5fb8d27136e95/s/t/steven-by-steve-madden-pryme-pump.jpg,"Shoes>Pump",299,0,true,"Steve Madden","Nothing will turn his head faster than you wearing the sexy Pryme pump from Steven by Steve Madden. This daring pump has a pretty patent leather upper with light shirring, a double stitch detail surrounding the collar, and a vampy almond shaped toe.",http://www.lifestylelabels.com/media/catalog/product/s/t/steven-by-steve-madden-pryme-pump.jpg
    108,http://www.lifestylelabels.com/nine-west-women-s-lucero-pump.html,"glittering red pump",http://www.lifestylelabels.com/media/catalog/product/cache/1/small_image/210x/9df78eab33525d08d6e5fb8d27136e95/n/i/nine-west-women-s-lucero-pump.jpg,"Shoes>Pump",289.99,0,true,"Nine West","The Lucero pump from Nine West may just leave him at a loss for words. This flirty pump has a leather upper, a pretty almond-shaped toe with a slight V-cut vamp, leather linings, and a cushioned insole for long-wearing comfort.",http://www.lifestylelabels.com/media/catalog/product/n/i/nine-west-women-s-lucero-pump.jpg
    112,http://www.lifestylelabels.com/womens-golf-shoes.html,"womens golf shoes",http://www.lifestylelabels.com/media/catalog/product/e/c/ecco-womens-golf-flexor-golf-shoe.jpg,"For Women>Shoes>Golf",159,0,false,"Ecco","Sporting with style, this is a durable and super-comfortable golf shoe built for performance.",http://www.lifestylelabels.com/media/catalog/product/e/c/ecco-womens-golf-flexor-golf-shoe.jpg

    Note that Emarsys considers both files exactly the same as redundant white spaces and redundant quoting are ignored. Also note that values containing the separator character must be enclosed in double-quotes (e.g. "Foo, Bar").

    Was this article helpful?

    Have more questions? Submit a request
    Return to top

    You may also be interested in:

    Related articles

    • Uploading your product data
    • Data collection JavaScript API reference
    • Uploading your sales data
    • Implementing the Web Extend data collection scripts
    • Predict product categories
    Copyright © 2019 Emarsys eMarketing Systems. All rights reserved
    Legal Notice Privacy Policy Master Services Agreement Anti-spam Policy
    test new search