Data Import

This guide is for users looking to import various data sources into SmartFeeds and covers supported file formats, import methods, and troubleshooting common XML errors. It will be especially useful for those managing complex product feeds or using third-party data integration on SmartFeeds.

Table of Contents

1. Supported File Formats for Import
2. Import Methods
3. File Naming for Arcane FTP/SFTP
4. Basic Setup & Scheduling
5. Special Import Configurations
6. Troubleshooting XML Errors
7. Actions and Insights on Configured Imports
8. Quick Answers to Common Issues
9. Related Articles

Supported File Formats for Import

SmartFeeds supports the following file formats:

• CSV: Delimiters accepted include comma (,), semicolon (;), pipe (|), and tab. Tab-separated files can also use the .tsv extension.
• JSON: Only standard JSON files are supported (no NDJSON format).
• XML: RSS 2.0 is compatible. 
• • Custom root tags (e.g., <products> instead of <channel> or <item>) are allowed. 
  • Child tags are concatenated with their parent.
    Example: <price>  <currency>EUR</currency>  <value>12</value> </price>
    SmartFeeds Result: price_currency,price_value "EUR,12"
• Compressed Files: Gzip (.gz) and Zip (.zip) are supported.

Import Methods

To create an import, go to ProductFlow, select + New Data Source, and choose your method.

File Naming for Arcane FTP/SFTP

If you use the Arcane FTP/SFTP method, you must follow a specific naming convention for your files. This allows SmartFeeds to automatically pick the most recent data.

• The Rule: Use the pattern <Date>-<FileName>.<extension>.
• Date Format: Use 8 digits (yyyyMMdd) or 14 digits (yyyyMMddHHmmss) for better precision.
• How it works: At the scheduled execution time, SmartFeeds looks at all files in the folder and selects the one with the most recent timestamp in the name.

Example:

• Option 1 (Daily): 20250417-ClientExtract_dev.zip
• Option 2 (Precise): 20250417162102-ClientExtract_dev.zip (Year-Month-Day-Hour-Minute-Second)

Basic Setup & Scheduling

Every data import in ProductFlow or AudienceFlow requires three core configurations:

1. Title: Give your import a clear name.
2. Source: Provide the URL or credentials to connect your data.
3. Fetch Frequency: Decide when SmartFeeds retrieves your data.

Scheduling Your Data Fetch

SmartFeeds provides full control over when your data refreshes:

• Daily Occurrences: Pick specific hours and minutes for your updates. If you enter times out of order, the system will automatically reorganize them.
• Day Selection: Choose specific days of the week (e.g., Monday to Friday only) for your data fetches.
• Time Intervals: Set regular intervals (such as every 15 minutes, every hour, or other frequencies) for consistent updating throughout the day.
• License Guidance: The interface displays the maximum number of updates allowed based on your license. This ensures you stay within your plan's capabilities.

Example: A consultant wants to refresh stock data only during business hours on weekdays. They select "Monday to Friday" and set specific times at "09:00", "13:00", and "17:00".

Special Import Configurations

Some platforms, such as Shopify or PrestaShop require a custom setup.

• CMS Platforms: Select the appropriate platform icon in SmartFeeds. Follow the detailed step-by-step instructions provided in the interface.
• • Example: You click the Shopify logo and follow the automated connection wizard.
• XML & Compressed Files: Open Advanced Settings to enable the XML File Option or to choose the correct Compression Type (Zip/Gzip).
• • Example: You select "Gzip" so SmartFeeds can open your compressed .csv.gz file.

Troubleshooting XML Errors

Errors in XML feeds are often due to special characters like &, <, and >. Here’s how to resolve them:

1. Open the XML file URL in your browser.
2. Save the file locally (Ctrl+S).
3. Use a code editor to locate and correct the problematic line.
4. Managing Special Characters:
5. • Use <![CDATA[value]]> to escape special characters.
   • In the SQL editor, use CONCAT("<![CDATA[", VALUE, "]]>") to format values.

Actions and Insights on Configured Imports

For each configured import, you can:

• Check the last retrieval time and data volume.
• Perform actions such as:
• • Edit the import configuration.
  • Refresh manually.
  • Download or explore the data.

Advanced Configuration Tabs

Beyond basic setup, SmartFeeds offers advanced tabs to structure and monitor your data directly at the source:

• Detailed History: Enable the Detailed History switch to track the granular evolution of your catalog. This highlights exactly how many items were added, modified, or deleted compared to the previous day. Refer to the Detailed Import History article to learn more.
   
• Managing Field Types: Use the Headers tab to adjust the type of each field (e.g., Text, Integer, Decimal). Correct typing is critical for fields like price to ensure formulas work correctly later in the flow.
   
• Tracking Import Usage: The Use tab displays exactly where this import is currently active across your account. Refer to the Deletion article for guidance on removing unused files safely.
   
• Aggregation: Use the Aggregation tab to restructure your file before it enters the flow. This allows you to group rows or concatenate values (e.g., turning a list of orders into a single client view) without external pre-processing.
  Learn more about Aggregation and see how to use it in an audience segmentation use case article.

Quick Answers to Common Issues

What should I do if my file format is unsupported?

Contact SmartFeeds support to see if a solution is available.

Can I manually update an import?

Yes, click the update arrow next to the data source to refresh it.

How do I configure alert settings for imports?

Go to Edit > Import Alerts to adjust notification preferences. Find more information on Product Flow Alerts

Is it possible to preprocess source files before uploading them to SmartFeeds?

Yes, we have the ability to preprocess your source files before uploading them to SmartFeeds. This requires the involvement of our Tech team and may incur additional licensing fees, depending on the complexity of the task.

Use Cases:

• Cross-referencing files
• Removing duplicates
• [Other specific tasks as needed]

If you have specific preprocessing requirements, please contact us for further details and a customized quote.

Related Articles

For further assistance with SmartFeeds, consider reviewing these articles:

• Data Export
• Deletion
• Product Flow Alerts

For additional help, feel free to reach out via our Contact Us page.