Updating External System Product Catalog
Updating External System Product Catalog#
Use Case Description#
This document outlines how an external system (e.g., an e-commerce website) can update its product catalog, including barcodes, prices and discounts, through API services. The process covers two primary use cases:1.
Initial Setup: Full catalog and product import.
2.
Subsequent Updates: Incremental updates to the catalog and product details.
Actors#
External System: The system that wants to update its product catalog and prices.
4ws.trade: Provides access to product and catalog data through the API services.
Product and Catalog Services: Backend systems that store and provide catalog and product data.
Preconditions#
The external system must have the necessary permissions and access to the services, please refer to Authentication. The catalog code must be known for updates, or the system must retrieve the catalog list first.
For updates, the system must store the date of the last successful synchronization, to be used as fromDate in the next incremental update. fromDate cannot be more than 24 hours old: if more time has passed, the external system must perform the initial setup again.
APIs Used#
1.
CATALOG_EXPORT: Retrieves catalog details.
PRODUCTS_CATALOG_EXPORT: Retrieves product details from the catalog.
2.
Get product prices: Retrieves the prices of a single product (sale price and discount), optionally filtered by price list and validity date. Get prices from price list: Retrieves all the prices of a price list, identified by price list type and code, with their validity period. use Get product prices to read the prices of a single product, for example right after it has been returned as updated by Get products by Catalog Code;
use Get prices from price list to download all the prices of a price list in a single paginated flow, with one request per page instead of one per product.
Main Success Scenario#
1.
The external system calls Get bulk CSV Export with CATALOG_EXPORT to obtain the catalog details. Once the catalog is known, it calls Get bulk CSV Export with PRODUCTS_CATALOG_EXPORT to retrieve the products of the catalog. The external system processes the data, updating its internal product catalog and prices.
The external system stores the date of the synchronization, to be used as fromDate for the subsequent updates.
2.
The external system calls Get products by Catalog Code with the fromDate parameter to get only the updated products, and returnDisabled=true to also receive the disabled products to remove. When the update completes successfully, the external system stores the new synchronization date.
Some services are paginated to handle large sets of data efficiently.
For detailed information on how to handle pagination, refer to our Paginated Responses Alternate Flows#
Catalog Code Not Known: If the catalog code is not known, the external system must first retrieve the list of available catalogs with Get bulk CSV Export using CATALOG_EXPORT. Last Synchronization Older than 24 Hours: Get products by Catalog Code does not accept a fromDate more than 24 hours old: the external system must perform the initial setup again.
Error Handling: If a request fails, the external system should retry the request and apply error handling mechanisms to prevent partial updates.
Sequence Diagrams#
Code Examples in Node.js#
Fetching Updated Products and Handling Pagination#
This example shows how to handle cursor-based pagination: while the response contains nextCursor, the next page is requested by passing it as cursor.Best Practices for CSV Processing#
1.
Use streaming to process large CSV files efficiently. Libraries like csv-parser in Node.js allow you to process rows one at a time without loading the entire file into memory.
Implement chunking mechanisms if the file is too large to process in one go (using a staging table is recommended).
2.
Unreadable Files: Implement retry logic for failed file downloads or unreadable files. Log errors for audit and debugging purposes.
Invalid Rows: If a specific row in the CSV is invalid, log the error but continue processing the remaining rows. Depending on the severity of the error, you may want to skip that row or mark it for manual review.
3.
Retries: Implement retry mechanisms with exponential backoff for failed API requests or file downloads.
Data Consistency: Ensure that updates are applied transactionally. If part of the file is processed but an error occurs, either roll back the changes or store the partial progress to resume processing later.
Logging and Monitoring: Track the progress of CSV processing and ensure that any issues (file corruption, invalid data) are logged and alerted to the system administrators.
Best Practices for Incremental Updates#
1.
Always store the date of the last successful synchronization: it is the fromDate value for the next request.
Update it only when the synchronization completes; keep track of the products that failed and retry them, so that they are not lost by moving fromDate forward.
Schedule incremental updates more often than every 24 hours, so that fromDate always stays within the accepted window.
2.
Implement robust logging for both success and failure states. Use monitoring tools to alert administrators to potential issues in file processing or data updates.
Use metrics like file size, processing time and error rates to identify performance bottlenecks.
Modified at 2026-10-27 14:05:55
