{"id":65,"date":"2025-01-24T10:56:35","date_gmt":"2025-01-24T10:56:35","guid":{"rendered":"https:\/\/magendoo.ro\/insights\/?p=65"},"modified":"2025-01-26T17:42:14","modified_gmt":"2025-01-26T17:42:14","slug":"product-import-strategies-in-magento-2-part-4-rest-api-and-microservice-integration","status":"publish","type":"post","link":"https:\/\/magendoo.ro\/insights\/product-import-strategies-in-magento-2-part-4-rest-api-and-microservice-integration\/","title":{"rendered":"Product Import Strategies in Magento 2 – Part 4: REST API and Microservice Integration"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

In this final installment of our series, we’ll explore implementing product imports using Magento 2’s REST API with a microservice architecture. This approach provides real-time feedback, granular control, and the ability to handle complex product relationships.<\/p>\n\n\n\n\n\n\n\n

System Architecture<\/h2>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Implementation<\/h2>\n\n\n\n

1. Microservice Core<\/h3>\n\n\n\n
\/\/ Product Import Service (Node.js\/TypeScript)\nimport axios, { AxiosInstance } from 'axios';\nimport { Queue } from '.\/queue';\nimport { ImageProcessor } from '.\/image-processor';\n\ninterface MagentoConfig {\n    baseUrl: string;\n    token: string;\n    batchSize: number;\n}\n\nclass ProductImportService {\n    private readonly client: AxiosInstance;\n    private readonly queue: Queue;\n    private readonly imageProcessor: ImageProcessor;\n\n    constructor(config: MagentoConfig) {\n        this.client = axios.create({\n            baseURL: config.baseUrl,\n            headers: {\n                'Authorization': `Bearer ${config.token}`,\n                'Content-Type': 'application\/json'\n            }\n        });\n        this.queue = new Queue(config.batchSize);\n        this.imageProcessor = new ImageProcessor();\n    }\n\n    async importProducts(products: Product[]): Promise<ImportResult> {\n        try {\n            \/\/ Queue products in batches\n            await this.queue.addBatch(products);\n\n            \/\/ Process queue\n            while (!this.queue.isEmpty()) {\n                const batch = await this.queue.getNext();\n                await this.processBatch(batch);\n            }\n\n            return this.generateReport();\n        } catch (error) {\n            this.handleError(error);\n        }\n    }\n\n    private async processBatch(products: Product[]): Promise<void> {\n        for (const product of products) {\n            try {\n                \/\/ Process images first\n                const processedImages = await this.imageProcessor.process(product.images);\n\n                \/\/ Prepare product data with processed image URLs\n                const productData = this.prepareProductData(product, processedImages);\n\n                \/\/ Check if product exists\n                const exists = await this.checkProductExists(product.sku);\n\n                if (exists) {\n                    await this.updateProduct(product.sku, productData);\n                } else {\n                    await this.createProduct(productData);\n                }\n            } catch (error) {\n                this.logError(product.sku, error);\n            }\n        }\n    }\n\n    private async createProduct(productData: any): Promise<void> {\n        await this.client.post('\/rest\/V1\/products', {\n            product: productData\n        });\n    }\n\n    private async updateProduct(sku: string, productData: any): Promise<void> {\n        await this.client.put(`\/rest\/V1\/products\/${sku}`, {\n            product: productData\n        });\n    }\n}<\/code><\/pre>\n\n\n\n
2. Product Data Transformer<\/h3>\n\n\n\n
class ProductTransformer {\n    transformForMagento(rawProduct: RawProduct): MagentoProduct {\n        return {\n            sku: rawProduct.sku,\n            name: rawProduct.name,\n            price: rawProduct.price,\n            status: 1,\n            visibility: 4,\n            type_id: 'simple',\n            attribute_set_id: 4,\n            extension_attributes: this.getExtensionAttributes(rawProduct),\n            custom_attributes: this.getCustomAttributes(rawProduct)\n        };\n    }\n\n    private getExtensionAttributes(product: RawProduct): any {\n        return {\n            website_ids: [1],\n            category_links: this.getCategoryLinks(product.categories),\n            stock_item: {\n                qty: product.quantity,\n                is_in_stock: product.quantity > 0\n            }\n        };\n    }\n\n    private getCustomAttributes(product: RawProduct): any[] {\n        const attributes = [];\n\n        \/\/ Handle each custom attribute\n        if (product.description) {\n            attributes.push({\n                attribute_code: 'description',\n                value: product.description\n            });\n        }\n\n        \/\/ Add more custom attributes as needed\n        return attributes;\n    }\n}<\/code><\/pre>\n\n\n\n
3. Image Processing Service<\/h3>\n\n\n\n
class ImageProcessor {\n    private readonly allowedTypes = ['jpg', 'jpeg', 'png', 'webp'];\n    private readonly maxSize = 2000; \/\/ pixels\n\n    async process(images: ProductImage[]): Promise<ProcessedImage[]> {\n        const processed: ProcessedImage[] = [];\n\n        for (const image of images) {\n            try {\n                \/\/ Download image\n                const buffer = await this.downloadImage(image.url);\n\n                \/\/ Validate image\n                await this.validateImage(buffer);\n\n                \/\/ Process image\n                const optimized = await this.optimize(buffer);\n\n                \/\/ Upload to storage\n                const url = await this.upload(optimized, image.label);\n\n                processed.push({\n                    original: image.url,\n                    processed: url,\n                    label: image.label,\n                    position: image.position,\n                    types: image.types || ['image']\n                });\n            } catch (error) {\n                this.logImageError(image.url, error);\n            }\n        }\n\n        return processed;\n    }\n\n    private async optimize(buffer: Buffer): Promise<Buffer> {\n        \/\/ Implement image optimization logic\n        \/\/ Example: using sharp for image processing\n        return sharp(buffer)\n            .resize(this.maxSize, this.maxSize, {\n                fit: 'inside',\n                withoutEnlargement: true\n            })\n            .webp({ quality: 85 })\n            .toBuffer();\n    }\n}<\/code><\/pre>\n\n\n\n
4. Queue Management<\/h3>\n\n\n\n
class ImportQueue {\n    private readonly batchSize: number;\n    private queue: Product[][];\n    private processing: boolean;\n\n    constructor(batchSize: number) {\n        this.batchSize = batchSize;\n        this.queue = [];\n        this.processing = false;\n    }\n\n    async addBatch(products: Product[]): Promise<void> {\n        const batches = this.createBatches(products);\n        this.queue.push(...batches);\n    }\n\n    private createBatches(products: Product[]): Product[][] {\n        const batches: Product[][] = [];\n\n        for (let i = 0; i < products.length; i += this.batchSize) {\n            batches.push(products.slice(i, i + this.batchSize));\n        }\n\n        return batches;\n    }\n\n    async processQueue(handler: (batch: Product[]) => Promise<void>): Promise<void> {\n        if (this.processing) {\n            return;\n        }\n\n        this.processing = true;\n\n        try {\n            while (this.queue.length > 0) {\n                const batch = this.queue.shift();\n                if (batch) {\n                    await handler(batch);\n                }\n            }\n        } finally {\n            this.processing = false;\n        }\n    }\n}<\/code><\/pre>\n\n\n\n
API Integration Patterns<\/h2>\n\n\n\n
1. Error Handling and Retry Logic<\/h3>\n\n\n\n
class RetryManager {\n    private readonly maxRetries: number;\n    private readonly backoffMs: number;\n\n    constructor(maxRetries: number = 3, backoffMs: number = 1000) {\n        this.maxRetries = maxRetries;\n        this.backoffMs = backoffMs;\n    }\n\n    async retry<T>(operation: () => Promise<T>): Promise<T> {\n        let lastError: Error;\n\n        for (let attempt = 1; attempt <= this.maxRetries; attempt++) {\n            try {\n                return await operation();\n            } catch (error) {\n                lastError = error;\n\n                if (!this.isRetryable(error) || attempt === this.maxRetries) {\n                    throw error;\n                }\n\n                await this.delay(attempt);\n            }\n        }\n\n        throw lastError;\n    }\n\n    private isRetryable(error: any): boolean {\n        \/\/ Define retry conditions\n        return error.response?.status >= 500 || \n               error.code === 'ECONNRESET' ||\n               error.code === 'ETIMEDOUT';\n    }\n}<\/code><\/pre>\n\n\n\n
2. Bulk Operations<\/h3>\n\n\n\n
class BulkOperationManager {\n    private readonly client: AxiosInstance;\n\n    async executeBulkOperation(operations: Operation[]): Promise<BulkResult> {\n        const bulkRequest = {\n            operations: operations.map(op => ({\n                entity: op.entity,\n                action: op.action,\n                data: op.data\n            }))\n        };\n\n        const response = await this.client.post('\/rest\/async\/bulk\/V1\/products', bulkRequest);\n        return this.trackBulkOperation(response.data.bulk_uuid);\n    }\n\n    private async trackBulkOperation(uuid: string): Promise<BulkResult> {\n        while (true) {\n            const status = await this.getBulkStatus(uuid);\n\n            if (status.is_complete) {\n                return this.processBulkResult(status);\n            }\n\n            await new Promise(resolve => setTimeout(resolve, 5000));\n        }\n    }\n}<\/code><\/pre>\n\n\n\n
Monitoring and Logging<\/h2>\n\n\n\n
class ImportMonitor {\n    private readonly metrics: MetricsClient;\n    private readonly logger: Logger;\n\n    recordMetrics(result: ImportResult): void {\n        this.metrics.gauge('import.products.total', result.total);\n        this.metrics.gauge('import.products.success', result.success);\n        this.metrics.gauge('import.products.failed', result.failed);\n        this.metrics.timing('import.duration', result.duration);\n    }\n\n    async monitorHealth(): Promise<void> {\n        const metrics = await this.gatherMetrics();\n\n        if (metrics.errorRate > this.threshold) {\n            await this.alertTeam({\n                type: 'error_rate',\n                metrics: metrics,\n                timestamp: new Date()\n            });\n        }\n    }\n}<\/code><\/pre>\n\n\n\n
Best Practices<\/h2>\n\n\n\n
1. API Usage<\/h3>\n\n\n\n
    \n
  • Implement rate limiting<\/li>\n\n\n\n
  • Use bulk operations when possible<\/li>\n\n\n\n
  • Cache API responses<\/li>\n\n\n\n
  • Handle token expiration<\/li>\n<\/ul>\n\n\n\n
    2. Error Handling<\/h3>\n\n\n\n
      \n
    • Implement circuit breakers<\/li>\n\n\n\n
    • Use exponential backoff<\/li>\n\n\n\n
    • Log detailed error information<\/li>\n\n\n\n
    • Handle partial success scenarios<\/li>\n<\/ul>\n\n\n\n
      3. Performance<\/h3>\n\n\n\n
        \n
      • Process images asynchronously<\/li>\n\n\n\n
      • Use connection pooling<\/li>\n\n\n\n
      • Implement request batching<\/li>\n\n\n\n
      • Cache frequently accessed data<\/li>\n<\/ul>\n\n\n\n
        Advantages<\/h2>\n\n\n\n
          \n
        1. Real-time Processing<\/li>\n<\/ol>\n\n\n\n
            \n
          • Immediate feedback<\/li>\n\n\n\n
          • Error handling<\/li>\n\n\n\n
          • Progress tracking<\/li>\n<\/ul>\n\n\n\n
              \n
            1. Scalability<\/li>\n<\/ol>\n\n\n\n
                \n
              • Independent scaling<\/li>\n\n\n\n
              • Resource optimization<\/li>\n\n\n\n
              • Load distribution<\/li>\n<\/ul>\n\n\n\n
                  \n
                1. Flexibility<\/li>\n<\/ol>\n\n\n\n
                    \n
                  • Custom business logic<\/li>\n\n\n\n
                  • Multiple data sources<\/li>\n\n\n\n
                  • Easy integration<\/li>\n<\/ul>\n\n\n\n
                    Limitations<\/h2>\n\n\n\n
                      \n
                    1. Complexity<\/li>\n<\/ol>\n\n\n\n
                        \n
                      • API maintenance<\/li>\n\n\n\n
                      • Token management<\/li>\n\n\n\n
                      • Version compatibility<\/li>\n<\/ul>\n\n\n\n
                          \n
                        1. Performance<\/li>\n<\/ol>\n\n\n\n
                            \n
                          • API rate limits<\/li>\n\n\n\n
                          • Network latency<\/li>\n\n\n\n
                          • Resource constraints<\/li>\n<\/ul>\n\n\n\n
                            When to Use This Approach<\/h2>\n\n\n\n
                            This approach is ideal when:<\/p>\n\n\n\n
                              \n
                            • Real-time processing is required<\/li>\n\n\n\n
                            • Complex validation rules exist<\/li>\n\n\n\n
                            • Integration with multiple systems is needed<\/li>\n\n\n\n
                            • Custom business logic is required<\/li>\n<\/ul>\n\n\n\n
                              Conclusion<\/h2>\n\n\n\n
                              The REST API with microservice approach offers:<\/p>\n\n\n\n
                                \n
                              • Real-time processing capabilities<\/li>\n\n\n\n
                              • Granular control over the import process<\/li>\n\n\n\n
                              • Flexibility in handling complex scenarios<\/li>\n\n\n\n
                              • Scalability for large operations<\/li>\n<\/ul>\n\n\n\n
                                Choose this approach when you need tight integration with Magento 2 and want to maintain control over the entire import process.<\/p>\n\n\n\n
                                \n\n\n\n
                                This concludes our series on Product Import Strategies in Magento 2. Each approach has its strengths, and the choice depends on your specific requirements and constraints.<\/p>\n\n\n\n
                                magento2 #api #microservices #ecommerce #nodejs #typescript #programming<\/h1>\n","protected":false},"excerpt":{"rendered":"
                                Introduction In this final installment of our series, we’ll explore implementing product imports using Magento 2’s REST API with a microservice architecture. This approach provides real-time feedback, granular control, and the ability to handle complex product relationships.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-container-style":"default","site-container-layout":"default","site-sidebar-layout":"default","disable-article-header":"default","disable-site-header":"default","disable-site-footer":"default","disable-content-area-spacing":"default","footnotes":""},"categories":[1,9],"tags":[],"class_list":["post-65","post","type-post","status-publish","format-standard","hentry","category-general","category-magento-2"],"_links":{"self":[{"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/posts\/65","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/comments?post=65"}],"version-history":[{"count":3,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/posts\/65\/revisions"}],"predecessor-version":[{"id":72,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/posts\/65\/revisions\/72"}],"wp:attachment":[{"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/media?parent=65"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/categories?post=65"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/magendoo.ro\/insights\/wp-json\/wp\/v2\/tags?post=65"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}