The problem with CSV import plugins

The quickest route to linking a web store to a supplier is with an e-commerce plugin. They are installed into your store and usually need CSV data files from your supplier. Some have the ability to grab data from FTP, run on a regular schedule, and do other basic things.

Unfortunately they won’t grow with your business.

Plugins are meant to cover the lowest-common-denominator. They can do the basics but then will hit a wall as soon as you need to create a custom pricing rule, or change the product meta-data, or simply change how the inventory is updated. Plugins are created for the purpose of selling in volume and therefore the developers who create them are not interested in your custom requirements. However, any substantial e-commerce store will need customization.

Some of the more common questions I see are:

– What about my ERP system? I want to relay WooCommerce orders back to my supplier and they need them uploaded via FTP or with a web service.

– My supplier doesn’t use CSV. It is XML format. How can I load this into my Shopify store?

– How can I get my supplier’s shipping data back into my PrestaShop store so that my customers can use the tracking number.

– My supplier’s data feed doesn’t provide variants but I need them to show different t-shirt sizes. How can I fix this?

– Can you import products into my store categories for me? They don’t match my supplier but I can tell you which ones to map.

These are all exceptions which we handle on a daily basis at Data Here-to-There. We can adapt our innovative platform to any supplier or online service.

It’s surprising how many data feeds out there have low-quality product information. When I first got started integrating supply feeds for my clients I was shocked at how bad the data was. These are, after all, companies which *want* to sell their products. Yet they have terse (or missing) product descriptions, missing images, and so on. Data Here-to-There can actually enhance data feeds – create variants, add tagging, automatically find product images using Google, or even web-scrape entire product catalogs if your supplier doesn’t have a data feed!

Plugins are a great tool to get started with and can be sufficient for a simple single store, single supplier setup. But to grow your e-commerce empire you’ll need customized automation that fits your business strategy.

How to make a great data feed

If you are a drop-shipper, you need a data feed.

A Data Feed provides a catalog of products that merchants can import into their store. Maybe it’s a collection of kids toys with the SKU, title, description, price, and other information that the store owner can use to entice potential buyers. Data feeds are a critical component of any e-commerce supply chain. So how do you make a data feed?

To get started you need a list of products. Perhaps this is simply an Excel spreadsheet, or maybe it is stored in your QuickBooks system or in an inventory management database. The first step is to decide which products can actually be drop-shipped. There might be products like gift cards or rare products that you don’t want to supply. So those need to be filered. Now we create key information for the vendors. We need the following information.

SKU – stock keeping unit. This is an unique identifier which will be used when placing orders.
Title – The name of the product.
Description – A friendly explanation of the product which commonly includes HTML formatting.
Cost – The cost to your vendor, they will mark it up based on this number.
MSRP – Manufacturer’s Suggested Retail Price. This is higher than cost and used to indicate sale prices.
MAP – Minimum Advertised Price. Sometimes you want to set a lower limit on the price. Vendors can’t sell below it.
Stock – How many units are available to be sold? Vendors use this to prevent oversells in their store.
Manufacturer – The brand name of the product.
UPC – Sometimes referred to as GTIN, UPC codes are required for most major marketplaces.
Weight/Height/Length/Width – Determines shipping costs
Images – Pictures showing the product from different angles. These are stored separately.

The above data is required by most web stores and marketplaces so it’s good to provide as much as possible. It’s also important not to have empty values in any of these fields because it can break their store. Missing prices, titles, or other data causes a lot of grief because the vendors won’t be able to list them.

The most common data feed is simply a set of CSV (Comma Separated Values) files that vendors can download. CSV files are very easy to create, in fact you can make them using Excel. If you have a large inventory it makes sense to split your data feed into multiple CSV files. Especially the stock numbers – these fluctuate the most! So if you have a stock CSV file that will allow your vendors to grab it quickly and update their stores frequently without having to download *everything*. It saves your bandwidth costs too.

The Cadillac of data feeds doesn’t stop here. We can create a CSV file for product variants as well. This allows us to group collections of similar products. For example, a watch which can have a blue, green, or red background. It’s the same watch just with different colors. So we call these ‘variants’. The vendor has to be told how to connect up these variants though so this data is critical. Otherwise they have to list each watch as a separate product and that looks bad.

It’s also a great idea to create attributes as well. Descriptive tags help vendors sell products because tags can be searched in their store. For example a book seller can create an author tag, number of pages tag, an ISBN tag and so on. This data can be imported and allows customers to search by author, number of pages, etc. What a great way to help vendors out!

It’s also handy to publish a list of any products that you removed. This provides a signal to the vendor that they have to remove those products from their store. Otherwise they will end up selling products that no longer exist.

Data Here-to-There can design your data feed for you. We can use your source data and transform it into standard web-store compliant CSV data files. These files are then downloaded by your vendors and imported into their store. We’ll automate it as well so that it stays fresh every day.

Stores that we like

As a company that automates web stores, we see a variety of different platforms. It’s like walking into a Starbucks and asking for “coffee.” It’s never just “coffee.” It’s frappy, frothy, mocha, mega, grande, and that’s before we get to the sweetener options. Except in our case we see Magento, WooCommerce, Shopify, PrestaShop, and lots of lesser known animals (I’m intentionally leaving out the marketplaces because that’s a whole story of its own). Each one has its own set of versions from beginning to end. Some have simple security and others are like Fort Knox. Some are easy to use and some act like a misbehaving child…

Us: Please add this picture to the Golden Boots, product 101.

Store: OK…no, wait! Error 505

Use: What is error 505?

Store: Hahah!

Use: (sigh) Please add this picture to product 101

System: OK!

This is a (slightly exaggerated) real life story based on a Magento integration. Using the Magento REST API we would weave product data into a beautiful catalog. Except once in a while it would just throw random errors. The solution was to drop the REST API entirely, eat a whole bottle of Aspirin, then code a solution that writes directly to the database. We have a lot of emotional scars from Magento.

If I was to set up a store for myself it would use WooCommerce. It’s like the 8′ kid that always gets picked first for basketball. WooCommerce has an awesome API, smart user interface, amazing upgrade facilities built-in, and a smart but not too smart security model. I used to think Shopify was the Cat’s pyjamas until I got under the hood with WooCommerce.

It comes with an API written by people who know how to write an API. If I need to list a set of products they make it easy. If I need to authenticate, that’s easy too.

Shopify is like that too because it’s based on the same engine. One thing Shopify has over Woo is the built-in attribute for a Manufacturer name. It’s a simple thing yet has such a big impact. Pretty much every supply feed has manufacturer names associated to products. When I integrate a Shopify store I can simply map it right over…voila! With other systems I have to reinvent the manufacturer name ‘wheel’.

PrestaShop’s XML output is pretty cool. I like it because I can grab a big chunk of products fast. Most systems use paging with ridiculously low limits which creates a pile of network traffic.

Are you choosing a platform right now? If you want to be able to automate it in the future make sure it has an API. Most systems use a REST (Representational State Transfer) API. At some point in time you’ll want to link up QuickBooks, a supplier (or multiple suppliers), shipping services, your Aunt Betsy’s coupon service, etc. Today I found out that it’s actually possible for an e-commerce platform to *not* have an API. SquareSpace promotes itself as an e-commerce platform but provides no way to integrate inventory, shipping, or anything else. Yikes!

Optimized WooCommerce Hosting

WordPress and WooCommerce can be very heavy.  We’ve seen stores with 30,000+ products which grind to a halt when using shared hosting.  Pages load slowly, the API connections drop out, or the store is just very slow to respond when a customer is shopping.  Some hosts do not understand WordPress very well and don’t dedicate the right resources to supporting stores.

We’ve partnered with the Fullhost team to deliver optimized WooCommerce stores:

Click here to see the plans

We run our automation processes in the same data centers which means that your store will benefit from the tight integration.