EPiServer Commerce essentials 4: Importing from external systems

Unless you're helping a completely new business build a new e-commerce site from scratch, you'll probably face external systems in one form or another. You might need to import data into your new Commerce site, or push data back to them. Things like order history, product catalogs, user data, reviews or comments come to mind. And along the way, some sort of data transformation is very likely to be required.

Most existing businesses willalready have systems for dealing with their sales, products, inventory and customer relations. You will probably need to integrate against one or more of these to push or pull data.

If you are importing products, which you most likely are, you have to know the products. There are two primary ways of creating products:

• Using EPiServer's IContentRepository, and creating produts in the same way as you create other kinds of content.
• Using the file import in the commerce manager. I found this to be faster when importing a lot of products (6000++), but a bit clumsy to get going with.

You also need to consider other things that you might need to import, and where and how to store it. Keep in mind: What you do at this point, is very likely going to be that way for years to come. Design your data structures carefully, and use existing structures in EPiServer and Commerce appropriatly. Other things you might need to import are:

• Users/customer profiles
• Existing orders
• Product reviews
• Comments on articles
• Stored files along with products (user manuals, reviews from printed magazines...)
• Images
• Other things that are completely customer specific - this could be anything of any complexity. If you do not know, ask.

Tips for doing the imports

Back up your database often

While testing these imports, be aggressive with how often you backup your database. It's no fun having imported ten thousand products into what will become a production database only to find out that everything about it it's screwed up in some way.

Test early with complete data sets

Test early with complete data sets. I can't overstate this. Don't do test imports of only the first 20 products until you're doing the complete import. Errors always occur on item #21 and onwards. Expect to spend time on completely unforseen issues, and let that be part of your estimate. Yes, this will increase your estimates to near stupid looking levels for something "trivial". And you might just be right - it might be trivial. But if it isn't, it's a whole lot cooler spending five hours on a twenty hour estimate, than spending a hundred.

Expect inconsistent data

Older data is likely to contain more inconsistencies and errors than recent data, and you can't find out what before testing a complete import. Again, expect to spend time on things nobody anticipated.

The older the data, and the older the systems to import from, the larger the chance of the data being inconsistet. Or plain bad. A couple of things that should ring a bell:

• The system has no proper integration points or easy way to retrieve data from (probably means it's old and messy)
• The different fields you have decided on your products (along with your customer, of course), don't really match anything in the old system
• The amount of data in the old system far exceeds what's currently available on their web pages
• To be honest: The people managing the data. You'll notice if they are awesome or not.

Use local databases

If you're importing a lot of data, consider running all databases on the same environment that is running the import job to speed up performance. A few minutes of overhead in installing SQL Server on a web server or on your own development environment will pay off if the import is going to take you an hour every time to run through completely.

This blog post is part of the series EPiServer Commerce project essentials which will give you a few tips on how to plan your projects.