I’ve been working on an eCommerce Project for the last 18 months built around Microsoft Commerce Server and over that time have picked up a lot of hints, tips, hacks etc. I’ve been meaning to document a lot of this stuff for a while so it may be old news to some of my current colleagues, but might help out those of you who are starting out on a similar project.

This time I’m looking at Commerce Server Staging (the most under appreciated feature of CS 2007, apparently) and in particular running incremental staging projects.

One of the biggest challenges that we face on my current project is the sheer size of our product catalog (approx 8 million items). This becomes an issue when you realise that the staging process works by exporting the source catalog to XML, transferring the file to the relevant locations and then importing the XML back into the destination catalog!

So, incremental staging is vital for our project to ensure that we only stage what we need – we were always going to have problems when we had to stage the full catalog due to the sheer size of the data. Unfortunately, it’s hard to avoid this as if we set up a new staging project to run incrementally, the first time this project runs, as it has never run before it will always attempt to stage the full catalog.

The staging process writes to two Access .mdb data files during operation. These files can be easily opened in Access which tells us the following:

  1. [Commerce Server 2007 install directory]\Staging\Events\Events.mdbThis database contains two tables – one reference table with event codes, and one event table which basically contains the same entries as in the event log, but grouped by staging project where possible.
  2. [Commerce Server 2007 install directory]\Staging\Data\StagingLog.mdbThis database contains only one simple table:

    Staging Log

    Staging Log

  • The first time an incremental staging project runs, if no entry is in this table, then the full catalog export is exported.
  • Once the first incremental staging project has completed (successfully) an entry is added to this table, with a time stamp of the current date and time.
  • The next time an incremental staging project runs, the latest time stamp for the project is used to perform an incremental export of the catalog.
  • The staging service does not seem to bother with any referential integrity on the ReplicationId in the StagingLog.mdb, my tests indicated that the service just selects all the entries from the table for the project name and gets the latest timestamp to use in the export.

So, we can trick the staging service into thinking that it has already run by simply adding a new entry in the StaginLog.mdb with the current date and time and an arbitrary GUID for the replication ID, with a status of Completed.

N.B. this assumes that the source and destination catalogs have been replicated by some other mechanism (e.g. SQL backup and restore) and are therefore in sync before this is done.

The incremental staging project now works as desired -the first (full) catalog staging job has never occurred and only incremental changes will be staged from now on.