James Broome

I’ve been working on an eCommerce Project for the last 18 months built around Microsoft Commerce Server and over that time have picked up a lot of hints, tips, hacks etc. I’ve been meaning to document a lot of this stuff for a while so it may be old news to some of my current colleagues, but might help out those of you who are starting out on a similar project.

This time I’m looking at Commerce Server Staging (the most under appreciated feature of CS 2007, apparently) and in particular running incremental staging projects.

One of the biggest challenges that we face on my current project is the sheer size of our product catalog (approx 8 million items). This becomes an issue when you realise that the staging process works by exporting the source catalog to XML, transferring the file to the relevant locations and then importing the XML back into the destination catalog!

So, incremental staging is vital for our project to ensure that we only stage what we need – we were always going to have problems when we had to stage the full catalog due to the sheer size of the data. Unfortunately, it’s hard to avoid this as if we set up a new staging project to run incrementally, the first time this project runs, as it has never run before it will always attempt to stage the full catalog.

The staging process writes to two Access .mdb data files during operation. These files can be easily opened in Access which tells us the following:

1. [Commerce Server 2007 install directory]\Staging\Events\Events.mdbThis database contains two tables – one reference table with event codes, and one event table which basically contains the same entries as in the event log, but grouped by staging project where possible.
2. [Commerce Server 2007 install directory]\Staging\Data\StagingLog.mdbThis database contains only one simple table:
   

   Staging Log

• The first time an incremental staging project runs, if no entry is in this table, then the full catalog export is exported.
• Once the first incremental staging project has completed (successfully) an entry is added to this table, with a time stamp of the current date and time.
• The next time an incremental staging project runs, the latest time stamp for the project is used to perform an incremental export of the catalog.
• The staging service does not seem to bother with any referential integrity on the ReplicationId in the StagingLog.mdb, my tests indicated that the service just selects all the entries from the table for the project name and gets the latest timestamp to use in the export.

So, we can trick the staging service into thinking that it has already run by simply adding a new entry in the StaginLog.mdb with the current date and time and an arbitrary GUID for the replication ID, with a status of Completed.

N.B. this assumes that the source and destination catalogs have been replicated by some other mechanism (e.g. SQL backup and restore) and are therefore in sync before this is done.

The incremental staging project now works as desired -the first (full) catalog staging job has never occurred and only incremental changes will be staged from now on.

I’ve been working on an eCommerce Project for the last 18 months built around Microsoft Commerce Server and over that time have picked up a lot of hints, tips, hacks etc. I’ve been meaning to document a lot of this stuff for a while so it may be old news to some of my current colleagues, but might help out those of you who are starting out on a similar project.

Recently I had to make some changes to our live Marketing data – update a load of existing Page Groups that were being used by Advertisements on the live site. The Marketing system does not have the same import/export capabilities that the catalog has – there’s no equivalent to exportcatalog.exe or importcatalog.exe so running incremental import scripts was not an option.

So, I decided to write a PowerShell script that could be run as part of our deployment to make the updates to the data. This was my first real experience of PowerShell – I’ve played around with it a bit, but hadn’t yet used it “for real”.

Here is the script – it creates a Marketing Context for the Marketing web service and then updates all the necessary Page Groups accordingly:

######################################################################################################
# Connect to the Marketing Web service and update the existing page groups
######################################################################################################

# Global Variables to set:

#Url of marketing webservice
$Url = “localhost”

#Connect to the Marketing web service and update the existing Test1 page groups to Test2

[reflection.assembly]::loadwithpartialname(“Microsoft.CommerceServer.Marketing.CrossTierTypes”);

$marketingAgent = new-object -typename Microsoft.CommerceServer.Marketing.MarketingServiceAgent -argumentlist (“http://” + $Url + “/MarketingWebService/MarketingWebService.asmx”);

$marketingContext = [Microsoft.CommerceServer.Marketing.MarketingContext]::Create($marketingAgent);
$marketingContext;

$pageGroups = $marketingContext.PageGroups.GetAllPageGroups();

foreach ($pageGroup in $pageGroups)
{

$pageGroup.Tag = $pageGroup.Tag -replace ‘Test1′,’Test2′;
$pageGroup.Description = $pageGroup.Description -replace ‘Test1′,’Test2′;

$pageGroup.Save();

}

######################################################################################################
# End
######################################################################################################

I found that (ignoring my PowerShell learning curve) this was fairly straightforward and realised that the same principles could be applied to any of the Commerce Server APIs. We already have complete automated build process in place on our project and now that I have realised how easy it is to call the Commerce Server APIs via PowerShell, I will start to consider this option more for future data updates.

Last week a few of us got together to go through an excercise in Behaviour Driven Development. The purpose of this was to work through a hypothetical specification together, developing a solution using a BDD approach. There was a very varied level of experience and knowledge of BDD within the group (which was the main reason for doing this) and so we went off on a lot of tangents and discussions about what it was we were trying to do.

One thing that came out of these discussions, which is a perception that I’ve encountered before is that BDD is only useful for ‘acceptance’ style specifications (tests), i.e. that can be presented to, understood and even written by a business user.

What happens when you move into areas of the solution that are not directly concerned with the domain model of the business? Does the business user care or need to understand how your logging infrastructure works, or your custom caching layer, or your IOC container etc. etc.

No, of course they don’t. However, this does not mean that BDD does not apply outside the core business domain model in your solution. Every aspect and advantage of this approach still applies, the difference is that the domain changes. I think the key point about BDD is that it is about language – describing the behaviours of the domain you’re working in using the ubiquitous language of that domain. If the domain changes, then so does the language.

It’s not the business who is the end user of your logging API, it’s your peers in the development team – the dev lead, TAs, another developer etc. These people understand the technical domain, so it’s perfectly valid to use technical terminology in your specifications when developing these areas. So, as you move around through the layers of your solution architecture, the specification langauge will change to describe the behaviors of different parts of the solution.

A great example of this is the SpecUnit BDD framework, used for writing BDD style specifications. The specifications (unit tests) for this framework are written in a BDD style and the full specification is available at http://code.google.com/p/specunit-net/downloads/list. As the users of this framework are technical users, the domain language is a technical too. Here’s an extract from this specification report:

Context specifications    7 contexts, 13 specifications

Context, when a test fixture class is a concern    3 specifications

• should be attributed with the ConcernAttribute
• should have a name that is prefixed with the concern name
• should have a concern name that is the name specified by the concern attribute

Context, when a test fixture class is not a concern    3 specifications

• should not be attributed with the ConcernAttribute
• should not have a name that is prefixed with the concern name
• should not have a concern name

Context, when building a context for a test fixture type    2 specifications

• should collect and build the specifications
• should create a specification for each test method in the type

Context, when creating a context for a type that is not a test fixture    1 specification

• should cause an error

Context, when two contexts represent the same test fixture    1 specification

• they should be equal

Context, when a context behaves like another context    1 specification

• should have a subclass whose name begins with “behaves like”

Context, when a context does not behave like another context    2 specifications

• should not have a subclass whose name begins with “behaves like”
• may not have a subclass

To a developer this makes sense. If you wanted to use this framework, or maybe contribute to it by adding a feature, this specification uses the right language for you in order for you to understand it. It might not seem that this specification is written in the ubiquitous language of the business, but it is – in this case the business is that of software development and, more specifically, in domain of automated software testing. You may have your own custom testing API within your solution with a whole load of specifications for it describing its behaviour. Not for the benefit of the business, but the other users of that API.

I would argue that BDD is not just for ‘acceptance’ testing, but can be used at all layers of an application. A full specification across an entire solution might incorporate multiple domains – those of the business and those of a technical nature. The business users will be concerned with the specifications that involve their domain.  This is one of the main selling points of BDD – that the business and development team and talking the same language. I would go one step further and say that its just as valid that the development team are talking the same language amongst themselves – I know that I would love to have full, clear, easy to understand executable specifications for all the APIs that I use as a developer, as is the case with SpecUnit – it would definitely make my job a lot easier!