Archive for August, 2009

Creating an Extraction Rule for VSTS 2008 Web Tests

Extraction rules are essentially for finding data in the HTTP response and placing it in the output context of the web test. There are a few built-in tests, but they mostly focus on the HTML tags themselves and the attributes. In my case I really needed the data between span tags. I think this could probably be done with the existing rules and some regular expressions, but I couldn’t resist the chance to write some code and learn something new.

All you need to do is place the class file in the Test Project and compile it. The rule automatically becomes available to the tests in the project.  Here is my class that finds a span for a given ClientId. It overrides the Execute method of the ExtractionRule base class and attempts to find a span for the given ID. If the span is found, it parses the HTML string to find the content of the span tag.

namespace WebTestDemo

{

    [System.ComponentModel.DisplayName(“Span Extractor”)]

    public class ExtractSpan : ExtractionRule

    {

        // The name of the desired input field

        private string nameValue;

        public string ClientId

        {

            get { return nameValue; }

            set { nameValue = value; }

        }

 

        public override void Extract(object sender, ExtractionEventArgs e)

        {

            string[] tagTypeFilter = new string[] { “span” };

 

            //Fail the test if nothing is found (this may need to be modified)

            e.Success = false;

 

            if (e.Response.HtmlDocument != null && e.Response.IsHtml)

            {

                try

                {

                    //Find the span tag based on ID. Exception if none found

                    HtmlTag result = e.Response.HtmlDocument.GetFilteredHtmlTags(tagTypeFilter).First(t => string.Equals(t.GetAttributeValueAsString(“ID”), this.nameValue, StringComparison.OrdinalIgnoreCase));

 

                    //The span was found (no exception), now find the data

 

                    //Get the location of the ID in the span tag

                    int startPosition = e.Response.BodyString.IndexOf(this.nameValue);

 

                    //Find the position of the data immediately following the closing angle bracket of the span,

                    //  accounting for the > character as well

                    startPosition = e.Response.BodyString.IndexOf(“>”, startPosition) + 1;

 

                    //Get the position of the closing tag for the span

                    int endPosition = e.Response.BodyString.IndexOf(“</span>”, startPosition);

 

                    //Fetch the content

                    string content = e.Response.BodyString.Substring(startPosition, endPosition – startPosition);

 

                    //Add the value to the context output. This could just as easily go to a file or a DB

                    // this step is not necessary for the extraction to succeed

                    e.WebTest.Context.Add(this.ContextParameterName, content);

 

                    //Mark the extraction as successful

                    e.Success = true;

                }

                catch (Exception)

                {

                    e.Success = false;

                    e.WebTest.Context.Add(this.ContextParameterName, string.Format(“span tag id={0} not found”, this.nameValue));

                }

            }

        }

    }

}

Now that I have the class, I need to wire it up to a URL in a web test. Right-click the URL and choose Add Extraction Rule…

AddRuleToTest[1]

I need to set two properties:

  1. Context Parameter Name. This comes from the base ExtractionRule class, and is the name for the data that ends up in the output.
  2. ClientId. This is a custom property from my class. It is the ClientId of the rendered control in the HTML output. The class finds the span with this name and returns the data.

Now when I run the test, if the ClientId I specified was found, it shows up in the Context output after running the test. The Context Parameter Name was “SpanData” in this case:

ExtractionRuleResults[1]

This could be made more robust by not coding specifically for spans. Certainly there could be issues if the tag ID is used more than once or if there is significant nesting of spans within the tag you are trying to find. This code is intended to prove out the concept, it could certainly be made stronger.

One thing I want to mention is that all the code samples I have run across (MSDN included) show the RuleName property as the way to display the extraction rule name in the Visual Studio UI. But under compilation this property comes up as obsolete. I found the answer on Ed Glas’s blog. The obsolete message mentioned using attributes, but this info was not discoverable, so I was quite grateful for that posting to get the syntax correct.

Helpful Links

Must Read VSTS – Testing Related Blogs and Introductory Articles

How to: Create a Custom Extraction Rule

Custom Extraction Rule and Generating a Code Test from VSTS

Adding A Relationship to the User Profile in Commerce Server 2009

For this post I am assuming you are familiar with Commerce Server 2009 and operation sequences, which in my opinion are the main reason to use the upgraded Commerce Server product. You can find some background on operation sequences on MSDN.

 

The problem we are trying to solve here is the addition of new properties to the Commerce Server Profile system, but we want a one-to-many relationship between the user profile and the new property. As an example, we will be adding a number of Tests to the user’s profile. In this case the user needs to participate in a number of tests that we can track, because these tests determine what products a user will be able to purchase. The more tests and test categories the user participates in, the more products they are eligible for.

 

The basic steps for this are:

  • Create the underlying SQL Server data store
  • Map the store to the user profile in Commerce Server manager
  • Create further mappings for Commerce Entities in the MetadataDefinitions.xml file
  • Create C# classes
  • Add the classes to the operation sequences in ChannelConfiguration.config

See How to Use the Profiles Schema Manager on MSDN for documentation on the Profiles editor in Commerce Server Manager.

  1. Create the underlying data table and fields in SQL Server in the profile database where the data will be stored. This is very important to do first, even though Commerce Server will let you create definitions before the underlying data store exists. (I’m assuming you are familiar with SQL Server). In my example I am going to add a number of tests per each person in the profile. The tests have and ID, name, and category.
  2. Assuming that we are adding our relationship to the User Profile, add a new field to the UserObject table that is nvarchar(255) and named after your custom profile object. Commerce Server uses this field to store the IDs (primary key) of the related items. You may need to make the field size larger if your key is large like a GUID or you expect to have many child records. 255 is the default number from the Commerce Server SDK examples.
  3. Save the database change scripts (for source control)
  4. Open Commerce Server Manager, expand the tree:
    Commerce Server Manager –> Global Resources –> Profiles –>Profile Catalog –> Data Sources –> SQL Data source (name varies) –> Data objects
  5. Create a new object: Right click the Data Objects node and choose New Data Object. Choose the database table name from the list box, the Display Name and Members will fill out automatically, even though the members are not propagated further.
  6. Add Members to your new object: For each field you want to add, right click the new node and choose New Data Member.
    1. Choose the Member Name by clicking the database field name in the list.
    2. Click the Add button.
    3. Repeat for other new members
    4. Click the Finish button.
  7. Move to the Profile Definitions node in Commerce Server Manager.
  8. Right-click the Profile Definitions node and choose New Profile Definition.
  9. Give the new profile definition a name and display name. Click the Finish button, ignoring the Custom Attributes section.
  10. Update the new node by first clicking the node and allowing the editor to load.
  11. Create a new Group by clicking the Add button and selecting Add a New Group.
    1. Name the first group “GeneralInfo” with a display name of “General Information”. This is a common Commerce Server pattern.
    2. Click the Apply button.
  12. For each property you want to add, click the Add button and choose to add a new property
  13. Give the property a name and display name. Case matters later, so pay attention to the details here.
  14. Choose the correct type
    1. If you are adding a relationship to a profile definition type:
      1. Choose Profile.
      2. Choose the definition type for the Type Reference field.
    2. If you are adding a new field
      1. Choose the appropriate data type
  15. Expand the advanced attributes node
    1. Choose the Map To Data and use the Data Source Picker to map the database field
    2. Click the Apply button, or the change won’t update the Properties panel.
  16. Be sure to define the primary key
  17. Save the Profile
  18. Now we will build the relationship between the User Object and the new Profile Item we created.
    Click the User Object node and allow the editor to load.
  19. Expand the General Information group
  20. Add a new property to the General Information group
    1. Name the property after your relationship
    2. Choose Profile as the Type
    3. In the Type Reference field navigate to your new Profile type
    4. In the Map To data field choose the database field you created in the UserObject table in step 2.
    5. Click the Apply button.
  21. Save the Profile
  22. Export the schema as an XML file (to put under source control)
  23. Now we need to map the changes we made in Commerce Server Manager into a CS2009 definition. In your web project in Visual Studio, edit MetadataDefinitions.xml
    1. Find he <CommerceEntities> node under the <DefaultChannel> node
    2. Create a new <CommerceEntity> node for the new item, Test in my example.

      <CommerceEntity name=Test>

          <DisplayName value=Test/>

          <EntityMappings>

              <EntityMapping

                csType=Microsoft.CommerceServer.Runtime.Profiles.Profile

                csAssembly=Microsoft.CommerceServer.Runtime, Version=6.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35

                csDefinitionName=Test

                csArea=Profiles>

                  <PropertyMappings>

                      <PropertyMapping property=TestId csProperty=GeneralInfo.TestId />

                      <PropertyMapping property=TestName csProperty=GeneralInfo.TestName />

                      <PropertyMapping property=TestCategory csProperty=GeneralInfo.TestCategory />

                  </PropertyMappings>

              </EntityMapping>

          </EntityMappings>

          <Properties>

              <Property name=TestId dataType=String />

              <Property name=TestName dataType=String />

              <Property name=TestCategory dataType=String />

          </Properties>

          <Relationships/>

      </CommerceEntity>
       

    3. Find the <CommerceEntity> node named UserProfile.
    4. Add a new PropertyMapping node in the ProperyMappings section.
    5. Add a property attribute to name the property after the name defined in step 20.
    6. Add a csProperty attribute that is the name you chose in step 20a. Case matters. The format is PropertyGroup.PropertyName (ex: GeneralInfo.Gender)
    7. Find the <relationships> node under UserProfile.
      1. Add a relationship:
        ex:

        <Relationships>

            <Relationship name=Tests type=Relationship modelName=Test isMultipleItems=true>

                <DisplayName value=Tests />

                <Description value=User’s Tests />

            </Relationship>

        </Relationships>

  24. Create the following types of classes for Commerce Server:
    1. A class to hold your new data


      namespace
      CommerceDemo

      {

          [Serializable]

          public class Test

          {

              public string TestId { get; set; }

       

              public string TestName { get; set; }

       

              public string TestCategory { get; set; }

          }

      }

    2. A processor class that inherits RelatedProfileProcessorBase

      namespace CommerceDemo

      {

          public class UserProfileTestProcessor : RelatedProfileProcessorBase

          {

              // Methods

              protected override bool CanDeleteRelatedItem(string relatedItemId)

              {

                  return true;

              }

       

              // Properties

              protected override string CommercePreferredIdKeyName

              {

                  get

                  {

                      return “GeneralInfo.Test”;

                  }

              }

       

              protected override string CommerceRelatedIdsKeyName

              {

                  get

                  {

                      return “GeneralInfo.Test”;

                  }

              }

       

              protected override string ParentProfileModelName

              {

                  get

                  {

                      return “UserProfile”;

                  }

              }

       

              protected override string PreferredItemRelationshipName

              {

                  get

                  {

                      return “TestId”;

                  }

              }

       

              protected override string ProfileModelName

              {

                  get

                  {

                      return “Test”;

                  }

              }

       

              protected override string RelationshipName

              {

                  get

                  {

                      return “Tests”;

                  }

              }

       

          }

      }

    3. A committer class that inherits ProfileCommitterBase

      namespace CommerceDemo

      {

          class UserProfileTestCommitter: ProfileCommitterBase

          {

              protected override string ProfileModelName

              {

                  get { return “Test”; }

              }

          }

      }

    4. A response builder class that inherits RelatedTestResponseBuilderBase

      namespace CommerceDemo

      {

           class TestResponseBuilder : ProfileResponseBuilderBase

          {

       

              protected override string ProfileModelName

              {

                  get

                  {

                      return “Test”;

                  }

              }

          }

      }

  25. Open ChannelConfiguration.config
    1. Find the <MessageHandler> node for “CommerceQueryOperation_UserProfile”
      1. Add <Component> nodes for the processor and response builder classes created above:

        <MessageHandler name=CommerceQueryOperation_UserProfile responseType=Microsoft.Commerce.Contracts.Messages.CommerceQueryOperationResponse, Microsoft.Commerce.Contracts, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35>

            <OperationSequence>

                <Component name=User Profile Loader type=Microsoft.Commerce.Providers.Components.UserProfileLoader, Microsoft.Commerce.Providers, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35/>

                <Component name=User Profile Test Processor type=CommerceDemo.UserProfileTestProcessor, CommerceDemo, Version=1.0.0.0, Culture=neutral,PublicKeyToken=fbeb08efdb2d0b73/>

                <Component name=User Profile Response Builder type=Microsoft.Commerce.Providers.Components.UserProfileResponseBuilder, Microsoft.Commerce.Providers, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35 />

                <Component name=User Profile Test Response Builder type=CommerceDemo.UserProfileTestResponseBuilder, CommerceDemo, Version=1.0.0.0, Culture=neutral,PublicKeyToken=fbeb08efdb2d0b73 />

            </OperationSequence>

        </MessageHandler>

    2. Find the <MessageHandler> node for “CommerceCreateOperation_UserProfile”
      1. Add <Component> nodes for the processor, Committer, and response builder classes created above, similar to the previous step.
    3. Find the <MessageHandler> node for “CommerceDeleteOperation_UserProfile”
      1. Add <Component> nodes for the processor, Committer, and response builder classes created above, similar to step 25a1.
  26. Create any converter classes which converts the collection of Commerce Server properties from a CommerceEntity into a typed C# object.

            public static Test ConvertTest(CommerceEntity testEntity)

            {

                Test result = new Test();

     

                result.Id = testEntity.Id;

                result.TestName = testEntity.Properties[“TestName”].ToString();

                result.TestCategory = testEntity.Properties[“TestCategory”].ToString();

     

                return result;

            }

  27. Redeploy the project
  28. Reset IIS so the Commerce Server cache is emptied.

So how does this look when wired up? That’s a complex question to be sure, but here is a simplified example, assuming we have a User object class with simple common properties:

public User GetUser(string userId)

{

    User user = null;

    OperationServiceAgent svc= new OperationServiceAgent();

 

    CommerceQuery<CommerceEntity> query = new CommerceQuery<CommerceEntity>(“UserProfile”);

    query.SearchCriteria.Model.Properties[“Email”] = userId;

 

    CommercePropertyCollection properties = new CommercePropertyCollection();

    properties.Add(“Email”);

    properties.Add(“FirstName”);

    properties.Add(“LastName”);

    query.Model.Properties = properties;

 

    CommerceQueryRelatedItem<CommerceEntity> test= new CommerceQueryRelatedItem<CommerceEntity>(“Tests”, “Test”);

    test.Model.Properties.Add(“TestId”);

    test.Model.Properties.Add(“TestNumber”);

    test.Model.Properties.Add(“TestCategory”);

    query.RelatedOperations.Add(test);

 

 

    CommerceResponse response = svc.ProcessRequest(GetRequestContext(), query.ToRequest());

   

    CommerceQueryOperationResponse  queryResponse = response.OperationResponses[0] as ommerceQueryOperationResponse;

    if (queryResponse !=null && queryResponse.CommerceEntities.Count > 0)

    {

        Convert(queryResponse.CommerceEntities[0]);

    }

    return user;

}

 

public static User Convert(CommerceEntity userEntity)

{

    User result = new User();

    result.Id = userEntity.Properties[“Id”] as string;

    result.Email = userEntity.Properties[“Email”] as string;

    result.LastName = userEntity.Properties[“FirstName”] as string;

    result.FirstName = userEntity.Properties[“LastName”] as string;

 

    CommerceRelationshipList relationships = userEntity.Properties[“Tests”] as CommerceRelationshipList;

    result.Tests = ConvertTest(relationships); //Function above in post

 

     return output;

}

Posted on:
Posted in ASP.NET, Commerce Server | Comments Off on Adding A Relationship to the User Profile in Commerce Server 2009

Useful LogParser Queries

If you deal with production servers and you don’t use LogParser, you should. It gives SQL-like abilities to query web server logs an other log types (like Event Logs). Coding Horror has a great article about the tool as well, and a link to a good article about forensic log parsing. Here are my most used web queries:

Find Pages with 500 errors
logparser “SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status >= 500) GROUP BY cs-uri-stem, code ORDER BY Hits DESC” -o:CSV >> C:\Data\ErrorPages.csv

Find 404 Requests
logparser “SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status = 404) GROUP BY cs-uri-stem, code ORDER BY Hits DESC” -o:CSV >> C:\Data\404Pages.csv

Find the Slowest Pages
logparser “SELECT TOP 100 cs-uri-stem AS Url, MIN(time-taken) as [Min], AVG(time-taken) AS [Avg], max(time-taken) AS [Max], count(time-taken) AS Hits FROM C:\ProdLogFiles\ex*.log GROUP BY Url ORDER BY [Avg] DESC” -o:CSV >> C:\Data\SlowPages.csv