Extending duplicate detection with Soundex

The duplicate detection included in Microsoft CRM 4.0 is great.  It provides a foundation for creating a variety of rules to detect duplicate records using "like" matches across multiple entity types.  However, "like" matches can start to breakdown when the spelling of names differs, which often occurs with proper names.  This article describes how to extend the duplication detection in Microsoft CRM 4.0 to include "fuzzy" matching based on the Soundex algorithm.

The Soundex algorithm converts a name into a four-character string, originally used for indexing, that can also be used for comparisons.  In Microsoft CRM 4.0 to implement Soundex duplicate detection we created a new field on the entities to be compared for storing the Soundex value and added JavaScript to the "onChange" event of the name field to store the value.  The steps involved are:

1. Add "Soundex" attribute to your entity
   
    
2. Add "Soundex" attribute to your form (set as read-only).  This step helps illustrate the example, validates the logic is working correctly, and makes the field available to the JavaScript in step # 3.  However, it is probably best not to display this value to the user and it could be hidden as described here on Ronald Lemmen's blog.
   
   
    
3. In the "onLoad" event set the ForceSubmit flag to true for the Soundex field
   
   
    
4. Apply "onChange" event to the primary name attribute using JavaScript such as the code found here to set the value of the Soundex field.
   
5. Create a new duplicate detection rule comparing the Soundex values
   
   
    

Once everything is in place, if you create an account named John's Bait Shop and try and create a new account named Jon's Bait Shop the duplication rules will detect a potential duplicate and alert the user accordingly.

What we found is not only is this rule effective in finding potential duplicates, but if Soundex rules are in place there is not really a need to use "like" matches based on the the primary field.  Once the Soundex values are in place they can be used across entities like other duplication rules.

It is worth noting the "onChange" event only fires when the data is input via the standard UI.  It is probably better to apply the Soundex logic via a server-side callout, but for the simplicity of this example we are just demonstrating a client-side approach.