Package com.ssgllc.fish.service.util.registered

Class StringComparisonUtil

java.lang.Object
com.ssgllc.fish.service.util.registered.StringComparisonUtil
@Component public final class StringComparisonUtil extends Object
String similarity and distance metrics for dedupe match expressions and blocking sets.

Read the direction before scoring on one of these. The *Sim methods and jaroWinklerDist(String, String) return a similarity, where 1.0 means identical. jaccardDist(String, String), cosDist(String, String) and levDist(String, String) return a distance, where 0.0 means identical. jaroWinklerDist is named for Apache commons-text's JaroWinklerDistance, which despite its name computes a similarity - a match expression that treats it as a distance scores every pair backwards.

The q-gram methods (cosSim(String, String), jaccardSim(String, String)) compare bags of character q-grams, so they tolerate reordering; the edit-based ones do not.

  • Method Summary

    Modifier and Type
    Method
    Description
    static double
    cosDist(String str, String strToCompare)
    Cosine distance of two values - the complement of cosSim(String, String).
    static double
    cosSim(String str, String strToCompare)
    Cosine similarity of two values, compared as bags of character q-grams. 1.0 is identical, 0.0 shares nothing.
    static String
    doubleMetaphone(String value)
    The primary Double Metaphone code of a value.
    static String
    doubleMetaphoneAlt(String value)
    The alternate Double Metaphone code of a value.
    static Set<String>
    doubleMetaphoneCodes(String value)
    The Double Metaphone codes of a value — the primary plus the alternate encoding, deduplicated (one entry when they coincide, two when the name has two plausible pronunciations).
    static boolean
    doubleMetaphoneMatch(String name, String nameToCompare)
    Whether two values match phonetically under Double Metaphone.
    static boolean
    isSoundexMatch(String name, String nameToCompare)
    Whether two values match phonetically under Soundex — i.e. their Soundex codes are equal (case-insensitively).
    static double
    jaccardDist(String str, String strToCompare)
    Jaccard distance of two values - the complement of jaccardSim(String, String).
    static double
    jaccardSim(String str, String strToCompare)
    Jaccard similarity of two values: the proportion of their distinct character q-grams that they share. 1.0 is identical, 0.0 shares nothing.
    static double
    jaroWinklerDist(String str, String strToCompare)
    Jaro-Winkler score of two values, which favours matches that agree at the start - useful for personal names, where typos cluster later in the string.
    static double
    levDist(String str, String strToCompare)
    Levenshtein edit distance of two values, normalised by the longer value's length - so this returns a proportion in [0.0, 1.0] rather than a raw edit count.
    static String
    soundex(String value)
    The Soundex code of a value, for storing on an entity and matching in a blocking-set query (e.g. block on s.firstNameSoundex = {#soundex(firstName)}).

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • cosSim

      public static double cosSim(String str, String strToCompare)
      Cosine similarity of two values, compared as bags of character q-grams. 1.0 is identical, 0.0 shares nothing. Returns 0.0 if either value is null or empty. Tolerates reordered tokens, so it suits names and addresses better than an edit-based metric.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      a similarity in [0.0, 1.0], higher meaning more alike

      Groovy example:
      stringComparisonUtil.cosSim("John Smith", "Smith John")
      SpEL example:
      #cosSim(#e.lastName, #p.lastName) * 40
      Returns:
      a match-score contribution of up to 40 points

    • jaccardSim

      public static double jaccardSim(String str, String strToCompare)
      Jaccard similarity of two values: the proportion of their distinct character q-grams that they share. 1.0 is identical, 0.0 shares nothing. Returns 0.0 if either value is null or empty. Unlike cosSim(String, String) it ignores how often each q-gram occurs, so repeated substrings count once.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      a similarity in [0.0, 1.0], higher meaning more alike

      Groovy example:
      stringComparisonUtil.jaccardSim("Main Street", "Main St")
      SpEL example:
      #jaccardSim(#e.line1, #p.line1)
      Returns:
      0.6

    • jaroWinklerDist

      public static double jaroWinklerDist(String str, String strToCompare)
      Jaro-Winkler score of two values, which favours matches that agree at the start - useful for personal names, where typos cluster later in the string. This is a similarity, not a distance: 1.0 means identical. The name follows Apache commons-text's JaroWinklerDistance, which computes a similarity in spite of its name.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      a similarity in [0.0, 1.0], higher meaning more alike

      Groovy example:
      stringComparisonUtil.jaroWinklerDist("MARTHA", "MARHTA")
      Returns:
      0.961
      SpEL example:
      #jaroWinklerDist(#e.firstName, #p.firstName) > 0.9
      Returns:
      true when the given names are near-identical

    • jaccardDist

      public static double jaccardDist(String str, String strToCompare)
      Jaccard distance of two values - the complement of jaccardSim(String, String). 0.0 means identical and larger values mean less alike.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      a distance in [0.0, 1.0], lower meaning more alike

      Groovy example:
      stringComparisonUtil.jaccardDist("abc", "abc")
      Returns:
      0.0
      SpEL example:
      #jaccardDist(#e.city, #p.city) < 0.2
      Returns:
      true when the cities are close enough to treat as equal

    • cosDist

      public static double cosDist(String str, String strToCompare)
      Cosine distance of two values - the complement of cosSim(String, String). 0.0 means identical and larger values mean less alike.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      a distance in [0.0, 1.0], lower meaning more alike

      Groovy example:
      stringComparisonUtil.cosDist("hello", "hello")
      Returns:
      0.0
      SpEL example:
      #cosDist(#e.employerName, #p.employerName)
      Returns:
      0.15

    • levDist

      public static double levDist(String str, String strToCompare)
      Levenshtein edit distance of two values, normalised by the longer value's length - so this returns a proportion in [0.0, 1.0] rather than a raw edit count. 0.0 means identical. One substitution in a three-character value scores 1/3, not 1. Normalising keeps a threshold meaningful across values of different lengths.
      Parameters:
      str - the first value
      strToCompare - the second value
      Returns:
      edits divided by the length of the longer value, lower meaning more alike

      Groovy example:
      stringComparisonUtil.levDist("cat", "car")
      Returns:
      0.333
      SpEL example:
      #levDist(#e.lastName, #p.lastName) < 0.25
      Returns:
      true when at most a quarter of the surname differs

    • isSoundexMatch

      public static boolean isSoundexMatch(String name, String nameToCompare)
      Whether two values match phonetically under Soundex — i.e. their Soundex codes are equal (case-insensitively). Two nulls match; one null does not. Non-letter characters are stripped before encoding (via soundex(String)), so accented/punctuated names don't error. The Double Metaphone equivalent — which also catches initial-sound variants like C/K that Soundex splits — is doubleMetaphoneMatch(String, String).
      Parameters:
      name - the first value
      nameToCompare - the second value
      Returns:
      true if the two share the same Soundex code

      Groovy example:
      stringComparisonUtil.isSoundexMatch("Smith", "Smyth")
      SpEL example:
      #isSoundexMatch("Smith", "Smyth")

    • soundex

      public static String soundex(String value)
      The Soundex code of a value, for storing on an entity and matching in a blocking-set query (e.g. block on s.firstNameSoundex = {#soundex(firstName)}). Accented characters are folded to their ASCII base (e.g. Muñoz → Munoz, François → Francois) and any remaining non-letters stripped before encoding, so accented and unaccented spellings encode alike. Returns null for a null, blank, or letter-free input.
      Parameters:
      value - the value to encode
      Returns:
      the Soundex code, or null if there are no letters to encode

      Groovy example:
      stringComparisonUtil.soundex("Robert")
      SpEL example:
      #soundex("Robert")

    • doubleMetaphone

      public static String doubleMetaphone(String value)
      The primary Double Metaphone code of a value. Use this to populate a stored/computed phonetic column (firstNameDmeta = #doubleMetaphone(firstName)) that a blocking-set query then matches against. For the query side (matching on either pronunciation) use doubleMetaphoneCodes(String). Returns null for a null/blank input or a value with no encodable letters.
      Parameters:
      value - the value to encode
      Returns:
      the primary Double Metaphone code, or null if none

      Groovy example:
      stringComparisonUtil.doubleMetaphone("Smith")
      SpEL example:
      #doubleMetaphone("Smith")

    • doubleMetaphoneAlt

      public static String doubleMetaphoneAlt(String value)
      The alternate Double Metaphone code of a value. Double Metaphone emits a second code for names with two plausible pronunciations (often different-origin names); when there is no distinct alternate it equals the primary. Store this alongside doubleMetaphone(String) as a second phonetic column so a blocking-set query can match candidates on either pronunciation. Returns null for a null/blank input or a value with no encodable letters.
      Parameters:
      value - the value to encode
      Returns:
      the alternate Double Metaphone code, or null if none

      Groovy example:
      stringComparisonUtil.doubleMetaphoneAlt("Angelo")
      SpEL example:
      #doubleMetaphoneAlt("Angelo")

    • doubleMetaphoneCodes

      public static Set<String> doubleMetaphoneCodes(String value)
      The Double Metaphone codes of a value — the primary plus the alternate encoding, deduplicated (one entry when they coincide, two when the name has two plausible pronunciations). Intended for the query side of a blocking-set query so a record matches candidates on either pronunciation, e.g. s.firstNameDmeta in ({#doubleMetaphoneCodes(firstName)}). Never returns null; returns an empty set for a null/blank input or a value with no encodable letters.
      Parameters:
      value - the value to encode
      Returns:
      the distinct primary/alternate Double Metaphone codes; empty if none

      Groovy example:
      stringComparisonUtil.doubleMetaphoneCodes("Zhang")
      SpEL example:
      #doubleMetaphoneCodes("Zhang")

    • doubleMetaphoneMatch

      public static boolean doubleMetaphoneMatch(String name, String nameToCompare)
      Whether two values match phonetically under Double Metaphone. Matches when they share any code — primary or alternate — on either side (so it catches names that agree on only one of their two plausible pronunciations), which is looser and more recall-friendly than comparing primary codes alone. Two nulls match; one null does not (mirrors isSoundexMatch(java.lang.String, java.lang.String)). The Soundex equivalent of this is the existing isSoundexMatch.
      Parameters:
      name - the first value
      nameToCompare - the second value
      Returns:
      true if the two are phonetically equal under Double Metaphone

      Groovy example:
      stringComparisonUtil.doubleMetaphoneMatch("Smith", "Schmidt")
      SpEL example:
      #doubleMetaphoneMatch("Smith", "Schmidt")