The Filter action filters complex lists (data models or complex JSON objects) based on multiple criteria. This documentation outlines the parameters,…

Overview

The Filter action filters complex lists (data models or complex JSON objects) based on multiple criteria. This documentation outlines the parameters, JSON configuration, and match types associated with the Filter action.

Parameters

The Filter action requires the following parameters:

Input (required): List to filter (list<object>)
Input (required): Filter JSON Configuration - this entry will allow JSON
Output (required): Filtered list (list<object>)

JSON Configuration

{
	"FieldMatches":[
		{
			"FieldName":"Field1",
			"MatchType":"Exact",
			"FilterExpression":"expression",
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field2",
			"MatchType":"Similar",
			"FilterExpression":"expression",
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field3",
			"MatchType":"Fuzzy",
			"FilterExpression":"expression",
			"FuzzyAlgorithm":"Levenstein",
			"FuzzyThreshold":0.7,
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field4",
			"MatchType":"Contains",
			"FilterExpression":"expression",
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field5",
			"MatchType":"Soundex",
			"FilterExpression":"expression",
			"SoundexDistance":4,
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field6",
			"MatchType":"SimilarWordMatch",
			"FilterExpression":"expression",
			"SimilarFirstNwords":1,
			"SimilarLastNwords":1,
			"KeepIfTrue":true
		},
		{
			"FieldName":"Field7",
			"MatchType":"RegEx",
			"FilterExpression":"RegEx expression",
			"KeepIfTrue":true
		}
	],
	"AlternateFields":[
		{
			"FieldName":"Field1",
			"FieldAlternateNames":["Field2","Field3"]
		},
		{
			"FieldName":"Field3",
			"FieldAlternateNames":["Field4","Field1","Field2"]
		}
	]
}

The JSON configuration for the Filter action includes two main components: FieldMatches and AlternateFields.

MatchType Types

Exact

An exact match requires validation (=) against the specified FilterExpression.

Similar

Compares against FilterExpression considering:
- Special characters: For example, "John-Doe" and "John Doe" are considered a match after removing special characters.
- Phone numbers: e.g., "376-323-1111" and "323-1111" are considered a match.
- Domain/URL: Ignores prefixes like "WWW" or "HTTP(s)://".
- Email format: Matches emails with various representations like "@" or "[at]".

Fuzzy

Utilizes algorithms from Similarity Search to compare fields against FilterExpression. More details here.

Contains

Checks if FilterExpression contains the field or vice versa.

Soundex

Evaluates if two words are similar when spoken, based on a scale from 0 to 4.

SimilarWordMatch

Compares the first or last N words in the field with those in FilterExpression.

RegEx

Validates if the field matches the pattern specified in FilterExpression. The RegEx pattern must be JSON safe/encoded (e.g. by using test json_encode online - general PHP functions - functions-online).

Properties

FuzzyAlgorithm & FuzzyThreshold

Only used when MatchType = Fuzzy. If MatchType is something else, those properties will be ignored so, they can be null or can even not be present.
FuzzyAlgorithm needs to be one from this here, written exactly as in the link provided.
FuzzyThreshold is a double between 0 and 1, where 0 means any match is accepted and 1 requires 100% confidence.

SoundexDistance

Used only when the MatchType = Soundex and it can be set to 0, 1, 2, 3, or 4, where 4 means the words are very similar when spoken.

SimilarFirstNwords & SimilarLastNwords

Only used when MatchType = SimilarWordMatch in which case at least one should be >0. If both are =0 then the SimilarWordMatch will not be executed at all since it does not make sense. If MatchType is something else, those properties will be ignored, so, they can have any value or can even not be present.

KeepIfTrue

Determines if the record should be kept based on the match criteria. If set to true, the record will be retained.

The matching algorithms work only on Fields that are of type primitive and not on data structures. A Row Match is TRUE only if all matches are TRUE, so the evaluation looks like this (pseudo-code for the JSON config from above):

IMPORTANT: If you have alternative fields for a match and the KeepIfTrue = true, then the match will be true if ANY (at least one) of the evaluations of the main OR the alternative fields is true.

IMPORTANT: If you have alternative fields for a match and the KeepIfTrue = false, then the match will be true if ALL of the evaluations of the main AND the alternative fields are false.

AlternateFields

The AlternateFields section specifies alternate fields for matching. When evaluating a match for a certain field, the same match assessment will be executed for the alternate fields as well, and the result of all those alternate matches will be evaluated with OR to determine the result of the overall match.

If Fields1 has the alternate fields Field2 & Field3 does not mean that Field2/Field3 has Field1 as an alternate field.

The matching algorithms work only on FieldName that are of type primitive and not on data structures. A Row Match is TRUE only if all matches are TRUE.

Filter

On this page