GDS Data Validation

A rule-based schema definition language and data validation library.

nice picture

Introduction

The original idea was to create a library for the validation of data structures which have been created by the GDS (General Data Structure) language.

Synonyms and related terms are data validation, validation checker, schema validation, schema validator, schema checker.

Schema definition

The schema definition is a specification describing which kind of data is expected and should be valid. The purpose is to validate incoming data against a schema.

Data to be validated

Incoming data to be validated can be

  • a simple value of a basic data type (e.g. integer, floating-point, string, boolean)
  • or a complex value of a composite data type (e.g. hash, array). A composite data type can be recursively nested by other hashes and arrays. A hash and an array can also include simple values of basic data types.

Rule-based schema definition

The validation schema is defined with a set of rules. This approach is borrowed from topics of syntax analysis and parser generation. The schema definition is similar to a set of production rules for the definition of a context free grammar describing valid sentences of a formal language. Usually for the notation of a context free grammar Backus-Naur form (BNF) or one of its variants like extended Backus-Naur form (EBNF) is used. The syntax of rule-based schema definitions is more similar to augmented Backus-Naur form (ABNF).

The first rule you define is important, because this is the starting rule for the overall schema definition. The order of following rules does not matter. As a name for the starting rule, you can basically choose any name. Depending on the context in which you are using the validation, abstract names like ‘main’, ‘top’, ‘start’, ‘base’, ‘spec’, ‘schema’, ‘contract’ or ‘value’ might be suitable, in other cases more specific names like ‘persons’, ‘addresses’ or ‘company’ might be more expressive.

In general, you have to pay attention and avoid collisions of rule names with keywords. Predefined keywords of the schema definition language are described later.

Please note that, even for the validation of a simple value you would need to define a starting rule with a rule name.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int
EOS

dataValidation.check( 4 )       # => true
dataValidation.check( "a" )     # => false
dataValidation.check( 4.0 )     # => false
dataValidation.check( true )    # => false

A more sophisticated schema definition:

dataValidation = GdsDataValidation.create( <<-EOS )
company = :name : @t_string, :address : address, :ceo : person, :employees : person*
person = :firstname : @t_string, :lastname : @t_string, :yearOfBirth : @t_int, :address : address
address = :street : @t_string, :zipcode : @t_int, :city : @t_string
EOS

dataValidation.check( nil )     # => false
dataValidation.check( 
  { name: 'My Company', address: { street: 'Broadway 300', zipcode: 22222, city: 'New York' },
    ceo: { firstname: 'John', lastname: 'McArthur', yearOfBirth: 1959, 
           address: { street: 'Rosedale Dr. 40', zipcode: 34003, city: 'Los Angeles' } },
    employees: [
      { firstname: 'Berry', lastname: 'Miller', yearOfBirth: 1989, 
        address: { street: 'South St. 12', zipcode: 48333, city: 'Chicago' } },
      { firstname: 'Jane', lastname: 'Smith', yearOfBirth: 1993, 
        address: { street: 'Mainstreet 4', zipcode: 62883, city: 'Seattle' } }
    ]
  } 
)   # => true

Same schema definition just in a different formatting style:

dataValidation = GdsDataValidation.create( <<-EOS )
company = :name        : @t_string, 
          :address     : address, 
          :ceo         : person, 
          :employees   : person*
person =  :firstname   : @t_string, 
          :lastname    : @t_string, 
          :yearOfBirth : @t_int, 
          :address     : address
address = :street      : @t_string, 
          :zipcode     : @t_int, 
          :city        : @t_string
EOS

Each rule starts with a rule name followed by an equal sign ‘=’ and a rule specification.

<rule name> = <rule specification>

The rule specification is used to define a valid structure or valid values. The rule specification also can refer to other rules by referencing other rules by their rule names and express dependencies to these rules. Each rule specification is part of the overall schema definition and is checked for validity. To be more precise, each rule specification which is directly or indirectly referenced by the starting rule is verified by the validation process.

The following rule specifications are available:

Each of these rule specifications define some kind of constraint or assertion which needs to be satisfied or fulfilled in order to successfully validate the incoming data.

Rule alternatives

If necessary you can define for one rule several alternative rule specifications. This list of rule specifications needs to be sparated by the slash character ‘/’.

This is corresponding to alternatives for a production rule of a context free grammar definition.

@maybe

Each rule specification can be prefixed with the @maybe specifier.

If the concerned value is nil, then this rule specification, that means this part of the schema definition is valid and it is not futher investigated.

Usage with Ruby

Installation

gem install gds-data-validation

Coding

require 'gds-data-validation'

Hash specification

The hash specification consists of a comma-separated list of key-value specifications.

The syntax of a single key-value specification is the following:

<key> : <value specification>

The key is always a symbol, it has to start with a colon ‘:’.

All specified keys are required, that means for a successful validation the presence of all specified keys is an obligatory requirement.

Value specification

The value specification describes requirements for the value part of a key-value pair.
As a value specification the following rule specifications are allowed:

As you see, the major difference compared to a rule specification is, that for a value specification of a hash another hash specification is not allowed. At least it is not allowed yet, maybe in the future this will be changed. Anyway you can express the fact that the value part of a key-value pair of a hash is another hash simply by referencing another rule which is specifying a hash.

dataValidation = GdsDataValidation.create( <<-EOS )
person = :name : @t_string, :address : address
address = :street : @t_string, :zipcode : @t_int, :city : @t_string
EOS

The above example contains two rule definitions. Each of them is specifying a hash. The person rule is specifying a key :address which is referencing for its value the rule named address. The address rule is specifying its own key-value pairs.

Another difference compared to a rule specification is, that you could even reference to the same rule, that means to the one you are currently defining. This self-reference allows to define schemas for recursive data structures.

The following example uses a self-reference

dataValidation = GdsDataValidation.create( <<-EOS )
person = :name : @t_string, :farther : person, :mother : person
       / nil
EOS

it could be written like this

dataValidation = GdsDataValidation.create( <<-EOS )
person = @maybe :name : @t_string, :farther : person, :mother : person
EOS

Each value specification can be prefixed with the @maybe specifier.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = :number : @maybe @t_int
EOS

dataValidation.check( {} )                   # => false
dataValidation.check( { number: "a" } )      # => false
dataValidation.check( { number: 1   } )      # => true
dataValidation.check( { number: nil } )      # => true

Optional key-value pairs

Usually you use a colon (‘:’) to separate a key from a value specification in the definition of a key–value-specification pair.

In this case for a successful validation the presence of the key is an obligatory requirement.

In the case the key and its corresponding value should be optional you use the notation ‘:?’ instead of ‘:’.

The syntax for an optional key-value specification is the following:

<key> :? <value specification>

dataValidation = GdsDataValidation.create( <<-EOS )
schema = :number :? @t_int
EOS

dataValidation.check( {} )                   # => true
dataValidation.check( { number: "a" } )      # => false
dataValidation.check( { number: nil } )      # => false
dataValidation.check( { number: 1   } )      # => true

@strict

So far we defined required keys and optional keys. But what about unwanted keys?

By default all additional, possibly unwanted keys are accepted.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = :number : @t_int
EOS

dataValidation.check( { number: 1 } )                      # => true
dataValidation.check( { number: 1, unwanted: "bang!" } )   # => true

By default the validation logic is only checking for specified keys making sure they are present.

If you want to allow only specified keys then you have to prefix the hash specification with the @strict specifier.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @strict :number : @t_int
EOS

dataValidation.check( { number: 1 } )                      # => true
dataValidation.check( { number: 1, unwanted: "bang!" } )   # => false

Array specification

The array specification consists of a rule name followed by a cardinality specification.

<rule name> <cardinality specification>

The cardinality specification can be one of the following:

The parameters size, minSize and maxSize have to be integer literals.

*

Verifies that the value is an array which could be empty or of any size.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number*
number = @t_int
EOS

dataValidation.check( nil )        # => false
dataValidation.check( 1 )          # => false
dataValidation.check( "a" )        # => false
dataValidation.check( [] )         # => true
dataValidation.check( [ 1 ] )      # => true
dataValidation.check( [ 1, 2 ] )   # => true

+

Verifies that the value is an array which has at least one element.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number+
number = @t_int
EOS

dataValidation.check( [] )         # => false
dataValidation.check( [ 1 ] )      # => true
dataValidation.check( [ 1, 2 ] )   # => true

( minSize .. maxSize )

Verifies that the value is an array which has at least minSize elements and not more than maxSize elements.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number(2..4)
number = @t_int
EOS

dataValidation.check( [] )                  # => false
dataValidation.check( [ 1 ] )               # => false
dataValidation.check( [ 1, 2 ] )            # => true
dataValidation.check( [ 1, 2, 3 ] )         # => true
dataValidation.check( [ 1, 2, 3, 4 ] )      # => true
dataValidation.check( [ 1, 2, 3, 4, 5 ] )   # => false

( minSize .. )

Verifies that the value is an array which has at least minSize elements.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number(2..)
number = @t_int
EOS

dataValidation.check( [] )                  # => false
dataValidation.check( [ 1 ] )               # => false
dataValidation.check( [ 1, 2 ] )            # => true
dataValidation.check( [ 1, 2, 3 ] )         # => true

( .. maxSize )

Verifies that the value is an array which has not more than maxSize elements.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number(..4)
number = @t_int
EOS

dataValidation.check( [] )                  # => true
dataValidation.check( [ 1 ] )               # => true
dataValidation.check( [ 1, 2 ] )            # => true
dataValidation.check( [ 1, 2, 3 ] )         # => true
dataValidation.check( [ 1, 2, 3, 4 ] )      # => true
dataValidation.check( [ 1, 2, 3, 4, 5 ] )   # => false

( size )

Verifies that the value is an array which has exact size elements.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = number(3)
number = @t_int
EOS

dataValidation.check( [] )                  # => false
dataValidation.check( [ 1 ] )               # => false
dataValidation.check( [ 1, 2 ] )            # => false
dataValidation.check( [ 1, 2, 3 ] )         # => true
dataValidation.check( [ 1, 2, 3, 4 ] )      # => false

Basic data type specification

The following basic data types are available:

  • @t_string
  • @t_int
  • @t_float
  • @t_numeric
  • @t_symbol
  • @t_true
  • @t_false
  • @t_bool
  • @t_nil
  • @t_any

A basic data type is a predicate which will be verified.

For several basic data types there are additional value restricting predicates available which are restricting the set of allowed values of a basic data type. They are used as refinements for the specification.

Each predicate can be prefixed with the logical not operator ‘!’ in order to negate the predicate. That means a basic data type predicate and an additional value restricting predicate can be prefixed with ‘!’.

All predicates only need to be separated by a space, however logically they represent a conjunction and you can think of an imaginary logical and operator ‘&’ being in between each of them.

As you would expect from other programming languages, the logical not operator ‘!’ has a higher precedence than the (imaginary) logical and operator and binds strong to the following predicate.

@t_string

The value is required to be of type String.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( "a" )       # => true
dataValidation.check( "hello" )   # => true

Value restricting predicates for @t_string

If one of the following predicates requires a compareValue, then it has to be noted as a double-quoted string literal.

== compareValue

Verifies that the string value is equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string == "hello"
EOS

dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => true

!= compareValue

Verifies that the string value is not equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string != "hello"
EOS

dataValidation.check( "a" )       # => true
dataValidation.check( "hello" )   # => false

[ <list of comma-separated double-quoted string literals> ]

Verifies that the string value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string [ "a", "b", "c" ]
EOS

dataValidation.check( "a" )       # => true
dataValidation.check( "hello" )   # => false
dataValidation.check( "b" )       # => true
dataValidation.check( "c" )       # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string ! [ "a", "b", "c" ]
EOS

dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => true
dataValidation.check( "b" )       # => false
dataValidation.check( "c" )       # => false

%( <list of space-separated unquoted string values> )

Verifies that the string value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string %( a b c )
EOS

dataValidation.check( "a" )       # => true
dataValidation.check( "hello" )   # => false
dataValidation.check( "b" )       # => true
dataValidation.check( "c" )       # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string ! %( a b c )
EOS

dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => true
dataValidation.check( "b" )       # => false
dataValidation.check( "c" )       # => false

empty

Verifies that the string value is empty.

empty is the opposite of something.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string empty
EOS

dataValidation.check( "a" )       # => false
dataValidation.check( "" )        # => true

something

Verifies that the string value is not empty, it needs to contain at least one character.

something is the opposite of emtpy.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string something
EOS

dataValidation.check( "a" )       # => true
dataValidation.check( "abc" )     # => true
dataValidation.check( " " )       # => true
dataValidation.check( "" )        # => false

blank

Verifies that the string value is blank.
A string is blank if it is empty or contains whitespace characters only.
This is equivalent to the method ‘blank?’ defined in Rail’s ActiveSupport extension for String class.

blank is the opposite of present.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string blank
EOS

dataValidation.check( nil )       # => false
dataValidation.check( "" )        # => true
dataValidation.check( " " )       # => true
dataValidation.check( "\t" )      # => true
dataValidation.check( "\n" )      # => true
dataValidation.check( "\n \t" )   # => true
dataValidation.check( "a" )       # => false

present

Verifies that the string value is present.
A string is present if it is not empty and contains not only whitespace characters.
This is equivalent to the method ‘present?’ defined in Rail’s ActiveSupport extension for String class.

present is the opposite of blank.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string present
EOS

dataValidation.check( nil )       # => false
dataValidation.check( "" )        # => false
dataValidation.check( " " )       # => false
dataValidation.check( "\t" )      # => false
dataValidation.check( "\n" )      # => false
dataValidation.check( "\n \t" )   # => false
dataValidation.check( "a" )       # => true

/regexp/

Verifies that the string value matches the given regular expression.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string /HELLO/
EOS

dataValidation.check( "HELLO" )   # => true
dataValidation.check( "hello" )   # => false
dataValidation.check( "Hello" )   # => false

You can extend the regular expression with option ‘i’ to perform a case insensitive match.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string /HELLO/i
EOS

dataValidation.check( "HELLO" )   # => true
dataValidation.check( "hello" )   # => true
dataValidation.check( "Hello" )   # => true

length <length specification>

Verifies that the length of the string value is in accordance with the stated specification.

The length specification can be one of the following:

The parameters compareValue, minValue and maxValue have to be integer literals.

== compareValue

Verifies that the length is equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string length == 3
EOS

dataValidation.check( "a" )      # => false
dataValidation.check( "abc" )    # => true
dataValidation.check( "abcd" )   # => false

!= compareValue

Verifies that the length is not equal to compareValue.

< maxValue

Verifies that the length is less than maxValue.

<= maxValue

Verifies that the length is less than or equal to maxValue.

> minValue

Verifies that the length is greater than minValue.

>= minValue

Verifies that the length is greater than or equal to minValue.

> minValue < maxValue

Verifies that the length is greater than minValue and less than maxValue.

> minValue <= maxValue

Verifies that the length is greater than minValue and less than or equal to maxValue.

>= minValue < maxValue

Verifies that the length is greater than or equal to minValue and less than maxValue.

>= minValue <= maxValue

Verifies that the length is greater than or equal to minValue and less than or equal to maxValue.

< maxValue > minValue

Verifies that the length is less than maxValue and greater than minValue.

<= maxValue > minValue

Verifies that the length is less than or equal maxValue and greater than minValue.

< maxValue >= minValue

Verifies that the length is less than maxValue and greater than or equal minValue.

<= maxValue >= minValue

Verifies that the length is less than or equal to maxValue and greater than or equal to minValue.

( minValue .. maxValue )

Verifies that the length is within the specified range, that means greater than or equal to minValue and less than or equal to maxValue.

( minValue .. )

Verifies that the length is greater than or equal to minValue.

( .. maxValue )

Verifies that the length is less than or equal to maxValue.

A contrived example

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_string present ! length (2..3) != "a" ! == "b" ! %( aaaa bbbb ) ! /\\Aab/
EOS

dataValidation.check( nil    )   # => false
dataValidation.check( "a"    )   # => false
dataValidation.check( "b"    )   # => false
dataValidation.check( "c"    )   # => true 
dataValidation.check( "ba"   )   # => false
dataValidation.check( "aaaa" )   # => false
dataValidation.check( "bbbb" )   # => false
dataValidation.check( "abbb" )   # => false
dataValidation.check( "babb" )   # => true 
dataValidation.check( "    " )   # => false

@t_int

The value is required to be of type Integer.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( "a" )       # => false
dataValidation.check( 3 )         # => true
dataValidation.check( 4 )         # => true

Value restricting predicates for @t_int

The parameters compareValue, minValue and maxValue have to be integer literals.

== compareValue

Verifies that the integer value is equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int == 4
EOS

dataValidation.check( 3 )         # => false
dataValidation.check( 4 )         # => true

!= compareValue

Verifies that the integer value is not equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int != 4
EOS

dataValidation.check( 4 )         # => false
dataValidation.check( 5 )         # => true

The result is the same as if you negate the == predicate:

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int ! == 4
EOS

< maxValue

Verifies that the integer value is less than maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int < 4
EOS

dataValidation.check( 3 )         # => true
dataValidation.check( 4 )         # => false

<= maxValue

Verifies that the integer value is less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int <= 4
EOS

dataValidation.check( 3 )         # => true
dataValidation.check( 4 )         # => true
dataValidation.check( 5 )         # => false

> minValue

Verifies that the integer value is greater than minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int > 4
EOS

dataValidation.check( 4 )         # => false
dataValidation.check( 5 )         # => true

>= minValue

Verifies that the integer value is greater than or equal to minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int >= 4
EOS

dataValidation.check( 3 )         # => false
dataValidation.check( 4 )         # => true
dataValidation.check( 5 )         # => true

( minValue .. maxValue )

Verifies that the integer value is within the specified range, that means greater than or equal to minValue and less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int (4..7)
EOS

dataValidation.check( 3 )         # => false
dataValidation.check( 4 )         # => true
dataValidation.check( 5 )         # => true
dataValidation.check( 7 )         # => true
dataValidation.check( 8 )         # => false

( minValue .. )

Verifies that the integer value is greater than or equal to minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int (4..)
EOS

dataValidation.check( 3 )         # => false
dataValidation.check( 4 )         # => true
dataValidation.check( 8 )         # => true

( .. maxValue )

Verifies that the integer value is less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int (..7)
EOS

dataValidation.check( 3 )         # => true
dataValidation.check( 7 )         # => true
dataValidation.check( 8 )         # => false

[ <list of comma-separated integer literals> ]

Verifies that the integer value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int [ 5, 7, 11 ]
EOS

dataValidation.check( 4 )    # => false
dataValidation.check( 5 )    # => true
dataValidation.check( 7 )    # => true
dataValidation.check( 11 )   # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int ! [ 5, 7, 11 ]
EOS

dataValidation.check( 4 )    # => true
dataValidation.check( 5 )    # => false
dataValidation.check( 7 )    # => false
dataValidation.check( 11 )   # => false

%( <list of space-separated integer literals> )

Verifies that the integer value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int %( 5 7 11 )
EOS

dataValidation.check( 4 )    # => false
dataValidation.check( 5 )    # => true
dataValidation.check( 7 )    # => true
dataValidation.check( 11 )   # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blcklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int ! %( 5 7 11 )
EOS

dataValidation.check( 4 )    # => true
dataValidation.check( 5 )    # => false
dataValidation.check( 7 )    # => false
dataValidation.check( 11 )   # => false

odd

Verifies that the integer value is an odd number.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int odd
EOS

dataValidation.check( 4 )    # => false
dataValidation.check( 5 )    # => true

even

Verifies that the integer value is an even number.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_int even
EOS

dataValidation.check( 4 )    # => true
dataValidation.check( 5 )    # => false

@t_float

The value is required to be of type Float.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( 4.0 )       # => true
dataValidation.check( 4.1 )       # => true

Value restricting predicates for @t_float

The parameters compareValue, minValue and maxValue have to be floating-point literals.

== compareValue

Verifies that the floating-point value is equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float == 4.0
EOS

dataValidation.check( 4.0 )       # => true
dataValidation.check( 4.1 )       # => false

!= compareValue

Verifies that the floating-point value is not equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float != 4.0
EOS

dataValidation.check( 4.0 )    # => false
dataValidation.check( 4.1 )    # => true

The result is the same as if you negate the == predicate:

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float ! == 4.0
EOS

< maxValue

Verifies that the floating-point value is less than maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float < 4.0
EOS

dataValidation.check( 3.9 )     # => true
dataValidation.check( 4.0 )     # => false

<= maxValue

Verifies that the floating-point value is less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float <= 4.0
EOS

dataValidation.check( 3.9 )     # => true
dataValidation.check( 4.0 )     # => true
dataValidation.check( 4.1 )     # => false

> minValue

Verifies that the floating-point value is greater than minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float > 4.0
EOS

dataValidation.check( 4.0 )     # => false
dataValidation.check( 4.1 )     # => true

>= minValue

Verifies that the floating-point value is greater than or equal to minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float >= 4.0
EOS

dataValidation.check( 3.9 )     # => false
dataValidation.check( 4.0 )     # => true

( minValue .. maxValue )

Verifies that the floating-point value is within the specified range, that means greater than or equal to minValue and less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float (4.1..7.2)
EOS

dataValidation.check( 4.1 )     # => true
dataValidation.check( 7.2 )     # => true
dataValidation.check( 7.3 )     # => false

( minValue .. )

Verifies that the floating-point value is greater than or equal to minValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float (4.1..)
EOS

dataValidation.check( 4.0 )     # => false
dataValidation.check( 4.1 )     # => true
dataValidation.check( 9.9 )     # => true

( .. maxValue )

Verifies that the floating-point value is less than or equal to maxValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float (..7.2)
EOS

dataValidation.check( 7.2 )     # => true
dataValidation.check( 7.3 )     # => false

[ <list of comma-separated floating-point literals> ]

Verifies that the floating-point value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float [ 5.3, 7.1, 11.8 ]
EOS

dataValidation.check( 4.9 )      # => false
dataValidation.check( 5.3 )      # => true
dataValidation.check( 7.1 )      # => true
dataValidation.check( 11.8 )     # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float ! [ 5.3, 7.1, 11.8 ]
EOS

dataValidation.check( 4.9 )      # => true
dataValidation.check( 5.3 )      # => false
dataValidation.check( 7.1 )      # => false
dataValidation.check( 11.8 )     # => false

%( <list of space-separated integer literals> )

Verifies that the floating-point value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float %( 5.3 7.1 11.8 )
EOS

dataValidation.check( 4.9 )      # => false
dataValidation.check( 5.3 )      # => true
dataValidation.check( 7.1 )      # => true
dataValidation.check( 11.8 )     # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_float ! %( 5.3 7.1 11.8 )
EOS

dataValidation.check( 4.9 )      # => true
dataValidation.check( 5.3 )      # => false
dataValidation.check( 7.1 )      # => false
dataValidation.check( 11.8 )     # => false

@t_numeric

The value is required to be of type Numeric. Type Integer and type Float are both of type Numeric.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_numeric
EOS

dataValidation.check( nil )     # => false
dataValidation.check( "a" )     # => false
dataValidation.check( 4 )       # => true
dataValidation.check( 4.0 )     # => true

@t_symbol

The value is required to be of type Symbol.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( :a )        # => true
dataValidation.check( :sym )      # => true

Value restricting predicates for @t_symbol

If one of the following predicates requires a compareValue, then it has to be noted as a symbol literal.

== compareValue

Verifies that the symbol value is equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol == :sym
EOS

dataValidation.check( :newsym )   # => false
dataValidation.check( :sym )      # => true

!= compareValue

Verifies that the symbol value is not equal to compareValue.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol != :sym
EOS

dataValidation.check( :newsym )   # => true
dataValidation.check( :sym )      # => false

[ <list of comma-separated symbol literals> ]

Verifies that the symbol value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol [ :a, :b, :")" ]
EOS

dataValidation.check( :a )       # => true
dataValidation.check( :sym )     # => false
dataValidation.check( :b )       # => true
dataValidation.check( :")" )     # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol ! [ :a, :b, :")" ]
EOS

dataValidation.check( :a )       # => false
dataValidation.check( :sym )     # => true
dataValidation.check( :b )       # => false
dataValidation.check( :")" )     # => false

%( <list of space-separated symbol values, without leading colon> )

Verifies that the symbol value is included in the given list.

The list acts like an allowlist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol %( a b ")" )
EOS

dataValidation.check( :a )       # => true
dataValidation.check( :sym )     # => false
dataValidation.check( :b )       # => true
dataValidation.check( :")" )     # => true

If you negate this predicate then it verifies that the value is excluded from the given list.

In this case the list acts like a blocklist.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol ! %( a b ")" )
EOS

dataValidation.check( :a )       # => false
dataValidation.check( :sym )     # => true
dataValidation.check( :b )       # => false
dataValidation.check( :")" )     # => false

/regexp/

Verifies that the symbol value matches the given regular expression.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol /MYSYMBOL/
EOS

dataValidation.check( :MYSYMBOL )   # => true
dataValidation.check( :mysymbol )   # => false
dataValidation.check( :Mysymbol )   # => false

You can extend the regular expression with option ‘i’ to perform a case insensitive match.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol /MYSYMBOL/i
EOS

dataValidation.check( :MYSYMBOL )   # => true
dataValidation.check( :mysymbol )   # => true
dataValidation.check( :Mysymbol )   # => true

length <length specification>

Verifies that the length of the symbol value is in accordance with the stated specification.

The length specification is the same as already described for @t_string. Please look it up there.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_symbol length == 3
EOS

dataValidation.check( :a )      # => false
dataValidation.check( :abc )    # => true
dataValidation.check( :abcd )   # => false

@t_true

The value is required to be of type TrueClass.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_true
EOS

dataValidation.check( nil )       # => false
dataValidation.check( "a" )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( false )     # => false
dataValidation.check( true )      # => true

@t_false

The value is required to be of type FalseClass.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_false
EOS

dataValidation.check( nil )       # => false
dataValidation.check( "a" )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( false )     # => true
dataValidation.check( true )      # => false

@t_bool

The value is required to be either of type TrueClass or of type FalseClass.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_bool
EOS

dataValidation.check( nil )       # => false
dataValidation.check( "a" )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( false )     # => true
dataValidation.check( true )      # => true

@t_nil

The value is required to be of type NilClass.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_nil
EOS

dataValidation.check( nil )       # => true
dataValidation.check( "a" )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( 4.0 )       # => false
dataValidation.check( false )     # => false
dataValidation.check( true )      # => false

@t_any

The value can be of any type. It is required to be of type Object.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = @t_any
EOS

dataValidation.check( nil )       # => true
dataValidation.check( "a" )       # => true
dataValidation.check( 4 )         # => true
dataValidation.check( 4.0 )       # => true
dataValidation.check( false )     # => true
dataValidation.check( true )      # => true
dataValidation.check( {} )        # => true
dataValidation.check( [] )        # => true

Literal value

Literal values can be used as a rule specification or as a value specification for the value part of a hash.

The following literals are available:

  • string literals
  • integer literals
  • floating-point literals
  • symbol literals
  • keyword literals

String literals

dataValidation = GdsDataValidation.create( <<-EOS )
schema = "hello"
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => true

dataValidation = GdsDataValidation.create( <<-EOS )
male = :title : "Mr.", :name : @t_string
EOS

dataValidation.check( { title: "Mrs.", name: "John" } )  # => false
dataValidation.check( { title: "Mr.", name: "John" } )   # => true

Integer literals

dataValidation = GdsDataValidation.create( <<-EOS )
schema = 4
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => true
dataValidation.check( 4.0 )       # => true     , true because of Ruby's type coercion
dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => false

Floating-point literals

dataValidation = GdsDataValidation.create( <<-EOS )
schema = 4.0
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => true     , true because of Ruby's type coercion
dataValidation.check( 4.0 )       # => true
dataValidation.check( "a" )       # => false
dataValidation.check( "hello" )   # => false

Symbol literals

dataValidation = GdsDataValidation.create( <<-EOS )
schema = :on
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( :off )      # => false
dataValidation.check( :on )       # => true

Keyword literals

The following keyword literals are available:

  • true
  • false
  • nil
  • null

In Ruby nil and null are considered as nil.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = null
EOS

dataValidation.check( nil )       # => true
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( :on )       # => false

Disjunction of basic data type specifications and literal values

You can define disjunctions of basic data type specifications and literal values and use it as a rule specification or as a value specification for the value part of a hash.

You have to use the logical or operator ‘|’ to create the disjunction.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = true | @t_int | @t_string
EOS

dataValidation.check( nil )       # => false
dataValidation.check( 4 )         # => true
dataValidation.check( "a" )       # => true
dataValidation.check( true )      # => true

dataValidation = GdsDataValidation.create( <<-EOS )
schema = :special : true | @t_int | :on | @t_string
EOS

dataValidation.check( { special: nil  } )   # => false
dataValidation.check( { special: 4    } )   # => true
dataValidation.check( { special: "a"  } )   # => true
dataValidation.check( { special: true } )   # => true
dataValidation.check( { special: :on  } )   # => true

You can negate the whole disjunction using the logical not operator ‘!’, but then you have to define the disjunction within parentheses.

dataValidation = GdsDataValidation.create( <<-EOS )
schema = ! ( true | @t_int | @t_string | 4.3 )
EOS

dataValidation.check( nil )       # => true
dataValidation.check( 4 )         # => false
dataValidation.check( "a" )       # => false
dataValidation.check( true )      # => false
dataValidation.check( 4.3 )       # => false
dataValidation.check( 4.4 )       # => true

Rule reference

A rule reference makes an indirection to another rule by stating the rule name of the other rule.

<rule name>

This is necessary, when you want to define schemas for nested hashes. If the value part of a key-value pair is another hash (sub-hash), then you have to use a rule reference for the value and you have to define the schema for the sub-hash with this referenced rule.

dataValidation = GdsDataValidation.create( <<-EOS )
person = :name : @t_string, :address : address
address = :street : @t_string, :zip : @t_int, :city : @t_string
EOS

The value for the key :address of the person schema is referencing the rule address which defines the schema for the sub-hash.

Sometimes using a rule reference as a rule definition could be suitable for development and testing. It behaves like an additional layer of indirection and serves as a decision point or single point of change to switch between different (sub) schema definitions.

dataValidation = GdsDataValidation.create( <<-EOS )
company = :name : @t_string, :visitor_address : address, :delivery_address : address, :employees : person
person = :name : @t_string, :address : address
address = address2
address1 = :street : @t_string, :zip : @t_int, :city : @t_string
address2 = :street : @t_string, :zip : @t_int, :city : @t_string, :country : @t_string
address3 = :street : @t_string, :street_number : @t_int, :zip : @t_int, :city : @t_string, :country : @t_string
EOS

Practical Examples

Rule alternatives

Use rule alternatives to let your validation recognize different kinds of forms of data.

In the following validation the attribute :payment_type acts like a discriminator to identify the type of payment.
If the payment is of type ‘cash’ or ‘cheque’, then only the attribute :amount has to be available.
If the payment is of type ‘card’ then in addition to :amount also the attribute :card_number has to be available.

dataValidation = GdsDataValidation.create( <<-EOS )
  payment = 
    :payment_type : @t_string %( cash cheque ),
    :amount       : @t_int
    / 
    :payment_type : @t_string %( card ),
    :card_number  : @t_int
    :amount       : @t_int
EOS

Web services

The following validation is used in production for a small web service which is parsing GDS (General Data Structure) expressions and pretty printing the resulting hash/array structure for various languages and styles. Actually this web service is used by the GDS Converter.
The parameters of the REST request have to comply with a certain schema:

There is a top-level hash containing one key-value pair with the key :gdseval4 and a value which is a sub-hash. This sub-hash contains key-value pairs with the keys :gdsexpr, :language and :style. The value for the key :gdsexpr defines the GDS expression, the value for the key :language defines the destination language and the :style defines the style of the pritty print. For the langages ‘ruby’, ‘elixir’, ‘python’ and ‘json’ there are the styles ‘compact’, ‘semiverbose’ and ‘verbose’ available. For the languages ‘xml’ and ‘yaml’ there is only the style ‘default’ available.

The following schema defines this specification for the request parameters.

GdsEvalValidation = GdsDataValidation.create(<<EOS)
schema   = :gdseval4 : gdseval4
gdseval4 =
  :gdsexpr  : @t_string,
  :language : @t_string %( ruby elixir python json ),
  :style    : @t_string %( compact semiverbose verbose )
  /
  :gdsexpr  : @t_string,
  :language : @t_string %( xml yaml ),
  :style    : @t_string %( default )
EOS

GdsEvalValidation.check( { gdseval4: { gdsexpr: 'key value', language: 'ruby', style: 'compact' } } )     # => true
GdsEvalValidation.check( { gdseval4: { gdsexpr: 'key value', language: 'xml', style: 'default' } } )      # => true
GdsEvalValidation.check( { gdseval4: { gdsexpr: 'key value', language: 'python', style: 'default' } } )   # => false
GdsEvalValidation.check( { gdseval4: { gdsexpr: 'key value', language: 'yaml', style: 'verbose' } } )     # => false

The web service expects a JSON object in the body of a REST HTTP POST request. A Rails or Sinatra web application loads this JSON data into the params hash, which is implemented as a hash with indifferent access, where the key as a Symbol (e.g. :gdsexpr) and key as a String (e.g. “gdsexpr”) is considered to be the same.

The validation can check straight this params hash for validity.

result = GdsEvalValidation.check( params )     # => true or false

In Rails usually this validation check is done inside an Action Controller.

Self-referential schema definitions

The following is an example for a self-referential schema definition.

dataValidation = GdsDataValidation.create( <<-EOS )
  person = @maybe :name : @t_string, :farther : person, :mother : person
EOS

dataValidation.check( { name: 'John', farther: nil, mother: nil } )
    # => true
dataValidation.check( { name: 'John', 
                        farther: { name: 'Berry', farther: nil, mother: nil }, 
                        mother: nil } )                                             
    # => true
dataValidation.check( { name: 'John',
                        farther: { name: 'Berry',
                                   farther: { name: "Robert", farther: nil, mother: nil },
                                   mother: nil },
                        mother: nil } )
    # => true
dataValidation.check( { name: 'John',
                        farther: { name: 'Berry', farther: nil, mother: nil },
                        mother:  { name: 'Mary', 
                                   farther: { name: "Martin", farther: nil, mother: nil }, 
                                   mother:  { name: "Olivia", farther: nil, mother: nil } } } )
    # => true
dataValidation.check( { name: 'John', 
                        farther: { name: 'Berry', farther: nil, mother: 2 },
                        mother: nil } )
    # => false

Contrived Examples

dataValidation = GdsDataValidation.create( <<-EOS )
  schema = @t_string  present  ! length (2..3)  != "a"  ! == "b"  ! %( aaaa bbbb )  ! /\\Aab/
EOS

dataValidation.check( nil )     # => false
dataValidation.check( "a" )     # => false
dataValidation.check( "b" )     # => false
dataValidation.check( "c" )     # => true
dataValidation.check( "ba" )    # => false
dataValidation.check( "aaaa" )  # => false
dataValidation.check( "bbbb" )  # => false
dataValidation.check( "abbb" )  # => false
dataValidation.check( "babb" )  # => true
dataValidation.check( "    " )  # => false

dataValidation = GdsDataValidation.create( <<-EOS )
  schema = @maybe @strict :number : @t_int, :name : @t_string
EOS

dataValidation.check( nil )                                          # => true
dataValidation.check( { number: 4, name: "a" } )                     # => true
dataValidation.check( { number: 4, name: "a", unwanted: "bang!" } )  # => false

dataValidation = GdsDataValidation.create( <<-EOS )
  schema = :special : true | @t_int | :on | @t_string
EOS

dataValidation.check( { special: nil   } )  # => false
dataValidation.check( { special: 4     } )  # => true
dataValidation.check( { special: "a"   } )  # => true
dataValidation.check( { special: true  } )  # => true
dataValidation.check( { special: false } )  # => false
dataValidation.check( { special: :on   } )  # => true
dataValidation.check( { special: :off  } )  # => false

dataValidation = GdsDataValidation.create( <<-EOS )
  schema = ! ( true | @t_int | @t_string | 4.3 )
EOS

dataValidation.check( nil   )  # => true
dataValidation.check( 4     )  # => false
dataValidation.check( "a"   )  # => false
dataValidation.check( true  )  # => false
dataValidation.check( false )  # => true
dataValidation.check( 4.3   )  # => false
dataValidation.check( 4.4   )  # => true

Ruby Gem

You can find it on RubyGems.org:

rubygems.org/gems/gds-data-validation

Source Code

You can find the source code on GitHub:

github.com/uliramminger/gds-data-validation