CODEWITHSUNDEEP
pythonpython-2.7jsonschema
Sachin Aryal
Sachin Aryal

Reputation: 801

jsonschema validation not returning error as expected?

I am using this schema, I was hoping for value-based conditional schema.

If app_name is "test" then property name should be required.

if app_name if "rest" then property ips should be required.

{
    "type": "object",
    "oneOf": [
        {
            "properties": {
                "app_name": {"enum": ["test"]}
            },
            "required": ["name"]
        },
        {
            "properties": {
                "app_name": {"enum": ["rest"]}
            },
            "required": ["ips"]
        },
    ],
    "properties": {
        "name": {"type": "string"},
        "ips": {
            "type": "array",
            "minItems": 1,
            "uniqueItems": True,
            "items": {
                "type": "string",
                "pattern": "[^ ]",
                "minLength": 1,
                "maxLength": 50
            }
        },
        "app_name": {
            "type": "string",
            "minLength": 1,
            "maxLength": 10,
            "enum": [
                "test",
                "rest"
            ]
        }
    },
    "required": [
        "app_name"
    ]
}

I am using following code 

formatted_data = {"app_name": "rest", "name": "test data"}
print jsonschema.exceptions.best_match(jsonschema.Draft4Validator(schema).iter_errors(formatted_data))

I got following validation error

'rest' is not one of ['test']

Failed validating 'enum' in schema[0]['properties']['app_name']: {'enum': ['test']}

On instance['app_name']: 'rest'

I am not sure if schema itself is invalid or there is a problem with library if self.

I am using

python 2.7

jsonschema 2.6.0

Upvotes: 4

Views: 7252

Answers (1)

PsychoFish
PsychoFish

Reputation: 1639

Ok, it seems there's a typo in schema. "True" instead of "true".

You've got:

"uniqueItems": True,

while, best to my knowledge, it should be (though it still may depend on schema validator implementation)

"uniqueItems": true,

(see: http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf section 5 JSON Values and in general https://www.json.org/ - JSON Schema is a JSON document and it conforms to JSON standard)

I've run it through .net online JSON schema validator at https://www.jsonschemavalidator.net/ and it immediately pointed out possible error with schema as above.

After typo correction it seems to work perfectly as per your comment from 2018/Nov/16:

As I was passing a value "rest" in app_name. I was expecting an error message "ips" field is required. – Sachin Aryal

Complete schema (please note the "examples" section - only last 2 examples will validate successfully against schema):

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "anyOf": [
        {
            "properties": {
                "app_name": {"enum": ["test"]}
            },
            "required": ["name"]
        },
        {
            "properties": {
                "app_name": {"enum": ["rest"]}
            },
            "required": ["ips"]
        },
    ],
    "properties": {
        "name": {"type": "string"},
        "ips": {
            "type": "array",
            "minItems": 1,
            "uniqueItems": true,
            "items": {
                "type": "string",
                "pattern": "[^ ]",
                "minLength": 1,
                "maxLength": 50
            }
        },
        "app_name": {
            "type": "string",
            "minLength": 1,
            "maxLength": 10,
            "enum": [
                "test",
                "rest"
            ]
        }
    },
    "required": [
        "app_name"
    ],
    "examples" : [
      {
        "app_name" : "rest",
      },
      {
        "app_name" : "test",
      },
      {
        "app_name" : "test",
        "ips" : [
         "something1",
         "something2"
        ]
      },
      {
        "app_name" : "rest",
        "name" : "qwerty"
      },
      {
        "app_name" : "test",
        "name" : "qwerty"
      },
      {
        "app_name" : "rest",
        "ips" : [
          "something1",
          "something2"
        ]
      }
    ]
}

Could you correct item in schema and give it a try with your tooling and let us know the result?

On top of that:

If you pass for validation JSON object like:

{
  "app_name" : "rest",
  "name" : "qwerty"
},

against your schema, where "oneOf" is used - validator will/should run object against all schemas in "oneOf" array to make sure it matches exactly one of provided schemas. Thus errors for "oneOf/0/" schema are valid - "app_name" : "rest" doesn't validate against defined enum. The validator has no clue what you meant by providing specific JSON for validation against schema, so if "allOf" (logical XOR) condition is not satisfied, I'd expect errors from all schemas the JSON was run against within "oneOf" array (even if these seem to be false-positives for you).

If some error message is missing, you might want to consider checking with/reporting to lib authors your exact case.

Hope it helped.

UPDATE

So it seems you are another one chasing meaningful errors ;-) Yes, that can be a pain when using logical operators.

For simple case like above, you can use for draft-07 if-then-else approach, however there's a caveat to it - see REMARK.

Schema first (note how I replaced "anyOf" with two "if-then"):

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "if": {
      "properties": {
        "app_name": {"enum": ["test"]}
      },      
    },
    "then" : { "required": ["name"] },
    "if" :  {
      "properties": {
        "app_name": {"enum": ["rest"]}
      },       
    },
    "then" : { "required": ["ips"]},
    "properties": {
        "name": {"type": "string"},
        "ips": {
            "type": "array",
            "minItems": 1,
            "uniqueItems": true,
            "items": {
                "type": "string",
                "pattern": "[^ ]",
                "minLength": 1,
                "maxLength": 50
            }
        },
        "app_name": {
            "type": "string",
            "minLength": 1,
            "maxLength": 10,
            "enum": [
                "test",
                "rest"
            ]
        }
    },
    "required": [
        "app_name"
    ],
    "examples" : [
      {
        "app_name" : "rest",
      },
      {
        "app_name" : "test",
      },
      {
        "app_name" : "test",
        "ips" : [
         "something1",
         "something2"
        ]
      },
      {
        "app_name" : "rest",
        "name" : "qwerty"
      },
      {
        "app_name" : "test",
        "name" : "qwerty"
      },
      {
        "app_name" : "rest",
        "ips" : [
          "something1",
          "something2"
        ]
      }
    ]
}

REMARK

Schema validator jsonschema.net tends to provide exact if-then-else errors in simple cases, when schema in "then" and "else" contains no nested "if-then" and consists of single statement/schema expression. However, whenever more complex case is structured, you may run into a general error message like: JSON does not match schema from 'then'. or JSON does not match schema from 'else'. without additional details (you'd need to check python output on your own). You can work around it to some point by properly shaping dependencies or sub-schemas, but with really complex schema, if you are after detailed error message, you may be facing the limit of validator implementation error messaging anyway. Please see also alternative schema 2 as an example in here: https://stackoverflow.com/a/53320222/2811843 (the whole point being to structure schema in a way, that if-then-else fails on a single keyword schema and rest of schema logic sits under other keywords)

Having said that, you may always check that approach with your tooling and if necessary file a report to your favourite lib authors about error messaging details for failed if-then-else schemas.

alternative with "dependencies"

Another alternative for your case is using "dependencies" keyword from draft-06, invert the initial logic and shape "definitions" nodes so it leads directly to unique error:

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "ips": {
            "type": "array",
            "minItems": 1,
            "uniqueItems": true,
            "items": {
                "type": "string",
                "pattern": "[^ ]",
                "minLength": 1,
                "maxLength": 50
            }
        },
        "app_name": {
            "type": "string",
            "minLength": 1,
            "maxLength": 10,
            "enum": [
                "test",
                "rest"
            ]
        }
    },
    "required": [
        "app_name"
    ],
    "dependencies" : {
     "ips" : {
        "properties": {
        "app_name": {"$ref":"#/definitions/allowed-app_name-value/rest"}
      },
     },
     "name" : {
      "properties": {
        "app_name": {"$ref":"#/definitions/allowed-app_name-value/test"}
      }, 
     }   
    },
    "definitions" : {
      "allowed-app_name-value" : {
        "test" : {
          "enum": ["test"]
        },
        "rest" : {
          "enum": ["rest"]
        }
      }
    },
    "examples" : [
      {
        "app_name" : "rest",
      },
      {
        "app_name" : "test",
      },
      {
        "app_name" : "test",
        "ips" : [
         "something1",
         "something2"
        ]
      },
      {
        "app_name" : "rest",
        "name" : "qwerty"
      },
      {
        "app_name" : "test",
        "name" : "qwerty"
      },
      {
        "app_name" : "rest",
        "ips" : [
          "something1",
          "something2"
        ]
      }
    ]
}

yet it's still a naming workaround aimed to identify exact error in a human readable way. For instance, the jsonschema .net will feed you with JSON line numbers and messages like documented in here: https://www.newtonsoft.com/jsonschema/help/html/JTokenIsValidWithValidationErrors.htm

Each tool has it's own error messaging approach. Please check JSON Schema team on github, as there's some work on unifying output for JSON Schema validation in progress for next draft.

Whenever analysing errors programatically, you may want to pay attention to error indexes (schemas are usually nested in schemas) if they appear, on which line the error occured etc..

Upvotes: 4

Related Questions