Get list of validation results for each rule/check?

[reinholdsson]

Hi,

Would it be possible to add support for returning (optionally) a dictionary or similar structure that maps each rule/check to its ValidationResult instance?

We have a use case for integrating dataframely's schema validation with Dagster asset checks. To enable this, we need a way to programmatically access each rule's validation result so we can dynamically map them to corresponding asset checks.

Currently, the validation output may look like:

ValidationError: 4 rules failed validation:
 - 'reasonable_bathroom_to_bedrooom_ratio' failed for 1 rows
 - 'minimum_zip_code_count' failed for 2 rows
 * Column 'zip_code' failed validation for 1 rules:
   - 'min_length' failed for 1 rows
 * Column 'num_bedrooms' failed validation for 1 rules:
   - 'nullability' failed for 2 rows

Hence, it would be great to have an API to obtain validation results in a structured mapping, e.g.:

{
    "reasonable_bathroom_to_bedrooom_ratio": ValidationResult(...),
    "minimum_zip_code_count": ValidationResult(...),
    "zip_code_min_length": ValidationResult(...),
    ...,
}

This would allow us to:

• Dynamically map each check to a Dagster asset check.
• Retrieve per-rule validation details programmatically, not just from the textual summary.

Thanks! :)

[AndreasAlbertQC]

Hey @reinholdsson ! Good suggestion, I think we can definitely do a better job of exposing this info. As of today, you can:

1. run .filter instead of .validate whenever there is any chance of failure.
2. This returns a FilterResult, which contains a FailureInfo

FailureInfo contains the information you are looking for, albeit only in a private member _lf, which is a dataframe where the rows are the rows you filtered and the columns for represent the checks (true = pass, false = fail, None=not evaluated). In our own internal code that uses dataframely, we have so far often converged on using this dataframe to infer the info we need. Clearly, that already hints we should either just make it public as is or enhance it a little and then make it public.
@borchero @delsner wdyt?