983 handle missing external dictionaries

[RakeshBobba03]

This PR fixes Issue #983, where CORE would crash with an unhandled MissingDataError if an external dictionary (e.g. MedDRA) could not be read. The change wraps dictionary installation in fill_cache_with_dictionaries with a try/except MissingDataError, logs a warning that the specific dictionary/path could not be loaded, skips installing that dictionary into the cache, and continues validation. As a result, validation runs complete and produce reports, while any rules that later fail due to missing dictionaries are surfaced as EXECUTION_ERROR at the rule level instead of terminating the entire process.

[RamilCDISC]

I think it will be nice to have a unit test for such cases. Could you please add a test? If you can think of some other cases too you can add those too.

[gerrycampion]

@RakeshBobba03 can you also provide an output report that shows the results.

[RakeshBobba03]

can you also provide an output report that shows the results.

@gerrycampion I ran the same validation twice (before and after the fix) with:

python core.py validate -s sdtmig -v 3.4 --dataset-path tests\resources\datasets\ae.xpt --meddra .\bad_meddra -l info -of json -o output_xxxxxx.json

For the “before” run I redirected stdout/stderr to logs_before.txt; for the “after” run I used logs_after.txt and wrote the report to output_after.json.

logs_before.txt shows the run crashing with an unhandled MissingDataError during dictionary load and no report. logs_after.txt shows a WARNING that MedDRA at .\bad_meddra could not be loaded and will be skipped, then validation continues and finishes normally.

There is no “before” report (run crashed). output_after.json is the full JSON report produced after the fix.

With the fix, rules that fail at run time (including those that need the missing dictionary) are reported with EXECUTION_ERROR for that rule/dataset in the JSON (e.g. “rule evaluation error - operation failed”) instead of the whole run stopping. You can see those entries in output_after.json.

logs_after.txt
logs_before.txt
output_after.json

[gerrycampion]

output_after.json

@RakeshBobba03
Which rules here are failing because of meddra? I see "rule evaluation error"s but none of them have descriptive enough text to indicate that it is because the external dictionary file could not be found.

[RakeshBobba03]

Which rules here are failing because of meddra? I see "rule evaluation error"s but none of them have descriptive enough text to indicate that it is because the external dictionary file could not be found.

I used that run as an example to show the fix: with a bad MedDRA path the engine no longer crashes, it logs a warning, and it still produces a full report.
The exact scenario (single AE dataset, no DM/SV/TV, no define.xml) wasn’t meant to show MedDRA-specific rule failures. In that example, all the “rule evaluation error - operation failed” entries come from other issues (e.g. missing domains like DM/TV/SV, missing columns, or missing define.xml), not from the missing dictionary. The only place the missing MedDRA dictionary is called out in that run is the WARNING at the start of logs_after.txt.
So the files illustrate “bad path → no crash, warning logged, report generated”; they don’t illustrate which rules would fail specifically because MedDRA wasn’t loaded.

[RamilCDISC]

All seems okay to me. @gerrycampion could you please let us know if you are okay with the PR too? Then i will do a final validation and approve.