Recipe: Handle Schema and Type Drift
Freeze inferred schemas, count type mismatches, and decide when to coerce or fail.
If you do not provide a schema, BlazeRules infers rule-referenced field types from the first evaluated batch. After that, the engine treats the schema as fixed.
Example:
{"account_age_days": 42}If a later record sends:
{"account_age_days": "forty-two"}the value is a type mismatch. The configured type_mismatch_mode decides what happens:
| Mode | Behavior |
|---|---|
NULL_ON_TYPE_ERROR | Treat the field as null for that row; normal comparisons return false. |
COERCE | Try lightweight coercion such as "42" to an integer. |
HARD_FAIL_TYPE | Throw a structured type error for the batch. |
Production default:
cfg = blazerules.EngineConfig()
cfg.ingest_error_mode = blazerules.IngestErrorMode.SKIP_AND_COUNT
cfg.type_mismatch_mode = blazerules.TypeMismatchMode.NULL_ON_TYPE_ERRORCLI equivalent: run the same policy in a shell script
python - <<'PY'
import sys
import blazerules
cfg = blazerules.EngineConfig()
cfg.ingest_error_mode = blazerules.IngestErrorMode.SKIP_AND_COUNT
cfg.type_mismatch_mode = blazerules.TypeMismatchMode.NULL_ON_TYPE_ERROR
engine = blazerules.RuleEngine(cfg)
engine.load_rules("rules.yaml")
result = engine.evaluate_ndjson(sys.stdin.buffer.read())
print(result.messages_processed, result.messages_skipped, result.error_counts)
PYWatch:
result = engine.evaluate_ndjson(payload)
print(result.error_counts)
print(result.error_samples)CLI equivalent: print drift diagnostics from stdin
cat events.ndjson | python drift_check.pydrift_check.py is the Python snippet above with payload = sys.stdin.buffer.read().
For important fields, pin the type in fields: so inference cannot guess differently across environments:
fields:
account_age_days: {type: int32, nullable: false}
amount: {type: float32, nullable: false}
event_ts_ms: {type: timestamp_ms, nullable: false}