Recipe: Plain-Text Logs

Convert unstructured text lines into NDJSON records with a message field.

BlazeRules ingests records, not arbitrary byte streams. For plain stdout/stderr or terminal text, wrap each line into JSON before sending it to stdin, HTTP, or file-tail input.

Wrap stdin text

some_command 2>&1 | \
  python -c 'import json,sys,time
for line in sys.stdin:
    print(json.dumps({"ts_ms": int(time.time()*1000), "source": "terminal", "message": line.rstrip()}), flush=True)' | \
  blazerules_agent --rules rules.yaml --input stdin --output stdout
Python equivalent: wrap process output and evaluate directly
import json
import subprocess
import time
import blazerules

engine = blazerules.RuleEngine()
engine.load_rules("rules.yaml")

proc = subprocess.Popen(["some_command"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
records = []
for line in proc.stdout:
    records.append(json.dumps({
        "ts_ms": int(time.time() * 1000),
        "source": "terminal",
        "message": line.rstrip(),
    }))

payload = ("\n".join(records) + "\n").encode()
result = engine.evaluate_ndjson(payload)
print(result.grouped_decision_indices())

Rule:

ruleset:
  rules:
    - id: terminal_error
      action: REVIEW
      conditions:
        field: message
        op: contains
        value: error

Wrap a file tail

tail -F /var/log/app.log | \
  python -c 'import json,sys
for line in sys.stdin:
    print(json.dumps({"service":"app","message":line.rstrip()}), flush=True)' | \
  blazerules_agent --rules rules.yaml --input stdin --output ndjson --output-path decisions.ndjson
Python equivalent: launch the wrapped tail pipeline
import subprocess

cmd = r'''
tail -F /var/log/app.log | \
python -c 'import json,sys
for line in sys.stdin:
    print(json.dumps({"service":"app","message":line.rstrip()}), flush=True)' | \
blazerules_agent --rules rules.yaml --input stdin --output ndjson --output-path decisions.ndjson
'''

subprocess.run(["bash", "-lc", cmd], check=True)

Why Wrap?

Wrapping gives rules stable fields:

  • message
  • service
  • source
  • level
  • ts_ms
  • any parsed values you extract before calling BlazeRules.

If your logs are already structured JSON, skip this step and use stdin Logs, File Tail, or HTTP Logs directly.