REX - Rules Engine eXtended

REX is a rules engine designed to process complex conditions and actions using a structured JSON format for rule definitions. It allows for defining rules, conditions, and actions that are compiled into bytecode by the REX Compiler, then executed by the REX Engine. REX currently uses Redis as its fact store and event transport. Pub/Sub provides the default best-effort path; Redis Streams provides retained, ordered durable processing with recovery.

The default v4 execution contract evaluates each affected rule once per batch, against the same snapshot. Outputs are staged, conflicts reject the round, and committed outputs feed bounded subsequent rounds. See the M4 migration guide before recompiling v3 rulesets. M6 removed JavaScript execution from every contract; see the script migration guide before upgrading a legacy v3 deployment that used scripts.

Rulesets may opt into the v5 closed typed-fact contract with a top-level facts declaration map; see the typed fact migration guide. Condition leaves may opt into the v6 processing-time contract with for, which requires the predicate to remain true for a bounded duration; see the temporal rules guide. Rules may opt into v7 emit: "on_change" behavior to suppress outputs already present in persisted state; see the change-only emission guide.

Use the M5 authoring tools to explain, lint, test, replay, and compare v4/v5/v6/v7 batch rulesets offline. rexd --dry-run --bundle ... emits simulated results without creating Redis connections.

The planning documents have distinct roles:

• The foundation roadmap tracks the next stage of architecture, performance, reliability, and tooling work, with milestone checklists and acceptance criteria.
• The revival plan preserves the earlier revival milestones.
• The engine semantics audit is the source of truth for verified findings and their remediation status.
• The evolution reference preserves longer-term design context, open decisions, and ideas worth revisiting.

For the compiled-artifact format, compatibility contract, and upgrade guidance, see the bytecode compatibility guide. For platform, toolchain, and runtime expectations, see the compatibility matrix.

Features

• Define rules using JSON
• Support for various data types and comparison operations
• Logical and control flow instructions
• Action execution based on rules
• Offline explain, lint, scenario-test, replay, and comparison tools
• Durable event processing, recovery, and exact partition ownership
• Safe ruleset reload and rollback with retained artifact history
• Optional typed facts, temporal conditions, and change-only emission

Getting Started

Prerequisites

• Go 1.26.6 or higher
• A standalone Redis server reachable at localhost:6379 for the default configuration. Durable scripted transactions require Redis 7.0 or later; durable WATCH mode supports Redis 6.2 or later. Redis Cluster is unsupported.

Installation

Clone the repository:

git clone https://github.com/rgehrsitz/rex.git

Navigate to the project directory:

cd rex

Running the Executables

The REX repository includes five command-line programs: rexc, rexd, redis_setup, rex_stressor, and rule_gen. Release archives contain all five.

1. Compiler (rexc)

Purpose: The rexc executable is the compiler that translates rules defined in JSON format into bytecode instructions that the runtime engine can execute.

How to Build:

go build ./cmd/rexc

How to Run:

./rexc -rules <path_to_rules.json> [-output <path>] [-loglevel <level>] [-logoutput <output>]
./rexc validate -rules <path_to_rules.json>

Command-line options:

• -rules: (Required) Path to the input JSON file containing the rules.
• -output: (Optional) Compiled-bytecode path. Default is output.bytecode; ignored by validate.
• -loglevel: (Optional) Set log level. Valid values are panic, fatal, error, warn, info, debug, trace. Default is "info".
• -logoutput: (Optional) Set log output. Valid values are console or file. Default is "console".

Example:

./rexc validate -rules examples/rules2.ruleset.json
./rexc -rules examples/rules2.ruleset.json -output rules.bytecode -loglevel debug -logoutput file

2. Runtime Engine (rexd)

Purpose: The rexd executable is the runtime engine that processes the bytecode generated by rexc. It uses a Redis store for managing and updating facts in real-time based on the rules.

How to Build:

go build ./cmd/rexd

How to Run:

./rexd -config <path_to_config.json>

Command-line options:

• -config: (Optional) Path to the configuration file. If not specified, rexd will look for a file named rex_config.json in the current directory, $HOME/.rex, and /etc/rex.

Configuration File (rex_config.json): The configuration file is JSON. This is a minimal Pub/Sub example; see the complete example for durable, batch-bound, and compatibility settings.

{
  "bytecode_file": "output.bytecode",
  "logging": {
    "level": "debug",
    "output": "console",
    "time_format": "unixnano",
    "trace_conditions": true
  },
  "redis": {
    "address": "localhost:6379",
    "username": "",
    "password": "",
    "database": 0,
    "connect_timeout": "5s",
    "health_check_interval": "1s",
    "health_check_timeout": "500ms",
    "event_mode": "pubsub",
    "tls": {
      "enabled": false,
      "server_name": "",
      "ca_file": ""
    },
    "channels": ["weather", "system", "network", "energy", "water"]
  },
  "engine": {
    "priority_threshold": 1,
    "scripts_enabled": false,
    "max_actions_per_evaluation": 32,
    "max_event_hops": 16
  },
  "observability": {
    "enabled": false,
    "address": "127.0.0.1:8080"
  }
}

Every setting can be supplied as an environment variable by uppercasing its path, replacing dots with underscores, and adding REX_. For example, redis.password is REX_REDIS_PASSWORD and redis.tls.ca_file is REX_REDIS_TLS_CA_FILE. Environment variables override the configuration file, which overrides built-in defaults. Comma-separate REX_REDIS_CHANNELS values. See the M3 operations guide for TLS, readiness, routing, limits, and metrics. Redis Pub/Sub remains the default event mode. For retained delivery and restart recovery, set redis.event_mode to streams, configure redis.durable.stream, group, and consumer, and follow the M7 durable processing runbook. Production durable mode uses one serial processor per daemon and a standalone Redis commit domain. The default scripted transaction mode requires Redis 7.0 or later; set redis.durable.transaction_mode to watch for Redis 6.2 or later. Concurrent workers sharing one Redis process and Redis Cluster are unsupported.

Optional ruleset reload validates and archives a candidate before switching at an event boundary. Configure engine.reload and preserve its history directory as described in the reload runbook. Managed partitions must pass the ownership rollout before production use.

scripts_enabled remains only as an M6 migration tripwire: false is accepted, while true fails startup. See the M6 migration guide.

In legacy v3, engine.priority_threshold only controls the additional high-priority diagnostic emitted after a matching rule. It does not filter candidates or change their execution order.

Example:

./rexd -config cmd/rexd/rex_config.json

Fact Event Format

rexd consumes JSON objects that map fact keys to their JSON values. In Pub/Sub mode the producer publishes the object to a configured channel. In Streams mode the producer appends it as the payload field of the configured input stream. For example, the following event updates a numeric fact, a boolean fact, and a string fact without type conversion:

{
  "weather:temperature": 30.5,
  "weather:alert": true,
  "weather:status": "storm watch"
}

During migration, rexd also accepts the legacy key=value form. Its value is decoded as JSON when possible, so weather:status="storm watch" is a string and weather:alert=true is a boolean. New producers should publish the JSON-object format.

Evaluation Traces

rexd assigns each incoming event a trace_id. A JSON event with several facts keeps that ID for every fact update it contains. At the configured info level, the runtime emits structured records for fact_event_received, fact_event_decoded, rule_evaluation_candidates, rule_condition_evaluated, action_completed (or action_skipped / action_failed), and rule_evaluation_completed. Filter by trace_id to follow an event from Redis ingress through its rule and action outcomes.

These trace records identify facts, rules, action types, and targets, but deliberately omit arbitrary fact and action values. Other diagnostic records—including the existing high-priority rule message—may contain values, so use those logs only while investigating a trusted deployment.

Set logging.trace_conditions to false to omit the per-condition rule_condition_evaluated records. It defaults to true for compatibility. Candidate, action, and rule-completion summaries, warnings, and failures remain enabled at their configured log levels. Embedded callers can use Engine.SetConditionTracing(false) before processing events. Successful Redis publication diagnostics now use the configured structured logger at debug; they no longer write event payloads through the standard-library logger.

Health and Metrics

Set observability.enabled to true to expose local HTTP endpoints at observability.address (default: 127.0.0.1:8080):

• /healthz confirms that the daemon process is serving HTTP.
• /readyz returns 200 only while Redis connectivity and the configured event source are healthy. Durable mode also requires the partition ownership lease. It returns 503 during startup, disconnection, lease loss, and shutdown.
• /metrics emits Prometheus text-format counters and an event-processing latency histogram. Rule and action outcome labels use fixed value sets.

The endpoint is disabled by default so an upgrade does not unexpectedly open a port. Durable mode reports consumer-group lag, pending work, retries, and dead letters. Redis Pub/Sub has no retained queue, producer timestamp, or broker-side drop counter, so its queue metrics are emitted as NaN.

Cycle Safety

rexd limits each rule evaluation to engine.max_actions_per_evaluation actions (default: 32) and limits a chain of Rex-derived Redis events to engine.max_event_hops hops (default: 16). Derived updates carry an internal _rex envelope containing the existing trace ID and incremented hop; independent producers can continue sending the canonical JSON fact-object format unchanged. An event over its hop limit is rejected before rule evaluation.

These controls bound accidental feedback loops, but they do not make actions exactly-once. Rules and external action consumers should remain idempotent: use stable business keys, tolerate duplicate updates, and avoid non-idempotent side effects (such as creating a new record) without a deduplication key.

3. Redis Setup (redis_setup)

Purpose: The redis_setup executable initializes the Redis database with default values necessary for some testing of the REX system. It also provides a CLI for modifying values during debugging.

How to Build:

go build ./tools/redis_setup

How to Run:

./redis_setup

This tool doesn't have any command-line options. It connects to Redis at localhost:6379 by default.

After running, it provides an interactive CLI with the following command:

set <group:key> <value>

Example:

./redis_setup
Enter command (set <group:key> <value> or exit): set weather:temperature 30.5

4. Rule Generator (rule_gen)

Purpose: The rule_gen executable generates a large number of random rules in JSON format, which can be used for testing and benchmarking the REX system.

How to Build:

go build ./tools/rule_gen

How to Run:

./rule_gen [-rules <number_of_rules>] [-output <output_file.json>]

Command-line options:

• -rules: (Optional) Number of rules to generate. Default is 1000.
• -output: (Optional) Output file name. Default is "generated_ruleset.json".

Example:

./rule_gen -rules 1000 -output generated_ruleset.json

5. Pub/Sub Stressor (rex_stressor)

rex_stressor generates random weather fact updates for local Pub/Sub load testing. It writes directly to Redis and is outside the durable Streams contract.

go build ./tools/rex_stressor
./rex_stressor -redis localhost:6379 -rate 10

Usage

Workflow

1. Define your rules in a JSON file.
2. Use rexc to compile the rules into bytecode.
3. Set up your Redis instance and initialize it with redis_setup if needed.
4. Run rexd with the compiled bytecode to start the rules engine.
5. The engine will listen for updates from Redis, evaluate rules, and perform actions accordingly.

For a self-contained compiler -> Redis -> runtime smoke test, see the Docker Compose demo.

Releases

Pushing an annotated semantic-version tag such as v0.2.0 or v0.2.0-alpha from a reviewed main commit publishes versioned archives for Linux, macOS, and Windows, together with a SHA-256 manifest. Reviewed operator notes are prepended to GitHub-generated change notes. See the release guide for the tag and verification procedure.

Development

Code Structure

• cmd/rexc: Main application entry point for the compiler
• cmd/rexd: Main application entry point for the runtime engine
• pkg/compiler: Contains the bytecode compiler and related functions
• pkg/runtime: Contains the runtime engine and related functions
• pkg/store: Contains the Redis store implementation
• pkg/logging: Contains logging utilities
• tools/redis_setup: Redis setup and CLI tool
• tools/rex_stressor: Local Pub/Sub load generator
• tools/rule_gen: Random rule generation tool

Defining Rules

Rules are defined in a JSON format. Each rule consists of conditions and actions. Here's an example:

{
  "rules": [
    {
      "name": "rule-1",
      "conditions": {
        "all": [
          { "fact": "weather:temperature", "operator": "GT", "value": 30 },
          { "fact": "weather:humidity", "operator": "LT", "value": 40.01 }
        ]
      },
      "actions": [
        {
          "type": "updateStore",
          "target": "weather:temperature_warning",
          "value": "high"
        }
      ]
    }
  ]
}

JSON Structure

Rules are defined in a JSON object with:

• rules: a required non-empty array of rule objects.
• facts: an optional closed map of v5 typed fact declarations.

Rule Object

A rule object has the following properties:

• name: a unique string identifying the rule
• priority: optional non-negative integer indicating execution priority. Lower numbers execute first; omitted priorities default to 10.
• conditions: an object containing exactly one lowercase all or any array.
• actions: a non-empty array of action objects.
• emit: optional "on_change" behavior; omission retains repeated emission.

Condition Group

A condition group contains exactly one non-empty lowercase all or any array. Each array entry is either a condition object or another condition group.

Condition Object

A condition object has the following properties:

• fact: the fact name to evaluate.
• operator: EQ, NEQ, LT, LTE, GT, GTE, CONTAINS, or NOT_CONTAINS.
• value: the scalar value to compare against.
• for: an optional v6 processing-time duration.

Condition leaves and nested groups may appear in either order; the compiler canonicalizes them deterministically.

Fact names may contain colons, as shown throughout the examples. Durable mode reserves its configured stream names, names beginning with rex:durable:, and the private temporal-state prefix; see the ownership guide before assigning a managed partition.

Action Object

An action object has the following properties:

• type: the supported action type, updateStore. Unsupported action types are rejected during compilation.
• target: the fact to update.
• value: the scalar value to write.

Removed scripting capability

REX no longer accepts JavaScript definitions or {script} action values. The compiler rejects them for every contract, and the runtime rejects legacy v3 artifacts containing script opcodes before execution. Move calculations into producer-supplied facts or express them with declarative rules. See the M6 migration guide.

Execution Order

Actions will be executed in the order they are defined in the rule.

Fact and Value Data Types

Fact names are strings. Values can be strings surrounded by quotation marks (e.g. "fact_a"), booleans, or finite JSON numbers. V5 declarations bind each name to one of those scalar types and may explicitly permit null.

Priority Ties

Candidate rules execute in ascending priority order, so lower numbers run first. Rules with the same priority execute in their original ruleset order. Rule evaluation is sequential for each event batch.

Testing

The current-v3 semantics safety net runs authored scenarios and seeded differential checks against an independent AST interpreter. Embedded tools can use store.NewMemoryStore(initial) for JSON facts without a Redis service; snapshots and ordered publications are available for inspection. The batch-v4 corpus independently checks the new contract while retaining current-v3 expectations.

To run the tests:

go test ./...

License

This project is licensed under the MIT License - see the LICENSE file for details.