Skip to content

Latest commit

 

History

History
336 lines (247 loc) · 7.72 KB

README.md

File metadata and controls

336 lines (247 loc) · 7.72 KB

Rules Compiler (Python)

Python API for compiling AdGuard filter rules.

Installation

# Install from source
cd src/rules-compiler-python
pip install -e .

# Install with development dependencies
pip install -e ".[dev]"

Prerequisites

Requirement Version Notes
Python 3.9+ Core language
Node.js 18+ For compilation engine

CLI Usage

# Use default config (compiler-config.json)
rules-compiler

# Use specific configuration file
rules-compiler -c compiler-config.yaml

# Compile and copy to rules directory
rules-compiler -c config.json -r

# Show version info
rules-compiler -v

# Enable debug output
rules-compiler -c config.yaml -d

# Show help
rules-compiler -h

# Disable validation before compilation
rules-compiler -c config.yaml --no-validate-config

# Fail on validation warnings
rules-compiler -c config.yaml --fail-on-warnings

CLI Options

Option Short Description
--config PATH -c Path to configuration file
--output PATH -o Path to output file
--copy-to-rules -r Copy output to rules directory
--format FORMAT -f Force format (json, yaml, toml)
--version -v Show version information
--debug -d Enable debug output
--validate Validate configuration only (no compilation)
--validate-config Enable configuration validation before compilation (default: true)
--no-validate-config Disable configuration validation before compilation
--fail-on-warnings Fail compilation if configuration has validation warnings
--help -h Show help message

Python API

Basic Usage (Synchronous)

from rules_compiler import RulesCompiler

# Create compiler
compiler = RulesCompiler()

# Compile rules
result = compiler.compile("compiler-config.yaml", copy_to_rules=True)

if result.success:
    print(f"Compiled {result.rule_count} rules")
    print(f"Output: {result.output_path}")
else:
    print(f"Error: {result.error_message}")

Async/Await Usage (Python 3.9+)

The Python compiler now supports asynchronous operations for better performance in I/O-bound scenarios:

import asyncio
from rules_compiler import RulesCompiler

async def main():
    compiler = RulesCompiler()
    
    # Use async API for better performance
    result = await compiler.compile_async(
        "compiler-config.yaml",
        copy_to_rules=True
    )
    
    if result.success:
        print(f"Compiled {result.rule_count} rules")
        print(f"Hash: {result.hash_short()}")
        print(f"Time: {result.elapsed_formatted()}")

# Run async function
asyncio.run(main())

Parallel Processing with Async

Compile multiple configurations in parallel:

import asyncio
from rules_compiler import compile_rules_async

async def compile_all():
    configs = ["config1.yaml", "config2.yaml", "config3.yaml"]
    
    # Compile all configurations in parallel
    tasks = [compile_rules_async(config) for config in configs]
    results = await asyncio.gather(*tasks)
    
    for result in results:
        if result.success:
            print(f"{result.config_name}: {result.rule_count} rules")
        else:
            print(f"Failed: {result.error_message}")

asyncio.run(compile_all())

Async File Operations

Use async functions for file operations:

import asyncio
from rules_compiler import count_rules_async, compute_hash_async

async def analyze_file(path):
    # Count rules and compute hash in parallel
    count, hash_value = await asyncio.gather(
        count_rules_async(path),
        compute_hash_async(path)
    )
    
    print(f"File: {path}")
    print(f"Rules: {count}")
    print(f"Hash: {hash_value[:32]}...")

asyncio.run(analyze_file("rules.txt"))

Performance Considerations

  • Async APIs are recommended for:

    • Large file operations
    • Processing multiple configurations
    • Integration with async frameworks (FastAPI, aiohttp, etc.)

  • Sync APIs are simpler for:

    • Single compilation tasks
    • Simple scripts
    • Interactive use

Note: The async APIs require the aiofiles package for optimal performance. If not installed, they will fall back to running sync operations in a thread pool.

Reading Configuration

from rules_compiler import read_configuration, ConfigurationFormat

# Auto-detect format from extension
config = read_configuration("config.yaml")
print(f"Name: {config.name}")
print(f"Sources: {len(config.sources)}")

# Force specific format
config = read_configuration("config.txt", format=ConfigurationFormat.YAML)

Version Information

from rules_compiler import get_version_info

info = get_version_info()
print(f"Module: {info.module_version}")
print(f"Python: {info.python_version}")
print(f"Node.js: {info.node_version}")
print(f"Platform: {info.platform.os_name}")

Using the Compiler Class

from rules_compiler import RulesCompiler, ConfigurationFormat

compiler = RulesCompiler(debug=True)

# Read and inspect configuration
config = compiler.read_config("config.yaml")
print(f"Will compile {len(config.sources)} sources")

# Compile with options
result = compiler.compile(
    config_path="config.yaml",
    output_path="my-rules.txt",
    copy_to_rules=True,
    format=ConfigurationFormat.YAML,
)

# Access result details
print(f"Success: {result.success}")
print(f"Rules: {result.rule_count}")
print(f"Hash: {result.output_hash}")
print(f"Time: {result.elapsed_ms}ms")

Configuration Formats

JSON

{
  "name": "My Filter Rules",
  "version": "1.0.0",
  "sources": [
    { "name": "Local", "source": "./rules.txt", "type": "adblock" }
  ],
  "transformations": ["Deduplicate", "Validate"]
}

YAML

name: My Filter Rules
version: 1.0.0
sources:
  - name: Local
    source: ./rules.txt
    type: adblock
transformations:
  - Deduplicate
  - Validate

TOML

name = "My Filter Rules"
version = "1.0.0"
transformations = ["Deduplicate", "Validate"]

[[sources]]
name = "Local"
source = "./rules.txt"
type = "adblock"

Running Tests

cd src/rules-compiler-python

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run with coverage
pytest --cov=rules_compiler --cov-report=term-missing

# Run specific test file
pytest tests/test_config.py

# Run with verbose output
pytest -v

Type Checking

# Run mypy
mypy rules_compiler

Linting

# Run ruff
ruff check rules_compiler

# Auto-fix issues
ruff check --fix rules_compiler

API Reference

Classes

Class Description
RulesCompiler Main compiler class
CompilerResult Result of a compilation operation
CompilerConfiguration Configuration file model
FilterSource Source filter list definition
VersionInfo Component version information
PlatformInfo Platform-specific information

Enums

Enum Values
ConfigurationFormat JSON, YAML, TOML

Functions

Function Description
compile_rules() Compile filter rules (functional API)
read_configuration() Read configuration from file
detect_format() Detect format from file extension
to_json() Convert configuration to JSON
get_version_info() Get version information

License

GPLv3 - See LICENSE for details.