Skip to content

Add syslog adaptor benchmarks#3463

Draft
ruslandoga wants to merge 2 commits into
Logflare:mainfrom
ruslandoga:ruslandoga-conductor/syslog-adaptor-benchmarks
Draft

Add syslog adaptor benchmarks#3463
ruslandoga wants to merge 2 commits into
Logflare:mainfrom
ruslandoga:ruslandoga-conductor/syslog-adaptor-benchmarks

Conversation

@ruslandoga

@ruslandoga ruslandoga commented May 13, 2026

Copy link
Copy Markdown
Contributor

Please ignore this PR for now :)


Adds benchmark-only syslog profiling scripts under test/profiling.

The benchmarks compare current batcher routing, no-batcher per-message sends, and preformatted batch sends using a local RFC6587 octet-counting TCP sink.

The Broadway benchmark runs through real Broadway processors/batchers, BufferProducer, a local NimblePool TCP sender, and fake cache entries so it can run with mix run --no-start.

Validated with mix format and small smoke runs for both benchmark scripts.

Copy link
Copy Markdown
Contributor Author

Benchmark smoke results from this workspace. These used tiny inputs to validate the scripts and the RFC6587 sink wiring, so treat the relative ordering as a sanity check only, not as production throughput data.

Direct formatter + TCP send:

Command:
SYSLOG_BENCH_EVENTS=20 SYSLOG_BENCH_MESSAGE_BYTES=200 SYSLOG_BENCH_BATCH_SIZE=5 SYSLOG_BENCH_TIME=0.1 SYSLOG_BENCH_WARMUP=0 mix run --no-start test/profiling/syslog_send_bench.exs

##### With input 200 byte message #####
Name                                                                          ips        average  deviation         median         99th %
handle_message routes, handle_batch formats+sends batch                    957.01        1.04 ms   +/-260.07%        0.74 ms       27.09 ms
no batcher: handle_message formats+sends each message                      466.68        2.14 ms    +/-64.21%        1.83 ms       10.36 ms
preformat in handle_message, handle_batch sends preformatted frames        275.50        3.63 ms    +/-98.59%        2.17 ms       15.66 ms

Comparison:
handle_message routes, handle_batch formats+sends batch                    957.01
no batcher: handle_message formats+sends each message                      466.68 - 2.05x slower +1.10 ms
preformat in handle_message, handle_batch sends preformatted frames        275.50 - 3.47x slower +2.58 ms

Broadway end-to-end:

Command:
SYSLOG_BENCH_EVENTS=20 SYSLOG_BENCH_MESSAGE_BYTES=200 SYSLOG_BENCH_BATCH_SIZE=5 SYSLOG_BENCH_CONCURRENCY=1 SYSLOG_BENCH_PUSH_CHUNK=10 SYSLOG_BENCH_TIME=0.1 SYSLOG_BENCH_WARMUP=0 mix run --no-start test/profiling/syslog_broadway_bench.exs

##### With input 200 byte message #####
Name                                                                                  ips        average  deviation         median         99th %
broadway: no batcher, handle_message formats+sends each message                     80.99       12.35 ms     +/-3.29%       12.17 ms       12.96 ms
broadway: handle_message routes, handle_batch formats+sends batch                   74.04       13.51 ms     +/-6.17%       13.38 ms       15.01 ms
broadway: handle_message preformats, handle_batch sends preformatted frames          6.86      145.79 ms     +/-0.00%      145.79 ms      145.79 ms

Comparison:
broadway: no batcher, handle_message formats+sends each message                     80.99
broadway: handle_message routes, handle_batch formats+sends batch                   74.04 - 1.09x slower +1.16 ms
broadway: handle_message preformats, handle_batch sends preformatted frames          6.86 - 11.81x slower +133.44 ms

Interpretation:

The direct send benchmark is mostly isolating formatter cost plus the number of TCP sends. With this tiny input, batching wins clearly because it does fewer :gen_tcp.send/2 calls and the local sink waits until all expected octet-counted frames are parsed.

The Broadway smoke run includes pipeline startup, BufferProducer polling, processor/batcher handoff, NimblePool checkout, TCP send, and receiver parsing. At this very small event count, fixed Broadway overhead dominates, so the no-batcher path can look slightly faster than the batched path. The preformat variant is much slower here because it pays formatting cost in handle_message/3 and still goes through batcher coordination before sending.

For useful throughput comparison, run the same scripts with larger inputs, for example:

SYSLOG_BENCH_MESSAGE_BYTES=200,2000,50000 \
SYSLOG_BENCH_CONCURRENCY=1,5 \
SYSLOG_BENCH_BATCH_SIZE=50 \
SYSLOG_BENCH_EVENTS=5000 \
SYSLOG_BENCH_PUSH_CHUNK=1000 \
mix run --no-start test/profiling/syslog_broadway_bench.exs

@ruslandoga ruslandoga force-pushed the ruslandoga-conductor/syslog-adaptor-benchmarks branch from aae1cb5 to 7d9eb2d Compare May 13, 2026 17:59
@behaviour Broadway.Acknowledger

def start_link(opts) do
Broadway.start_link(__MODULE__,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of copying the pipeline, can we make the pipeline parameterized such that we can pass opts to it, see BigQuery.Pipeline for how the args are passed for benchmarking

@Ziinc Ziinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be condensed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants