A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Use Cases
We ingest EVE JSON over a unix datagram socket at high message rates
(hundreds of thousands of small messages per second).
The source cannot keep up. Profiling points at per-message overhead rather than
decoding or downstream work: the current listen() loop does one recv_from
syscall per datagram, and calls out.send_batch() once per datagram as well. At
this message size, syscall and channel-send overhead dominate the actual work and we cannot drain our load.
Proposal
Read many datagrams per wakeup with recvmmsg(2), and push the decoded events
downstream as a single batch:
- Preallocate N buffers of
max_length and call recvmmsg.
- Decode each returned message, collect the events into one
Vec<Event>, and
issue a single send_batch per recvmmsg call.
New option on the unix datagram socket source, defaulting to 1 so existing
behaviour is unchanged:
sources:
suricata:
type: socket
mode: unix_datagram
path: /path/to_/socket
recvmmsg_buffers: 64 # messages per recvmmsg call, default 1
Memory held per socket is recvmmsg_buffers * max_length, so the option is a
direct throughput/memory trade-off the operator controls.
Happy to open the PR — we have a working implementation.
A note for the community
Use Cases
We ingest EVE JSON over a unix datagram socket at high message rates
(hundreds of thousands of small messages per second).
The source cannot keep up. Profiling points at per-message overhead rather than
decoding or downstream work: the current
listen()loop does onerecv_fromsyscall per datagram, and calls
out.send_batch()once per datagram as well. Atthis message size, syscall and channel-send overhead dominate the actual work and we cannot drain our load.
Proposal
Read many datagrams per wakeup with
recvmmsg(2), and push the decoded eventsdownstream as a single batch:
max_lengthand callrecvmmsg.Vec<Event>, andissue a single
send_batchperrecvmmsgcall.New option on the unix datagram socket source, defaulting to 1 so existing
behaviour is unchanged:
Memory held per socket is recvmmsg_buffers * max_length, so the option is a
direct throughput/memory trade-off the operator controls.
Happy to open the PR — we have a working implementation.