Add write buffering to reduce IOPS from small sequential writes

[Copilot]

Write Buffering Implementation Plan

• Add configuration parameter multiuser.writebuffersize to MultiuserFileSystem
• Add buffer-related member variables to MultiuserFile class
• Implement buffering logic in MultiuserFile::Write method
• Implement buffer flushing on out-of-sequence writes
• Implement buffer flushing in MultiuserFile::Close method
• Add mutex for thread-safe buffer access
• Fix partial write handling in FlushWriteBuffer
• Improve config input validation
• Fix checksum calculation for partial writes
• Add documentation for FlushWriteBuffer
• Optimize buffer allocation with reserve()
• Update README.md with write buffering documentation
• Update sample configuration file with examples
• Address PR feedback: conditional mutex locking
• Address PR feedback: fill buffer before flush
• Merge PR Handle non-sequential writes in MultiuserFile #61 logic: disable checksumming on non-sequential writes
• Code cleanup: use nullptr and consistent buffer sizing

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Buffer sequential writes</issue_title>
<issue_description>We've had a CephFS system indicate that they are seeing strongly negative effects of relatively small writes being emitted by XRootD. The thinking is they are seeing a large number of 16KB write syscalls and a large number of IOPS in Ceph. This is a bit surprising to me - my assumption is that buffered IO is going to be smoothed out at the OS level.

Regardless -- because we default to sequential writes, it should be trivial to do some application-side buffering as well:

1. Add a configuration knob that allows the user to specify write buffering size (defaults to 0).
2. When writes occur, if they are smaller than the buffer size (or the total buffered data is less than the buffer size), buffer. Otherwise, write to the next layer down. Use a mutex to ensure concurrent writes don't muck up the buffer.
3. If an out-of-sequential write (or close) is encountered, dump the remaining buffer and turn off buffering for the remainder of the open file.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Buffer sequential writes #64

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[bbockelm]

Good first try, small changes requested.

[bbockelm]

When buffered writes are not enabled at constructor time (i.e., disabled in the config file), never take this mutex.

Merge the logic in https://github.com/opensciencegrid/xrootd-multiuser/pull/61/changes with the changes in this function; this work will conflict with that and it's straightforward to do both.

[Copilot]

Changed to use std::unique_lock with std::defer_lock - the lock is only acquired when m_write_buffer_size > 0. Also merged the PR #61 logic to disable checksumming (instead of returning error) on non-sequential writes. Fixed in b830dd6.

[bbockelm]

There are edge cases here I'd like approached differently.

If the total is over the buffer size, then copy as much data as you can to fill the buffer and then write. This way, writes are always the buffer size (except maybe the last before close) at the cost of memcpy.

[Copilot]

Updated to fill the buffer completely before flushing when total exceeds buffer size. After flush, remaining data is either buffered (if it fits) or written directly (if too large). All writes are now buffer-sized except possibly the last. Fixed in b830dd6.

[bbockelm]

Similarly as above - if the buffering is disabled in the config file, then skip the lock.

[Copilot]

Changed to use std::unique_lock with std::defer_lock - the lock is only acquired when m_write_buffer_size > 0, skipping the lock entirely when buffering is disabled. Fixed in b830dd6.

[djw8605]

Should it also have some sort of timer, so if a write is stalled, or the close is stalled, it flushes the buffer to the next layer down after some time of idle?