Vendor LZ4 as a git submodule and link statically

[carlhoerberg]

Summary

Vendor the LZ4 C library as a git submodule and link against a statically compiled liblz4.a instead of a system-installed shared library. This removes the runtime dependency on liblz4 and makes builds reproducible across machines and CI.

Changes

• vendor/lz4 — added as a git submodule, pinned to the v1.10.0 release tag.
• src/lz4/lib.cr — replaced @[Link("lz4")] (dynamic system link) with @[Link(ldflags: "#{__DIR__}/../../vendor/lz4/lib/liblz4.a")] for static linking; bumped VERSION_* constants to 1.10.0 to match the vendored source.
• shard.yml — added a postinstall script that checks out the submodule and builds the static archive (git submodule update --init --recursive && make -C vendor/lz4/lib liblz4.a) when installed as a dependency.
• Makefile — root-level targets for local builds (make, make spec, make clean).
• README.md — documents the vendored/static approach and the build step.

Verification

• crystal spec → 11 examples, 0 failures.
• A built binary reports LZ4_versionString = 1.10.0 (the vendored copy) and ldd shows no liblz4.so dependency — confirming static linkage with no runtime shared-library requirement.

🤖 Generated with Claude Code

[carlhoerberg]

Tradeoff note: downsides of vendoring + static linking vs. the system shared library

For the record, the downsides we're accepting with this change (roughly in order of how much they matter for this shard):

1. We own security updates now. With the system .so, the distro pushes CVE fixes (LZ4 has had real ones, e.g. CVE-2021-3520, a heap overflow in the decompressor) and every app picks them up on the next run — no rebuild. Vendored at v1.10.0, we're frozen until someone bumps the submodule, recompiles, and redistributes. We've taken on the job of watching upstream advisories. Mitigation: pin to release tags (done) and treat an LZ4 CVE as a prompt to bump the submodule promptly.

2. Build now needs a C toolchain + the submodule present. Previously a consumer just needed liblz4 installed; now shards install must have the submodule checked out and compile C via make/cc. More that can fail, and fragile in a few real cases:

• GitHub "Download ZIP" / source tarballs don't include submodules — those builds break.
• Whether shards reliably runs postinstall, and whether the installed copy retains a .git for git submodule update, isn't guaranteed across all install paths.
• Cross-compilation gets harder — we compile LZ4 ourselves for every target arch instead of using a prebuilt, distro-tested .so.

3. No shared-library deduplication. Each binary embeds its own copy of LZ4, and multiple processes can't share one mapped .so in memory. Negligible here — LZ4 is tiny (~280 KB archive, less after dead-code elimination) — but worth naming.

4. Risk of a double copy of LZ4. If the consuming application links LZ4 through some other path too, the final binary can end up with two copies / duplicate symbols. Unlikely, but vendoring makes it possible.

5. We forgo distro hardening. Packaged .sos are often built with the distro's hardening/optimization flags and tested on that platform; our local build won't match unless we replicate them.

Overall: the calculus is mostly favorable for this shard — LZ4 is small and stable, and static linking buys reproducible builds and zero runtime dependency. The one to keep an eye on is #1. If we'd rather not own that, supporting both (system link by default, vendored static as an opt-in build flag) is a middle ground, but it adds complexity that probably isn't worth it unless someone needs it.