Unexpected Uploads Causing Long Build Times

[JojOatXGME]

I just tried out DeterminateSystems/magic-nix-cache-action@v13, but the action uploads a lot of data which it isn't supposed to upload. Specifically, it uploads the results of various deviations which were already downloaded from a cache.

Besides wasting storage capacity, this takes a lot of time. While the builds inside my workflow took between 3 and 8 minutes, awaiting the unnecessary uploads further delayed the workflow by up to 20 minutes.

Workflow

Here is a full copy of my current workflow. I made a few changes along the way, which means some of my builds have used a different configuration. However, none of my changes seem to have changed the behavior in a noticeable and relevant way.

name: Check

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  check:

    name: Check
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Install Nix
        uses: DeterminateSystems/determinate-nix-action@v3
      - name: Setup Cache
        uses: DeterminateSystems/magic-nix-cache-action@v13
        with:
          use-flakehub: false
      - name: Run `nix flake check`
        run: nix -L flake check

Example

To show some example from the logs, the following line gets logged by “Run `nix flake check`”:

copying path '/nix/store/xjdfkfaa925fj504f53b2ilggaddr51x-linux-headers-6.12.7' from 'http://127.0.0.1:37515'...

Later, as part of “Post Setup Cache”, I get:

  2025-07-24T22:33:15.436098Z  INFO magic_nix_cache::gha: Uploaded '/nix/store/xjdfkfaa925fj504f53b2ilggaddr51x-linux-headers-6.12.7' to the GitHub Action Cache
    at magic-nix-cache/src/gha.rs:216

There is no point in uploading /nix/store/xjdfkfaa925fj504f53b2ilggaddr51x-linux-headers-6.12.7, which has just been downloaded from the cache.

Also note that /nix/store/xjdfkfaa925fj504f53b2ilggaddr51x-linux-headers-6.12.7 is originally coming from upstream (https://cache.nixos.org/) and was never build by me. This is another reason why it should have never been uploaded according to upstream-cache.

Store paths in this store aren't cached in GitHub Actions' cache.

The same happens to hundreds of packages during each build. The exact amount seems to be a bit random. The last build did upload 698 store paths to the cache according to the log of “Post Setup Cache”. The build before that did upload exactly 800 store paths. Two builds earlier, the action was uploading 544 store paths.

Potentially related:

• Don't cache fresh copies of previously-cached content? #75
• Caching already cached artifacts #147

[JojOatXGME]

Maybe this code comment is related?

magic-nix-cache/magic-nix-cache/src/gha.rs

Lines 96 to 99 in e994df0

	// FIXME: make sending the closure optional. We might want to
	// only send the paths that have been built by the user, under
	// the assumption that everything else is already in a binary
	// cache.

Do I understand this correctly, that the action is always re-uploading the entire closure? Doesn't that mean that you are always re-uploading everything, even if only one derivation has changed?

• If you changed any dependency of the main derivation, the main derivation also changes
• If the main derivation changes, the action uploads the entire closure with all dependencies, including dependencies provided by upstream

Is that correct?

Is there a good reason to upload derivations (i.e. dependencies) which have not been built by the user/CI?

[geekodour]

@grahamc should we pick this up? another work around would be to parallelize the uploads, think they're happening sequentially now? I have never written rust code but if this is something we'd like to do can pick this up and try making the uploads concurrent so that even if we're uploading the entire closure, atleast we'll do it faster. (?)

[JojOatXGME]

Btw, what is actually the reason to upload the entire closure? Are Nix caches supposed to guarantee that, if they provide a derivation, that they also contain its closure? If that is the case and not complying would actually cause Nix to fail instead of rebuilding the root of the closure, then only uploading the individual derivation is obviously quite risky.

Anyway, even in this case, you could just replace the overwrite-logic and just skip the upload if the cache entry already exists. I have seen you add a random suffix to all uploads to allow for easy, effective overwrites. What happens if you just remove the random suffix and silence upload-failures due to existing cache entries?

In my case, I am building a system configuration (which has overwrites for maybe 4 packages). The current implementation now uploads all the base packages of NixOS. What makes it worse, is that due to this overwrite-logic, I actually get a new separate but identical copy of all the base packages in my cache after every build.

It would be very nice if I could skip uploading all these NixOS packages. Anyway, if this is not reasonable due to the risk mentioned above, just keeping one copy of these packages would already be a huge improvement.

[Mic92]

Is the magic nix cache also uploading derivation files or this just logging?

https://github.com/Mic92/nix-update/actions/runs/17927188199/job/50976108263?pr=463

  2025-09-22T20:14:42.322242Z  INFO magic_nix_cache::gha: Uploaded '/nix/store/7byvrfyi1f00i8jxsc1q8z997j9r10y5-attr-2.5.2.tar.gz.drv' to the GitHub Action Cache
    at magic-nix-cache/src/gha.rs:214

  2025-09-22T20:14:42.982998Z  INFO magic_nix_cache::gha: Uploaded '/nix/store/7cv6bgl5s5pv7h9m32karq8vgndwak1s-perl5.40.0-Module-Build-Tiny-0.047.drv' to the GitHub Action Cache
    at magic-nix-cache/src/gha.rs:214

  2025-09-22T20:14:43.425411Z  INFO magic_nix_cache::gha: Uploaded '/nix/store/7d5p0ip9nd3aj3r1a2pmhsxxizqqnis8-openssl-3.5.1.drv' to the GitHub Action Cache
    at magic-nix-cache/src/gha.rs:214

[b521f771d8991e6f1d8e65ae05a8d783]

same here:

https://github.com/b521f771d8991e6f1d8e65ae05a8d783/from-scratch/actions/runs/19922186594/job/57115836245
https://github.com/b521f771d8991e6f1d8e65ae05a8d783/from-scratch/actions/runs/19920771442/job/57108859283

Using the action does not reduce the build time as claimed, but bumps it up by a factor of three.

[jozefRudy]

so i think

- name: Magic Nix Cache
  uses: DeterminateSystems/magic-nix-cache-action@v8
  with:
    use-gha-cache: ${{ github.ref == 'refs/heads/main' }}

this re-uploads cache even on branch. so use-gha-cache: false is not respected and uploads cache