Skip to content

Deduplicate GPU and CPU seed outputs#15

Open
mrzetti wants to merge 1 commit into
MinecraftAtHome:mainfrom
mrzetti:dupe-fix
Open

Deduplicate GPU and CPU seed outputs#15
mrzetti wants to merge 1 commit into
MinecraftAtHome:mainfrom
mrzetti:dupe-fix

Conversation

@mrzetti

@mrzetti mrzetti commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

fixes #10

  • Added GPU-side-output dedupe runs on the host after final GPU candidates are copied back. It sorts only the final candidate list for that batch, which should be very small compared to the billions of seeds/positions scanned.
  • Added CPU final-output dedupe as an unordered_set for results that already passed CPU filters, so it only runs on final hits. (Memory grows by one hash entry per unique printed result)

Verification

  • make NO_GPU=1 passed.
  • make gpu.o passed.
  • make passed.

Performance test needed to be safe!

@BoySanic

Copy link
Copy Markdown
Contributor

I'm not really looking for a dedupe. I want to know why we have a duplicated result to begin with as it could be causing us to do duplicate work.

@BoySanic BoySanic closed this Jun 22, 2026
@BoySanic BoySanic reopened this Jun 22, 2026
Comment thread src/common.h
h ^= std::hash<int32_t>{}(output.score) + 0x9e3779b97f4a7c15ULL + (h << 6) + (h >> 2);
return h;
}
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably specialize std::hash<CpuOutput> instead of writing a custom hasher struct.

Comment thread src/gpu.cu
});
h_buffer.erase(std::unique(h_buffer.begin(), h_buffer.end(), [](const SeedPos &a, const SeedPos &b) {
return a.seed_index == b.seed_index && a.x == b.x && a.z == b.z;
}), h_buffer.end());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably overload the comparison operators instead of using a lambda. There is a nice way to implement these by wrapping the members in a temporary tuple.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Duplicate results

3 participants