Repository navigation
ggml-quantization: add llama.cpp GGUF quantization kernels for Metal - #1120
Conversation
37b7a60 to
78f269e
Compare
|
cc @sayakpaul |
78f269e to
4877056
Compare
| backends = ["metal"] | ||
|
|
||
| [general.hub] | ||
| repo-id = "kernels-community/ggml-quantization" |
There was a problem hiding this comment.
Okay for now but should we have this under the GGML org?
There was a problem hiding this comment.
hmmm for now, i think it is better to have it here as it will be easier to maintain no ?
There was a problem hiding this comment.
GGML is HF :) So, even kernels-community is easier to maintain, we should be cognizant about growing kernels out of that single HF org. So, I think we should consider it.
There was a problem hiding this comment.
I don't mind moving it to ggml but maybe let's see if this takes off or not before moving them to ggml org ? Both are fine to me cc @LysandreJik if you have a preference
|
/kernel-bot build-and-stage ggml-quantization |
|
Build request processed. Command: Dispatched (1):
|
e7c90a3 to
ddc9bf3
Compare
|
/kernel-bot build ggml-quantization |
|
Build request processed. Command: Dispatched (1):
|
|
Since the topk is a different package not related to ggml, i prefer to keep it seperate |
0ae07b6 to
ac88f29
Compare
|
/kernel-bot build-and-stage ggml-quantization |
|
Build request processed. Command: Dispatched (1):
Hub uploads: |
ac88f29 to
fb04d63
Compare
|
/kernel-bot merge-and-upload ggml-quantization |
|
Build request processed. Command: Merge result: Pull Request successfully merged Dispatched (1):
Hub uploads: |
What does this PR do ?
This PR adds kernels from llamacpp related to quantization for metal only.