Skip to content

ggml-cpu: extend RVV quantization vec dot to higher VLENs #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
taimur-10x wants to merge 12 commits into
base: 10x/riscv-quant-vec-dot-128b
Choose a base branch
from
Open

ggml-cpu: extend RVV quantization vec dot to higher VLENs #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
1440762
ggml-cpu: add rvv quantize_row_q8_K kernel
taimur-10x Dec 15, 2025
481cb2e
ggml-cpu: add rvv vec_dot for iq4_nl, mxfp4, iq2_xxs
taimur-10x Dec 15, 2025
8cc2254
ggml-cpu: add rvv vec_dot for iq4_xs, refactor
taimur-10x Dec 17, 2025
3d682db
ggml-cpu: remove ifunc for rvv vec dot
taimur-10x Dec 19, 2025
6769950
ggml-cpu: add vec_dot for iq2_xs, iq3_xxs
RehanQasim-dev Dec 19, 2025
08cfbe0
ggml-cpu: refactor quants.c
taimur-10x Mar 4, 2026
97d1218
ggml-cpu: add 128-bit impls for i-quants, ternary quants
taimur-10x Feb 13, 2026
2785c94
ggml-cpu: add 128-bit impls for iq2_xs, iq3_s, iq3_xxs, tq2_0
RehanQasim-dev Feb 24, 2026
adb8b4e
ggml-cpu: add rvv 512b,1024b impls for iq4_xs
taimur-10x Feb 13, 2026
7f2bc2c
ggml-cpu: refactor; add rvv 512b, 1024b impls for q6_K, i-quants
taimur-10x Feb 14, 2026
3326b3e
added 512 and 1024 implementations of tq3_s, iq3_xxs, iq2_s, iq2_xs, …
RehanQasim-dev Feb 24, 2026
bd69a20
ggml-cpu: improve iq2_xs impl for rvv 256b
RehanQasim-dev Feb 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions ggml/src/ggml-cpu/arch-fallback.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,13 +194,6 @@
#define ggml_gemm_q8_0_4x8_q8_0_generic ggml_gemm_q8_0_4x8_q8_0
#elif defined(__riscv)
// quants.c
#define quantize_row_q8_K_generic quantize_row_q8_K
#define ggml_vec_dot_iq2_xxs_q8_K_generic ggml_vec_dot_iq2_xxs_q8_K
#define ggml_vec_dot_iq2_xs_q8_K_generic ggml_vec_dot_iq2_xs_q8_K
#define ggml_vec_dot_iq3_xxs_q8_K_generic ggml_vec_dot_iq3_xxs_q8_K
#define ggml_vec_dot_iq4_nl_q8_0_generic ggml_vec_dot_iq4_nl_q8_0
#define ggml_vec_dot_iq4_xs_q8_K_generic ggml_vec_dot_iq4_xs_q8_K
#define ggml_vec_dot_mxfp4_q8_0_generic ggml_vec_dot_mxfp4_q8_0
// repack.cpp
#define ggml_quantize_mat_q8_0_4x4_generic ggml_quantize_mat_q8_0_4x4
#define ggml_quantize_mat_q8_0_4x8_generic ggml_quantize_mat_q8_0_4x8
Expand Down Expand Up @@ -244,7 +237,7 @@
#define ggml_vec_dot_q2_K_q8_K_generic ggml_vec_dot_q2_K_q8_K
#define ggml_vec_dot_iq2_xxs_q8_K_generic ggml_vec_dot_iq2_xxs_q8_K
#define ggml_vec_dot_iq2_xs_q8_K_generic ggml_vec_dot_iq2_xs_q8_K
#define ggml_vec_dot_iq2_s_q8_K_generic ggml_vec_dot_iq2_s_q8_K
// #define ggml_vec_dot_iq2_s_q8_K_generic ggml_vec_dot_iq2_s_q8_K
#define ggml_vec_dot_iq3_xxs_q8_K_generic ggml_vec_dot_iq3_xxs_q8_K
#define ggml_vec_dot_iq3_s_q8_K_generic ggml_vec_dot_iq3_s_q8_K
#define ggml_vec_dot_iq1_s_q8_K_generic ggml_vec_dot_iq1_s_q8_K
Expand Down Expand Up @@ -292,7 +285,7 @@
#define ggml_vec_dot_tq1_0_q8_K_generic ggml_vec_dot_tq1_0_q8_K
#define ggml_vec_dot_tq2_0_q8_K_generic ggml_vec_dot_tq2_0_q8_K
#define ggml_vec_dot_iq2_xxs_q8_K_generic ggml_vec_dot_iq2_xxs_q8_K
#define ggml_vec_dot_iq2_xs_q8_K_generic ggml_vec_dot_iq2_xs_q8_K
// #define ggml_vec_dot_iq2_xs_q8_K_generic ggml_vec_dot_iq2_xs_q8_K
#define ggml_vec_dot_iq2_s_q8_K_generic ggml_vec_dot_iq2_s_q8_K
#define ggml_vec_dot_iq3_xxs_q8_K_generic ggml_vec_dot_iq3_xxs_q8_K
#define ggml_vec_dot_iq3_s_q8_K_generic ggml_vec_dot_iq3_s_q8_K
Expand Down
Loading
Loading