Skip to content

[SYCL] enhance the api to support peer-to-peer copy - #27550

Draft
arthw wants to merge 1 commit into
ggml-org:masterfrom
arthw:en_dev2dev
Draft

[SYCL] enhance the api to support peer-to-peer copy#27550
arthw wants to merge 1 commit into
ggml-org:masterfrom
arthw:en_dev2dev

Conversation

@arthw

@arthw arthw commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Overview

It maybe fix the issue: #27198.

Additional information

There is missed to call the API to enable peer-to-peer memory copy.
This PR adds it to make the code following the guide and stable.

Requirements

@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Aug 22, 2026
@stolk

stolk commented Aug 22, 2026

Copy link
Copy Markdown

Applied your patch:

commit 7e4ccad92c66c272c6b0ed8f2ebc5e80109323c0 (HEAD -> master)
Author: zhang.jianyu <zhang.jianyu@outlook.com>
Date:   Sat Aug 22 23:57:57 2026 +0800

    enhance the api to support peer-to-peer copy

commit b21e4de74567f5eef213765c9476a843c2e43f0d (tag: b10586, origin/master, origin/HEAD)
Author: Xuan-Son Nguyen <son@huggingface.co>
Date:   Sat Aug 22 16:33:47 2026 +0200

    mtmd: use ggml_rope_set_offset (#27521)
    
    * mtmd: use ggml_rope_set_offset
    
    * add comment

Rebuild my tree with SYCL back end enabled, took out the export GGML_SYCL_DEV2DEV_MEMCPY=2 and launced llama.cpp

No improvement: still crashes on first prompt input, in:

__syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56
⚠️ warning: 56	../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S: No such file or directory
#0  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56
56	in ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S
#1  0x000079d228ea061c in __internal_syscall_cancel (a1=49577, a2=0, a3=0, a4=0, a5=0, a6=0, nr=61) at ./nptl/cancellation.c:49
⚠️ warning: 49	./nptl/cancellation.c: No such file or directory
#2  __syscall_cancel (a1=a1@entry=49577, a2=a2@entry=0, a3=a3@entry=0, a4=a4@entry=0, a5=a5@entry=0, a6=a6@entry=0, nr=61) at ./nptl/cancellation.c:75
75	in ./nptl/cancellation.c
#3  0x000079d228f1cd4f in __GI___wait4 (pid=pid@entry=49577, stat_loc=stat_loc@entry=0x0, options=options@entry=0, usage=usage@entry=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
⚠️ warning: 30	../sysdeps/unix/sysv/linux/wait4.c: No such file or directory
#4  0x000079d228f1cd9b in __GI___waitpid (pid=pid@entry=49577, stat_loc=stat_loc@entry=0x0, options=options@entry=0) at ./posix/waitpid.c:38
⚠️ warning: 38	./posix/waitpid.c: No such file or directory
#5  0x000079d23f67abb6 in ggml_print_backtrace () at /usr/src/llama.cpp/ggml/src/ggml.c:234
234	        waitpid(child_pid, NULL, 0);
#6  0x000079d23f679d89 in ggml_abort (file=0x79d22aec1f9b "/usr/src/llama.cpp/ggml/src/ggml-sycl/../ggml-sycl/common.hpp", line=155, fmt=0x79d22aec1fd9 "SYCL error") at /usr/src/llama.cpp/ggml/src/ggml.c:268
268	        ggml_print_backtrace();
#7  0x000079d22a0256a8 in ggml_sycl_error (stmt=<optimized out>, func=0x79d22aec213e "dev2dev_memcpy", file=0x79d22aebfc82 "/usr/src/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp", line=line@entry=714, msg=<optimized out>) at /usr/src/llama.cpp/ggml/src/ggml-sycl/../ggml-sycl/common.hpp:155
155	  GGML_ABORT("SYCL error");
#8  0x000079d22a02d55b in dev2dev_memcpy (device_dst=<optimized out>, q_dst=..., device_src=<optimized out>, q_src=..., ptr_dst=ptr_dst@entry=0x79bb78166000, ptr_src=ptr_src@entry=0x79b377c00000, size=3563520) at /usr/src/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:714
714	            SYCL_CHECK(CHECK_TRY_ERROR(q_dst.memcpy(ptr_dst, ptr_src, size).wait()));
#9  0x000079d22a02cc7c in ggml_backend_sycl_comm_allreduce_tensor (comm_ctx_v=<optimized out>, tensors=<optimized out>) at /usr/src/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:6686
6686	    dev2dev_memcpy(ctx0->device, *q0, ctx1->device, *q1, inbox0, outbox1, bf16_bytes);
#10 0x000079d23f6a2fdc in ggml_backend_meta_graph_compute (backend=<optimized out>, cgraph=<optimized out>) at /usr/src/llama.cpp/ggml/src/ggml-backend-meta.cpp:2215
2215	                backend_allreduce_success = backend_ctx->comm_allreduce(backend_ctx->comm_ctx, nodes.data());
#11 0x000079d23f69d2c6 in ggml_backend_graph_compute_async (backend=0x360eba20, cgraph=0x0) at /usr/src/llama.cpp/ggml/src/ggml-backend.cpp:452
452	    return backend->iface.graph_compute(backend, cgraph);
#12 ggml_backend_sched_compute_splits (sched=0x350b5c70) at /usr/src/llama.cpp/ggml/src/ggml-backend.cpp:1743
1743	            enum ggml_status ec = ggml_backend_graph_compute_async(split_backend, &split->graph);
#13 ggml_backend_sched_graph_compute_async (sched=0x350b5c70, graph=<optimized out>) at /usr/src/llama.cpp/ggml/src/ggml-backend.cpp:1973
1973	    return ggml_backend_sched_compute_splits(sched);
...
#30 0x000079d23f08c07f in llama_server (argc=39, argv=0x7ffdb8a2d5f8) at /usr/src/llama.cpp/tools/server/server.cpp:112
112	    return llama_server(params, argc, argv);
#31 0x000079d228e2a601 in __libc_start_call_main (main=main@entry=0x4064e0 <main(int, char**)>, argc=argc@entry=39, argv=argv@entry=0x7ffdb8a2d5f8) at ../sysdeps/nptl/libc_start_call_main.h:59
⚠️ warning: 59	../sysdeps/nptl/libc_start_call_main.h: No such file or directory
#32 0x000079d228e2a718 in __libc_start_main_impl (main=0x4064e0 <main(int, char**)>, argc=39, argv=0x7ffdb8a2d5f8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffdb8a2d5e8) at ../csu/libc-start.c:360
⚠️ warning: 360	../csu/libc-start.c: No such file or directory
#33 0x0000000000406415 in _start ()

I double checked: your commit is properly applied, as I see my line nr shift by one and matches your change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants