How to work with sglang on multi nodes?
#1
by
lalaland9
- opened
I have two nodes, each with two A6000 GPUs, and obviously TP ≤ 2 won’t work.
lalaland9
changed discussion title from
work in sglang?
to How to work with sglang on multi nodes?
Hi sorry for the late replies. I did try to test on SGLang and it also did not work in my case. I tried to modify SGLang source code, but it seems that more changes needed to be done for it to work.