I got Qwen running on the XDNA1 NPU in my Ryzen 7 250 on Linux

I've been messing around with the XDNA1 NPU in my Ryzen 7 250 because having a 16 TOPS NPU sitting mostly unused under Linux was bothering me. So I ended up building a small LLM inference runtime around it. Right now I have Qwen2.5 0.5B running with AIE2 kernels on the NPU. The runtime currently has NPU paths for Q/K/V and output projections,... Weiterlesen