●Change8

b9876

📦 llama-cppView on GitHub →
🐛 1 fixes🔧 3 symbols

Summary

This release primarily addresses a critical crash occurring in MoE models when tensor parallelism and CPU offloading are used together by correcting tensor contiguity checks within the GGML backend.

Migration Steps

  1. The fix involved moving the split-state lookup above the contiguity assertion in ggml_backend_meta_buffer_{get,set}_tensor and allowing the mirrored case, which should resolve the crash without requiring user code changes unless custom backend logic was involved.

🐛 Bug Fixes

  • Fixed a crash when using tensor parallelism (-sm tensor) combined with CPU-offloaded MoE experts (-ncmoe) on MoE models, caused by an assertion failure on a non-contiguous MoE router output tensor during warm-up.

Affected Symbols

ggml_backend_meta_buffer_get_tensorggml_backend_meta_buffer_set_tensorggml-backend-meta.cpp