b10699
📦 llama-cppView on GitHub →
✨ 2 features🔧 2 symbols
Summary
Introduced `ggml_backend_op_alloc_size_may_expand` to better manage memory allocation for specific operations across various backends. The RPC backend has been updated to utilize this new helper function.
✨ New Features
- Added `ggml_backend_op_alloc_size_may_expand` to backend utilities to handle ops requiring additional memory for fleeting data in backends like Metal, SYCL, and WebGPU.
- The RPC backend now uses the new `ggml_backend_op_alloc_size_may_expand` helper to determine whether to query the remote server for actual alloc size, replacing a hardcoded list.