b9673
📦 llama-cppView on GitHub →
✨ 2 features🔧 1 symbols
Summary
This release introduces an optional SYCL feature for USM system memory allocations for large GPU buffers, potentially allowing VRAM overcommit. Various pre-built binaries for different platforms are provided.
✨ New Features
- Introduced an optional feature for SYCL to allocate large GPU buffers (>= 1GB) using USM system allocations if supported by the device, allowing system memory management for host/device migration.
- USM system allocations are disabled by default and require setting the GGML_SYCL_USM_SYSTEM environment variable to enable.