b9714
📦 llama-cppView on GitHub →
✨ 1 features
Summary
This release enhances streaming reliability by adding a crucial Nginx header to streaming endpoints and provides extensive pre-compiled binaries across multiple operating systems and hardware accelerators.
✨ New Features
- Added "X-Accel-Buffering": "no" header to streaming endpoints to prevent Nginx from buffering responses, which resolves streaming issues with certain applications like the Pi coding harness.