●Change8

b9110

📦 llama-cppView on GitHub →
🐛 2 fixes🔧 6 symbols

Summary

This release updates documentation regarding server metrics and fixes a type issue for a newly added metric, while providing numerous pre-compiled binaries across different operating systems and hardware configurations.

🐛 Bug Fixes

  • Fixed the type for the n_busy_slots_per_decode metric in the server.
  • Updated server README to correctly describe the required model query parameter for router mode.

Affected Symbols

llamacpp:kv_cache_usage_ratio (removed metric)llamacpp:kv_cache_tokens (removed metric)llamacpp:prompt_seconds_total (added metric)llamacpp:tokens_predicted_seconds_total (added metric)llamacpp:n_decode_total (added metric)llamacpp:n_busy_slots_per_decode (added metric)