b8634
Breaking Changes📦 llama-cppView on GitHub →
⚠ 1 breaking✨ 2 features🐛 1 fixes🔧 3 symbols
Summary
This release introduces explicit support for the Granite 4.0 chat template, resolving tool calling issues for this version by updating role mapping. The existing Granite 3.x template constant was renamed for clarity.
⚠️ Breaking Changes
- The constant `LLM_CHAT_TEMPLATE_GRANITE` has been renamed to `LLM_CHAT_TEMPLATE_GRANITE_3_X`. Code relying on the old name for Granite 3.x templates will break and must be updated to use the new constant name.
Migration Steps
- Rename usages of `LLM_CHAT_TEMPLATE_GRANITE` to `LLM_CHAT_TEMPLATE_GRANITE_3_X` if targeting Granite 3.x behavior.
- Use the new constant `LLM_CHAT_TEMPLATE_GRANITE_4_0` for Granite 4.0 models.
✨ New Features
- Added support for the Granite 4.0 chat template, including correct tool_call role mapping using `<tool_call>` XML tags.
- Introduced detection logic to differentiate between Granite 3.x and 4.0 templates based on the presence of `<|start_of_role|>` combined with `<tool_call>` or `<tools>`.
🐛 Bug Fixes
- Fixed tool calling for Granite 4.0 when not using the Jinja flag, by correctly mapping the `assistant_tool_call` role.