●Change8

b10431

📦 llama-cppView on GitHub →
✨ 5 features🐛 1 fixes🔧 2 symbols

Summary

Introduced recurrent state rollback for nemotron on CPU and CUDA, and improved ssm_scan to support K > 1 for all backends. This release also includes runtime checks for operation support and fallback mechanisms.

Migration Steps

  1. Update to the latest version to incorporate the recurrent state rollback and K > 1 support in ssm_scan.

✨ New Features

  • Recurrent state rollback for nemotron for CPU and CUDA.
  • Support for K > 1 in ssm_scan for all backends.
  • Runtime API to check if op is supported, replacing static CUDA detection.
  • Fallback to CPU when SSM rollback is not supported.
  • Graph selection based on fused_ssm_op support and handling of memory->n_rs_seq > 1 case when op is not supported.

🐛 Bug Fixes

  • Fix CI Issues.

Affected Symbols

ggml_ssm_scanggml