b8100
📦 llama-cppView on GitHub →
✨ 4 features🐛 1 fixes🔧 2 symbols
Summary
This release introduces full support for modern BERT models, including specific architectural adjustments like GELU in rank pooling and dense first layers. Several internal file updates and bug fixes related to mean pooling were also implemented.
✨ New Features
- Added full modern BERT support.
- Added GELU operation in rank pooling for modern BERT.
- Added mean calculation before the classifier head for modern BERT processing.
- Implemented the first layer as dense, aligning with modern BERT research papers.
🐛 Bug Fixes
- Fixed set input for mean pooling to correctly check if the pooling type is ranking, supporting both mean & rank for modern BERT.