quant

This release bumps the llama.cpp version to 0.2.0 and ggml to 0.21.0, incorporating numerous backend improvements, bug fixes, and new model support across various hardware accelerators like SYCL, OpenCL, Metal, and Vulkan.

This release introduces enhanced quantization capabilities, allowing manual tensor types with the --pure flag. It also provides pre-compiled binaries for various platforms and hardware configurations.

This release introduces a fix where manual tensor type overrides now correctly take precedence within the quant module and provides extensive pre-compiled binaries for macOS, Linux, Windows, and openEuler across various hardware and accelerator configurations.