Fix NotImplementedError

✅ Solution

The "NotImplementedError" in vllm usually arises when a requested feature, often a specific CUDA or optimized operation for a particular data type (like Float8), hasn't been coded or compiled for your hardware (especially ROCm or older GPUs) or a specific model architecture. To fix this, either use a supported data type (like float16 or bfloat16), ensure you're using a vllm version with ROCm support if needed, or if applicable, wait for a future update with the specific operation implemented for your hardware/model or contribute the missing implementation yourself according to the vllm documentation.