Loved the responses from OpenHermes 2.5, however found the inference on the slower side especially when comparing it to other 7B models like Zephyr 7B or Vicuna 1.5 7B
shivam2979
joined 10 months ago
Loved the responses from OpenHermes 2.5, however found the inference on the slower side especially when comparing it to other 7B models like Zephyr 7B or Vicuna 1.5 7B
Haven’t you noticed slower inference from OpenHermes 2.5 compared to other 7B models?