Happy_Chicken9835

joined 10 months ago
sorted by: new top controversial old
Deploy LLaMA 2 on AWS: What's the best option? in c/localllama@poweruser.forum
[โ€“] Happy_Chicken9835@alien.top 1 points 9 months ago

I deployed Llama 2 (GGUF and using CPU) as an Amazon ECS fargate service

I just bundled my entire Docker build into ECR and fired up my container

ORCA 2 Released open source! in c/localllama@poweruser.forum
[โ€“] Happy_Chicken9835@alien.top 1 points 10 months ago

The bloke has a few quantized variants

A gguf 7B: https://huggingface.co/TheBloke/Orca-2-7B-GGUF