this post was submitted on 01 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Trying to get a better understanding of how prompts work in relation to fine-tunes, and trying to see if any of them are actually reliable enough to be used in a "production" type environment.
My end goals are basically
A lot of it comes down to just wanting to learn, but a big piece of it is that I have consistency, stability and privacy when running an LLM at home.
As for how I run it? Ho ho ho... a bit overkill, since as a developer I have a lot of hardware available to me.
I usually connect the mistral to Continue.Dev in Visual Studio code.
are you running exllama on phind for 4090? was there a reason you'd need to run it on m2 ultra when switching to 100k context?
also, I didn't know mistral could do coding tasks, how is it?