SideShow_Bot

joined 11 months ago
sorted by: new top controversial old
Models Megathread #2 - What models are you currently using? in c/localllama@poweruser.forum
[–] SideShow_Bot@alien.top 1 points 11 months ago (1 children)

IIUC, for coding you suggest deepseek-coder-6.7b-instruct.Q4_K_M.gguf, right? Can I run it with 16 Gb? I'm on a i5 Windows machine, using LM Studio.

LLM Web-UI recommendations in c/localllama@poweruser.forum
[–] SideShow_Bot@alien.top 1 points 11 months ago

I'll have a look into it and compare it to LM Studio.

LLM Web-UI recommendations in c/localllama@poweruser.forum
[–] SideShow_Bot@alien.top 1 points 11 months ago (2 children)

So, in the end which one would you recommend for someone just beginning to run LLMs locally? Windows machine (thus Sanctum is out of the question for now). I'm interested in 3 use cases, so maybe there would be a different answer for each of them:

  1. Python coding questions
  2. Linux shell questions
  3. RAG: in particular, I would like to be able to ask questions and have the model retrieve an answer online, supported by one or more working hyperlinks