LocalLLaMA

11 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

Alternatives to LLamaSharp? (alien.top)

submitted 2 years ago by kimberly1818@alien.top to c/localllama@poweruser.forum

7 comments fedilink hide all child comments

Hi I am a newbie c# dev, I am trying to create a home project and until recently I was using Llamasharp. There is little support for it and since the last updates I've been unable to get it to work at all with the recent updates.

i'm trying to build a little chat wpf application which can either load AWQ or GGUF LLM files. Are there any simple and easy to use libraries out there which I can facilitate in c#?

I have a GTX 3060 and I'd preferably like to use my GPU RAM if it's faster than using DDR4 RAM. I admit I am under a few misconceptions. Ideally I'd like to be able to load the Mistral models in c#.

https://preview.redd.it/6tx5ij2imm2c1.jpg?width=877&format=pjpg&auto=webp&s=53e2a07f53e5d7e15ebbe727d6930bfd3bbea25b

you are viewing a single comment's thread
view the rest of the comments

[–] mrjackspade@alien.top 1 points 2 years ago

Gonna be honest, you can totally just skip LlamaSharp and call the Llama.dll methods using interop in C#

Its really not difficult to do and it cuts an entire layer of dependency out of your project.