Jelegend

joined 11 months ago
sorted by: new top controversial old
M1/M2/M3: increase VRAM allocation with `sudo sysctl iogpu.wired_limit_mb=12345` (i.e. amount in mb to allocate) in c/localllama@poweruser.forum
[โ€“] Jelegend@alien.top 1 points 11 months ago (1 children)

I am getting the following error on running this command on Mac Studio M2 Max 64GB RAM

sysctl: unknown oid 'iogpu.wired_limit_mb'

Can soeome help me out here on what to do here?

llama.cpp server rocks now! ๐Ÿค˜ in c/localllama@poweruser.forum
[โ€“] Jelegend@alien.top 1 points 11 months ago (2 children)

I am using it as http://localhost:8000/v1/completions and it is working perfectly

Yi-23B-Llama: Distil version of Yi-34B-Llama in c/localllama@poweruser.forum
[โ€“] Jelegend@alien.top 1 points 11 months ago

Yeah, exactly the same thing. Produced absolutely rubbish whatever i tried. I tried 8B 15B and 23B