this post was submitted on 22 Sep 2023
82 points (91.8% liked)

Linux

48031 readers
1280 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
nooter692@lemmy.ml
AgreeableLandscape@lemmy.ml
MarcellusDrum@lemmy.ml
cypherpunks@lemmy.ml
cyclohexane@lemmy.ml
d3Xt3r@lemmy.nz
82
What is FOSS answer to BingGPT & Google Bard? (lemmy.ml)
submitted 1 year ago* (last edited 1 year ago) by Subject6051@lemmy.ml to c/linux@lemmy.ml
22 comments fedilink hide all child comments
 

There was a post asking people their opinions about Edge and many people seemed to liked the idea of Edge and seemed to be ok having it on Linux (Blasphemy)

Also, can we all agree how fast Edge went from joke to a threat? I mean, it's good now alright! It was good back then, but it's better now. Money man!!! Money! Personally I hate MS, but I can't help but see the fact that there is no alternative to Bing GPT and many features Bing offers on Linux.

If there is an open source ChatGPT how would it look? Who would bear the costs? How would we solve the server problem? i.e., it would take a ton of server space and bandwidth. Just wondering.

I am pretty sure MS products will improve greatly due to their integration with GPT what do us poor folks on Linux do?

Just want to know the answers, I don't want to discuss (aka can't comment, I need to study), but just curious!

you are viewing a single comment's thread
view the rest of the comments
[–] lloram239@feddit.de 27 points 1 year ago* (last edited 1 year ago) (3 children)

what do us poor folks on Linux do?

Run llama.cpp and any of the models listed here, that stuff has been around for months.

TheBloke has a lot of models converted to GGUF format which you need for llama.cpp.

Quick Start Guide (requires Nix, otherwise compile llama.cpp manually):

$ GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/TheBloke/guanaco-7B-GGUF
$ cd guanaco-7B-GGUF
$ git lfs pull --include=Guanaco-7B.Q4_0.gguf
$ nix run github:ggerganov/llama.cpp -- -m Guanaco-7B.Q4_0.gguf --instruct
> Write haiku about a penguin
 A penguin walks on ice,
 Takes a plunge in the sea,
 Hides his feet from me!
[–] 257m@lemmy.ml 1 points 1 year ago (1 children)

I ran it on my pc with a gtx 1070 with cuda enabled and compiled with the cuda compile hint but it ran really slowly how do you get it to run fast?

[–] lloram239@feddit.de 1 points 1 year ago (1 children)

To make use of GPU acceleration you have to compile it with the proper support (CUDA, OpenCL, ROCM) and add --gpu-layers 16 (or a larger number, however much your VRAM can handle). If that's not enough, than the GPU/CPU is probably to slow.

You can try a smaller model, those run faster, but give worse results.

[–] 257m@lemmy.ml 1 points 1 year ago

Thanks I might try that out later.

load more comments (1 replies)