Cazzah

joined 1 year ago
[–] Cazzah@alien.top 1 points 1 year ago

There are already companies out there who are generating what they term small language models - basically hybrid models of say gpt 3.5 plus a large volume of corporate data - but they are all cloud based

I think you will find most of these are not small language models, but are instead the thing I said above - a llm like gpt + a search engine. Even small language models require millions of texts and only perform very specialised tasks.

[–] Cazzah@alien.top 1 points 1 year ago

As a general rule, self hosting is not cheaper, in the same way that it's not cheaper to sew your own clothes, make your own furniture, raise your own chickens, etc, because by the time you invest all the time and effort into tools and labour, you're well over the cost of the actual thing.

What it is a satisfying hobby and in the longer longer term might save you a little bit.

[–] Cazzah@alien.top 1 points 1 year ago (2 children)

That's because LLMs don't do that.

The companies that offer those services basically do some tricks behind the curtain.

Like let's say you want an LLM to learn your corporate docs. LLMs can't do that because they need millions of text from across the internet just to learn to speak English.. You can't feed your 1000 docs and 10,000 emails in and point to it and say "Forget the billion documents you injested and pay attention to this.... but also retain the ability to speak English"

What they actually implement is a standard text search engine, that returns matching paragraphs from the relevant documents, prompts to LLM with something like "This paragraph may contain an answer to user question X. If it does, please paraphrase it.

[–] Cazzah@alien.top 1 points 1 year ago

Not as much as you'd think.

Not having to render the graphics at all really helps reduce the performance impact of games a lot, because it basically brings it down to gameplay code only, which is usually not too complex. Any game that has server simulated physics might still get spicy though.