behohippy

joined 1 year ago
[–] behohippy@lemmy.world 1 points 1 year ago

The advancements in this space have moved so fast, it's hard to extract a predictive model on where we'll end up and how fast it'll get there.

Meta releasing LLaMA produced a ton of innovation from open source that showed you could run models that were nearly the same level as ChatGPT with less parameters, on smaller and smaller hardware. At the same time, almost every large company you can think of has prioritized integrating generative AI as a high strategic priority with blank cheque budgets. Whole industries (also deeply funded) are popping up around solving the context window memory deficiencies, prompt stuffing for better steerability, better summarization and embedding of your personal or corporate data.

We're going to see LLM tech everywhere in everything, even if it makes no sense and becomes annoying. After a few years, maybe it'll seem normal to have a conversation with your shoes?

 

I host a ton of services running behind my nginx reverse proxy (basic auth + lets encrypt). On the whole it works really well with nearly everything I throw at it. Lately, there's been a lot of gradio/websocket/python stuff coming from the AI community like the local llama and stable diffusion stuff. Not sure what's causing it but there's always weird issues when I try to reverse proxy them.

Does anyone have some magic settings that "just work" with these weirdo web apps?

 

Ryzen 5900X, 64 gig DDR4-3200, 2tb ssd,10tb hdd and an RTX2070. Hosting Stable Diffusion, various llama.cpp instances with python bindings, jellyfin, sonarr, multiple modded minecraft servers, and a network file share.