this post was submitted on 11 Oct 2023
459 points (92.4% liked)
Technology
59219 readers
3980 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Oh surprise surprise, looks like generative AI isn't going to fulfill Silicon Valley and Hollywood studios' dream of replacing artist, writers, and programmers with computer to maximize value for the poor, poor shareholders. Oh no!
As I said here before, generative AIs are not universal solution to everything that has ever existed like they are hyped up to be, but neither are they useless. At the end of the day, they are ultimately tools. Complex, powerful, useful tools, but tools nonetheless. A good artist can create better work faster with the help of a diffusion model, the same way LLM code generation can help a good programmer finish their project faster and better. (I think). All of these AI models are trained on data from data from everyone on Internet, which is why I think its reasonable that everyone should have access to these generative AI models for the benefit of humanity and not profit, and not just those who took other people's work for free to trained the models. In other words, these generative AI models should belong to everyone.
And here lies my distaste for Sam Altman: OpenAI was founded as a nonprofit for the benefit of humanity, but at the first chance of money he immediately started venture capitalisting and put anything from GPT-2 onwards under locks and keys for money, and now it looks like that they are being crushed under the weight of their own operating costs while groups like Facebook and Stability catches up with actual open models, I will not be sad if "Open"AI fails.
(For as much crap as I give Zuck for the other awful things they do, I do admire their commitment to open source.)
I have to admit, playing with these generative models is pretty fun.
Hm. I think you should zoom out a bit and try to recognize that AI isn't stagnant.
Voice recognition and translation programs to years before they were appropriate for real-world applications. AI is also going to require years before it's ready. But that time is coming. We haven't reached a 'ceiling' for AI's capabilities.
Breakthrough technological development usually can be described as a sigmoid function (s-shaped curve), while there is an exponential progress in the beginning, it usually hit a climax then slow down and plateau until the next breakthrough.
There are certain problem that are not possible to resolve with the current level of technology for which development progress has slowed to a crawl, such as level 5 autonomous driving (by the way, better public transport is a way less complex solution.), and I think we are hitting the limit of what far transformer based generative AI can do since training has become more and more expensive for smaller and smaller gains, whereas hallucination seems to be an inherent problem that is ultimately unfixable with the current level of technology.
One thing that I think makes AI a possibility to deviate from that S model is that it can be honed against itself to magnify improvements. The better it gets the better the next gen can get.
that is a studied, documented, surefire way to very quickly destroy your model. It just does not work that way. If you train an llm on the output of another llm (or itself) it will implode.
Also at best it's an refinement, not a new sigmoid. So are new hardware/software designs for even faster dot products or advancements in network topology within the current framework. T3 networks would be a new sigmoid but so far all we know is why our stuff fundamentally doesn't scale to the realm of AGI, and the wider industry (and even much of AI research going on in practice) absolutely doesn't care as there's still refinements to be had on the current sigmoid.
It really is incredible how much this rhymes with the crypto hype. To be fair, the technology does actually have uses but, as someone in the latter category, after I saw it in action, I quickly felt less worried about my job prospects.
Fortunately, enough people in charge of staffing seem to have listened to people with technical knowledge to not make my earlier prediction (mass layoffs directly due to LLMs, followed by mass, panicked re-hirings when said LLMs ruined the business) come true. But, the worry itself, along with the RTO pushes (not to mention exploitation of contractors and H1B holders) really underscore his desperately the industry needs to get organized. Hopefully, what's going on in the games industry with IATSE gets more traction and more of my colleagues on the same page but, that's one area where I'm not as optimistic as I'd like to be - I'll just have to cheer on SAG, WGA, and UAW for the time being.
Absolutely agreed. There's a surprising amount of good in the open source world that has come from otherwise ethically devoid companies. Even Intuit donated the Argo project, which has evolved from a cool workflow tool to a toolkit with far more. There is always the danger of EEE, however, so, we've got to stay vigilant.
A powerful tool maybe, but useless
If your drill needs a nuclear plant and monthly subcription to drill a hole, it's a shitty tool
Going to have to disagree with you there. I've gotten plenty of use out of chat GPT in multiple scenarios. I find it difficult to imagine what exactly you think is useless about it because it seems so indispensable to me at this point.
Indispensable, nothing less. lmao
Have fun when they decide to multiply the price x10 and you are too dependant to have an alternative, or when it becomes stupid or malevolent 👍
Sorry, I'm not sure I understand how that makes it useless. I get the feeling that you just want to feel smug, so if it makes you feel better go ahead, I guess.
Because it's too fragile and not ready to be use at scale without causing massive damage
Not useless for now (even if i'd like to know more about the domains where it's really "indispensable"), but as useless as a drill with a dead battery the day they decide to cut it.
I don't find it future-proof, as impressive as some results are
Nowdays LLM can be ran on consumer hardware, so the "dead battery" analogy fall short here too.
With the same efficiency ? I'm interested in an example
Why everyone using these crappy SaaS then ?
Llama 2 and its derivatives, mostly. Simple local ui available here.
Not as good as chatGPT 3.5 in my experience. Just kinda falls apart on anything too complex, and is a lot more likely to get things wrong.
I tried it out using the 'Open-Orca/OpenOrcaxOpenChat-Preview2-13B' 4 bit 32g model. Its surprisingly fast to generate. It seems significantly faster than ChatGPT on my 3060. (with ExLlama)
There are also some models tuned specifically to actually answer your requests instead of the 'As an AI language model' kind of stuff.
Edit: just tried a newer model and its a lot better. (dolphin-2.1-mistral-7b)
For the same reason SaaS is popular in general: yes, you could get a VPS, install all the needed software on it, keep it up to date, oor you could pay a company to do all that for you.
You sound like the people who thought credit cards would never replace cash.
And you sound like the people who thought cryptos would replace credit cards ;)