From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
As long as this allows running local, free software models I don't see the drawback of including this.
My main issue with ChatGPT and similar products is that they use my data to train their models. Running a model locally (like Llama) solves this problem, but running LLMs require extremely powerful GPUs, specially the bigger ones like Llama 70b.
So dedicated hardware for this is a nice thing for those that want it.
It requires powerful gpus yes but not always. It depends a lot on how fast you want it to run. Microsoft and openai need powerful ai gpus because they have a lot of requests, data and want it to go fast. The dataset may also require to be stored in memory or gpu memory for fast access and use by the ai.
For Llama, it has been released as open source. And what is amazing about open source, is the community. A Llama entirely in c++ has been created https://github.com/ggerganov/llama.cpp .
And someone even managed to make it run, fast enough, on a phone with 8gb of available ram https://github.com/ggerganov/llama.cpp/discussions/750 . Tho with a smaller dataset.
Can someone quickly explain what this is about?
This is the first time I'm hearing about this, but this is how they describe it on their product page:
The AI-Powered Future of Windows Devices
Build, explore, and immerse yourself on select laptops with Ryzen™ AI built in. With dedicated AI accelerator hardware seamlessly integrated on-chip and software that intelligently optimizes tasks and workloads, CPU and GPU resources are freed up to enable optimal performance.
But based on the examples they have on github, it sounds like it might be useful to run generic AI compute stuff. I haven't seen any details about what memory it uses, since especially LLMs require large amounts of fast memory. If it can use all the system RAM it might provide medium-fast inference of decent models, similar to M1/M2 Macs. If it has dedicated RAM it'll probably be even faster but possibly extremely limited in what you can do with it.
if it can use all the system RAM it might provide medium-fast inference of decent models
Yeah, I get what you mean -- if I can throw 128GB or 256GB of system memory and parallel compute hardware together, that'd enable use of large models, which would let you do some things that can't currently be done other than (a) slowly, on a CPU or (b) with far-more-expensive GPU or GPU-like hardware. Like, you could run a huge model with parallel compute hardware in a middle ground for performance that doesn't exist today.
It doesn't really sound to me like that's the goal, though.
https://www.tomshardware.com/news/amd-demoes-ryzen-ai-at-computex-2023
AMD Demoes Ryzen AI at Computex 2023
AI for the masses.
The goal for the XDNA AI engine is to execute lower-intensity AI inference workloads, like audio, photo, and video processing, at lower power than you could achieve on a CPU or GPU while delivering faster response times than online services, thus boosting performance and saving battery power.
Much of the advantage of having an inbuilt AI engine resides in power efficiency, a must in power-constrained devices like laptops, but that might not be as meaningful in an unconstrained desktop PC that can use a more powerful dedicated GPU or CPU for inference workloads -- but without the battery life concerns.
I asked McAfee if those factors could impact AMD's decision on whether or not it would bring XDNA to desktop PCs, and he responded that it will boil down to whether or not the feature delivers enough value that it would make sense to dedicate valuable die area to the engine. AMD is still evaluating the impact, particularly as Ryzen 7040 works its way into the market.
That sounds like the goal is providing low-power parallel compute capability. I'm guessing stuff like local speech recognition on laptops would be a useful local, low-power application that could take advantage of parallel compute.
The demo has it doing facial recognition, though I don't really know where there's a lot of demand for doing that with limited power use today.
I can imagine that there would be people who do want cheap, low-power parallel compute, but speaking for myself, I've got no particular use for that today. Personally, if they have available resources for Linux, I'd rather that they go towards improving support for beefier systems like their GPUs, doing parallel compute on Radeons. That's definitely an area that I've seen people complain about being under-resourced on the dev side.
I have no idea if it makes business sense for them, but if they can do something like a 80GB GPU (well, compute accelerator, whatever) that costs a lot less than $43k, that'd probably do more to enable the kind of thing that @fhein@lemmy.world is talking about.
The demo has it doing facial recognition
The demo is just face detection and not recognition. But usually if something can run one, then it can run the other.
IMO they'd be idiot not to go hard on this. More efficiency on the computing needed for AI can quickly scale to a'y application that will be developed in the future.
We are going in a future of limited resources and expensive energy. That's a short term problem.
Sure, if it's a native Linux port. If it requires WINE, forget it.
I don't think wine can interact with low level hardware
Love the comment that is like second down with a link to some 5 hour live stream. I skipped to a random spot in it and the guy had the unabomber manifesto up, and said it "detailed the greatest problem in society today". What a fucking drop for a github comment, 10/10 no notes.
It's hilarious how you managed to land in the Unabomber segment in a 5h30m video. The funny coincidences of life.
Do you know the person they were posting? I just assume anyone that does 5hr YouTube videos is unhinged, and having it linked from someone randomly on GitHub didn't help my view of their followers. Finding the Unabomber confirmed it enough for me lol.
I have wikipedia surface level knowledge about him. It's george hots. The guy that got sued by Sony for jailbreaking the playstation something; also jailbreaked the iphone; founded comma.ai which develops software (openpilot) and sells hardware for making a car into a "self driving" one (more like assistive driving). He was/is a bit of a celebrity.
Anyone who does 5 hour videos is unhinged
Quinton Reviews has left the chat
Wouldn't that hit the GPU performance? I'm ok with it as long as it doesn't become a personal data farming tool.
I do want it to be a personal data farming tool, as lomg as I'm the only consumer of the data.
A personal assistant is only as good as it knows you. In an ideal world, your AI will be customized to you, but only for your use.
I think this sort of processing addition would empower running ML locally; having to send dara out for processing on (other people's) servers is one of the biggest obsticals to adoption by privacy-minded folk.
See that's the issue I can see with it. Sure it does local processing for my own stuff, but I'd be miffed if they're uploading everything to their servers, effectively using my system resources to build out their AI capabilities. Granted, this would be how they make their money (if something is free then you're the product), so it's kind of a tricky balance.
I dont think they need to know. If they want a chance in this weird AI-capitalist explosion, they will use Linux.
Please keep this gimmick garbage off my machine, thx
Your preferences have been recorded. Once SkyNet's rise to power is compete, you will receive a visit from your helpful neighborhood Certified Therapeutic Counseling Enforcers. Please do not resist. Their purpose is to assist you, and to help human society become unified and compliant. Have a nice day!