this post was submitted on 24 Jul 2024
65 points (94.5% liked)
Tech
1526 readers
24 users here now
A community for high quality news and discussion around technological advancements and changes
Things that fit:
- New tech releases
- Major tech changes
- Major milestones for tech
- Major tech news such as data breaches, discontinuation
Things that don't fit
- Minor app updates
- Government legislation
- Company news
- Opinion pieces
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Anyone can download, but practically no one can run it.
With the absolute highest memory compression settings, the largest model that you can fit inside of 24GB of VRAM is 109 billion parameters.
Which means even with crazy compression, you need at the very least, ~100GB of VRAM to run it. That's only in the realm of the larger workstation cards which cost around $24,000 - $40,000 each, so y'know.
Ouch.