this post was submitted on 01 Nov 2023
1 points (100.0% liked)
Machine Learning
1 readers
1 users here now
Community Rules:
- Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
- Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
- Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
- Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.
founded 11 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The compute limit is 10^26 operations. For reference, NVIDIA trained GPT-3 on ~3500 H100s in just under 11 minutes which, assuming FP8 which is the highest op count, comes out to ~10^22 operations. With the same setup, they'd have to train for over 81 days to reach the 10^26 limit so it's likely not going to impact anyone except for those training incredibly large models.
Edit: MLPerf link
That's just the initial set of conditions for reporting requirements; the EO instructs the Secretary of Commerce to come up with new technical conditions for reporting that will supersede those.
See section 4.2 (b): https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/
Yet. Limiting this on todays compute power and requirements seems short sighted