overview for Amgadoz

what is the best 7b right now ? in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

In no specific order:

Zephyr B OpenHermes2.5 OpenChat3.5

1

Why didn't gpt4 work at first and how did they "fix it"? (alien.top)

submitted 11 months ago by Amgadoz@alien.top to c/localllama@poweruser.forum

11 comments fedilink

According to this tweet,

when gpt4 first finished training it didn’t actually work very well and the whole team thought it’s over, scaling is dead…until greg went into a cave for weeks and somehow magically made it work

So gpt-4 was kind of broken at first. Then greg spent a few weeks trying to fix it and then it somehow worked.

So why did it not work at first and how did they fix it?
I think this is an important question to the OSS community,

Are there people who have ran MI25s, MI60s, etc for LLMs? in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

u/ehartford

ORCA 2 Released open source! in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

It's source available, not open source. They don't even allow commercial usage.

Orca 2: Teaching Small Language Models How to Reason in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago (2 children)

Important: researcher only, non commercial license.

Many people here are probably getting started on machine learning, here's a blog post from Greg Brockman who recently left OpenAI with Sam Altman. I found this quite inspiring. in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

I think Karpatgy's nanogpt series is a great alternative to this.

https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ&feature=shared

Many people here are probably getting started on machine learning, here's a blog post from Greg Brockman who recently left OpenAI with Sam Altman. I found this quite inspiring. in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

OpenAI hires ml experts AND software engineers. For the first few years at OpenAI, he was much more of the second as his knowledge of ml was very minimal.

Top 3B model is a distilled llama 7B in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

Are your finetunes full training or lora/qlora?

Microsoft announced the Maia 100 AI Accelerator Chip. It's also expanding the use of the AMD MI300 in it's datacenters. Is this the beginning of the end of CUDA dominance? in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago (1 children)

Do they plan on offering Mi200 or MI300 to the public?

Can't wait to try out a GPU with more than 80gb vram

fastchat-t5 cheapest hosting? in c/localllama@poweruser.forum

[–] Amgadoz@alien.top 1 points 11 months ago

How about T4 GPU or something like 3090 from runpod? the 3090 costs around 0.5$ per hour which is around 350 dollars per month and it gives you 24 GB which should be enough for t4