visarga

hey, can I do the fine-tuning on my own computer or only in your cloud?

how do you add your own gguf into ollama? it seems to be storing models as cryptic binary blobs in a folder.

yeah I was put off by the lack of mention on the base model

for now we might be able to 10x our language data, but the top quality content has already been used

beyond that I think synthetic data will rule; it needs to be validated or filtered somehow; I think we need to use agents and RL to make it high quality

no ML technique has been shown to do anything more than just mimic statistical aspects of the training set

That's ok when the agent creates its own training set, like AlphaZero. It is learning from feedback as opposed to learning from next token prediction.

Tried the models, the 13B is very slow, the 7B is speedy but a little quirky. It made the plan how to solve the task but didn't actually proceed in solving the task. It doesn't have good conversational flair.

models are born with random init,

everything comes from the training data

On top of that, many OAI people will leave spreading their inside knowledge to other companies. All the secrets will be out.

But the irony is that AI doom fears triggered this avalanche, hurting AI safety while doing it for safety. And if it wasn't fear, then it was greed, but that is not any better for our risk level.

That gives him perpetual license to train AI models on the reddit corpus.

Maybe he gets 3 people, maybe half the company. But one thing is sure - a lot of people will leave OpenAI taking their OpenAI experience to other places. That will be good in the long term, less concentration of power.

VSCode with Copilot

You already mentioned hierarchical methods but I got my best class-count agnostic clustering with fclusterdata from scipy:

labels = fclusterdata(data, t=threshold, criterion='distance')