tothatl

joined 10 months ago
[โ€“] tothatl@alien.top 1 points 10 months ago

Lllama.cpp supports BNF grammars. You basically tell the model runner the exact format of the output and where the data goes in it and even which list of values you expect, and it will only produce output in that format.

Useful for generating JSON and for document classification.

[โ€“] tothatl@alien.top 1 points 10 months ago (3 children)

Ironic if this was done to try to remove a monopolistic entity controlling AI and to slow things down.

Because now a monopolistic company has what it needs to control AI and accelerate in whatever direction it likes, regardless of any decel/EA feelings.

Yes, some of this know-how will fall over the industry and other labs, but few places in the world can offer the big fat checks Microsoft will offer these people. Possibly NVIDIA, Meta and Google and a few more, but many of them are former employees of those firms to begin with. Google in particular, has been expelling any really ambitious AI people for a while.