this post was submitted on 20 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
How to prompt engineering non-openai models to output structured json (alien.top)
submitted 11 months ago by backtrack432@alien.top to c/localllama@poweruser.forum
2 comments fedilink hide all child comments
 

Does anyone know how function calling works under the hood?

you are viewing a single comment's thread
view the rest of the comments
[–] tothatl@alien.top 1 points 11 months ago

Lllama.cpp supports BNF grammars. You basically tell the model runner the exact format of the output and where the data goes in it and even which list of values you expect, and it will only produce output in that format.

Useful for generating JSON and for document classification.