this post was submitted on 21 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
What’s the prompt you guys use for function calling in LLMs? (alien.top)
submitted 10 months ago by Tasty-Lobster-8915@alien.top to c/localllama@poweruser.forum
2 comments fedilink hide all child comments
 

My understanding of LLM function calling is roughly as follows:

  1. You “list” all the functions the model can call in the prompt
  2. ???
  3. The model knows when to return the “function names” (either in json or otherwise) during conversation

Does anyone have any advice or examples on what prompt should I use?

you are viewing a single comment's thread
view the rest of the comments
[–] amemingfullife@alien.top 1 points 10 months ago

I usually add context free grammar to make sure it always outputs valid JSON. Here’s the json.gbnf I use.

root   ::= object
value  ::= object | array | string | number | ("true" | "false" | "null") ws

object ::=
  "{" ws (
            string ":" ws value
    ("," ws string ":" ws value)*
  )? "}" ws

array  ::=
  "[" ws (
            value
    ("," ws value)*
  )? "]" ws

string ::=
  "\"" (
    [^"\\] |
    "\\" (["\\/bfnrt] | "u" [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F]) # escapes
  )* "\"" ws

number ::= ("-"? ([0-9] | [1-9] [0-9]*)) ("." [0-9]+)? ([eE] [-+]? [0-9]+)? ws

# Optional space: by convention, applied in this grammar after literal chars when allowed
ws ::= ([ \t\n] ws)?