We used a similar system in our project here,we had the llm also identify which location on the map the actions should be done at. https://youtu.be/e2UGwXpu_zc?si=hrV0Vnq7C-9BZ42i
this post was submitted on 30 Oct 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
How do you get it to work with ExLlama or ExLlamav2?
It works beautifully with llama.cpp, but with GPTQ models the responses are always empty.
zephyr-7B-beta-GPTQ:gptq-4bit-32g-actorder_True:
{"emotion": "surprised", "affectionChange": 0, "location": "^", "feeling": "^", "action": [],"reply": "^"}
zephyr-7b-beta.Q4_K_M.gguf:
{"emotion":"surprised","affectionChange":0.5,"location":"office","feeling":"anxious","action":["looking around the environment"],"reply":"Hello! I'm Lilla, nice to meet you!"}
This is my grammar definition:
root ::= RoleplayCharacter
RoleplayCharacter ::= "{" ws "\"emotion\":" ws Emotion "," ws "\"affectionChange\":" ws number "," ws "\"location\":" ws string "," ws "\"feeling\":" ws string "," ws "\"action\":" ws stringlist "," ws "\"reply\":" ws string "}"
RoleplayCharacterlist ::= "[]" | "[" ws RoleplayCharacter ("," ws RoleplayCharacter)* "]"
Emotion ::= "\"happy\"" | "\"sad\"" | "\"angry\"" | "\"surprised\""
string ::= "\"" ([^"]*) "\""
boolean ::= "true" | "false"
ws ::= [ \t\n]*
number ::= [0-9]+ "."? [0-9]*
stringlist ::= "[" ws "]" | "[" ws string ("," ws string)* ws "]"
numberlist ::= "[" ws "]" | "[" ws string ("," ws number)* ws "]"
Do you need to "prime" the models using prompts to generate the proper output?