this post was submitted on 09 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
 

Hi all,

I'm trying to make it work Guidance with LM Studio. This is my code:

from langchain.llms import OpenAI
import guidance

llm = OpenAI(
        openai_api_key="NULL",
        temperature=0,
        openai_api_base="http://localhost:1234/v1",
        max_tokens=-1
)

guidance.llm = llm

program = guidance("""Tweak this proverb to apply to model instructions instead.

{{proverb}}
- {{book}} {{chapter}}:{{verse}}

UPDATED
Where there is no guidance{{gen 'rewrite' stop="\\n-"}}
- GPT {{#select 'chapter'}}9{{or}}10{{or}}11{{/select}}:{{gen 'verse'}}""")

# execute the program on a specific proverb
executed_program = program(
    proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
    book="Proverbs",
    chapter=11,
    verse=14
)

I'm getting this error at the line program = guidance(...): guidance is not callable. Can you explain me what I'm missing?

you are viewing a single comment's thread
view the rest of the comments
[–] spackenheimer@alien.top 1 points 10 months ago

I get a different Error: "guidance\library\_select.py", line 121, in recursive_select for k,v in logprobs_result["top_logprobs"][0].items(): TypeError: 'NoneType' object is not subscriptable Error in program: 'NoneType' object is not subscriptable"
LM Studio sent a response " "text": "there is no safety,\nbut with many advisers there is success.", "logprobs": null, "
I guess the "logprobs" are the Problem here. (I have no Idea what I'm doing, TBH).
I modified only Line 4 of the Example to : guidance.llm = guidance.llms.OpenAI("text-davinci-003", api_key = "foobar", api_base = "http://localhost:1234/v1")