this post was submitted on 17 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
How are people here observing their experiments and production models? (alien.top)
submitted 11 months ago by thedabking123@alien.top to c/localllama@poweruser.forum
1 comments fedilink hide all child comments
 

I'm currently working on some RAG-based tooling for some non-profits and am having difficulty doing the following things. Wondering what people are using?

  1. Tracking model performance across experiments and productized pipelines
    1. changes in test or finetuning data sets
    2. Changes in chunking strategy
    3. changes in RAG tooling (e.g. RAG Fusion or RAG-DIT)
    4. Changes in underlying models and/or finetuning strategies
  2. Tracking pipeline performance (e.g. speed, throughput, latency, etc.) as we change items laid out above

What products do you use and how do you choose them?

you are viewing a single comment's thread
view the rest of the comments
[โ€“] m18coppola@alien.top 1 points 11 months ago

I had so much success with text embeddings and retrieval, I didn't end up needing to deploy an LLM at work. I do however have a secret Mistral-Trismegistus-7B@Q4_K hosted on a retired cheapo dell optiplex with a tarot card reader system prompt that I share with my teammates ๐Ÿ˜