Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[D] How do you deal with LLM observability? What tools do you guys use? (alien.top)

submitted 1 year ago by Ok_Cartographer5609@alien.top to c/machinelearning@academy.garden

10 comments fedilink hide all child comments

I want to know the tools and methods you use for the observability and monitoring of your ML (LLM) performance and responses in production.

you are viewing a single comment's thread
view the rest of the comments

[–] Traditional_Swan_326@alien.top 1 points 1 year ago

Hi there, Langfuse founder here. We're building observability & analytics in open source (MIT). You can instrument your LLM via our SDKs (JS/TS & Python) or integrations (e.g. LangChain) and collect all the data you want to observe. The product is model-agnostic & customizable.

We've pre-built dashboards you can use to analyze e.g. cost, latency and token usage in detailled breakdowns.

Now, we're starting to build (model-based) evaluations right now to get a grip on quality. You can manually ingest scores via our SDKs, too. + export as .csv and via get API.

Would love to hear feedback from folks on this reddit on what we've built and feel free to message me here or at contact at langfuse dot com

We have an open demo so you can have a look around a project with sample data.