Machine Learning

1 readers

1 users here now

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 11 months ago

MODERATORS

submitted 10 months ago by Ok_Cartographer5609@alien.top to c/machinelearning@academy.garden

10 comments fedilink hide all child comments

I want to know the tools and methods you use for the observability and monitoring of your ML (LLM) performance and responses in production.

you are viewing a single comment's thread
view the rest of the comments

[–] kennysong@alien.top 1 points 10 months ago

If you're open to using an open source library, you can use LangCheck to monitor and visualize text quality metrics in production.

For example, you can compute & plot toxicity of users prompts and LLM responses from your logs. (A very simple example here.)

(Disclaimer: I'm one of the contributors of LangCheck)