Machine Learning

1 readers

1 users here now

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

submitted 1 year ago by Ok_Cartographer5609@alien.top to c/machinelearning@academy.garden

10 comments fedilink hide all child comments

I want to know the tools and methods you use for the observability and monitoring of your ML (LLM) performance and responses in production.

you are viewing a single comment's thread
view the rest of the comments

[–] kennysong@alien.top 1 points 1 year ago

If you're open to using an open source library, you can use LangCheck to monitor and visualize text quality metrics in production.

For example, you can compute & plot toxicity of users prompts and LLM responses from your logs. (A very simple example here.)

(Disclaimer: I'm one of the contributors of LangCheck)