this post was submitted on 23 Nov 2023
1 points (100.0% liked)

Machine Learning

1 readers
1 users here now

Community Rules:

founded 1 year ago
MODERATORS
 

I had an interview question regarding LLM. How exaclty do you deploy LLM, what are your consideration in terms of speed, resource, imbalance load, and all that stuff?

top 4 comments
sorted by: hot top controversial new old
[–] pm_me_your_pay_slips@alien.top 1 points 11 months ago

If you're asking that question here, you ma not be qualified for the job.

[–] Slightlycritical1@alien.top 1 points 11 months ago

I think you’re looking at the problem wrong by focusing just on the LLM aspect of it. If you’re deploying any type of application then it will depend on the demand you’re expecting from users and the use cases the application will be used. A failure rate for medical applications is probably a lot more important than a low budget game service.

[–] milkteaoppa@alien.top 1 points 11 months ago (1 children)

Call the API using requests.post(..)

[–] HPLaserJetM140we@alien.top 1 points 11 months ago