Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[R] [Research] Tokenizing and processing arithmetic by AI (alien.top)

submitted 11 months ago by sean_con@alien.top to c/machinelearning@academy.garden

3 comments fedilink hide all child comments

Hello

x-posting from t/askprogramming : Original here.

There was a comment stating I might ask here as well.

Hi all

I am looking for a comparison of known AI models method of handling arithmetic operations.

How do these AI compute 2+2 ? That is to ask :

Do they perform a gradient descent, and figure out that (2+1) < (2+2) < (2+3) hence 2+2 must be between 3 and 5 ? Or do they actually invoke an arithmetic operation in the host language.

I understand that some popular AI model is closed source, and we will not know. But for the rest, is there an already compiled meta study / publications comparing how they tokenize numbers, and handle arithmetic?

Thank you.

you are viewing a single comment's thread
view the rest of the comments

[–] lotonototo@alien.top 1 points 11 months ago (1 children)

There's work on tool-assisted LMs regarding math problems: https://arxiv.org/abs/2305.15017. TL;DR: they train the LM to extract the numbers, then feed it into an calculator and return back the result to the LM during inference.

Vanilla LMs, which are neural nets, don't compute anything - they guess the result. The guess is based on previously seen distribution of results related to the given problem. I saw a paper that explored the distribution but can't find it anywhere...

[–] sean_con@alien.top 1 points 11 months ago

This is exactly the answer I was looking for. Thank you