this post was submitted on 02 Oct 2026
43 points (92.2% liked)

Technology

88816 readers
4551 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
43
Another cost for software (jeferson.me)
submitted 2 weeks ago by Shin@piefed.social to c/technology@lemmy.world
38 comments fedilink hide all child comments
 

As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.

Hope you enjoy this reading, all feedback is welcome.

you are viewing a single comment's thread
view the rest of the comments
[–] fruitycoder@sh.itjust.works 1 points 1 week ago (1 children)

We are saying the samethings but you are adding no to it.

Right, during inference random numbers are generated, that plus the numbers from input are added to the weight values and the matrix multiplication happens. If you used the same random numbers and inputs it is deterministic. For regular use you don't do that because you want a stochastic output, if you wanting to do forensics and trace what led to an output you would benifit from that determinism.

[–] Shin@piefed.social 2 points 1 week ago (1 children)

Gotcha,

But this means we need to provide not only the same query (and be sure that the tokenizer is the same) but also provide the seed for the number generation. In this case we will have a deterministic outcome. (Unless we provide the list of numbers used, which for me feels wasteful)

But at this point none of the providers have this feature.

And I don't think the open source have this also. But open source can be updated/changed.

[–] fruitycoder@sh.itjust.works 1 points 1 week ago

Not that I see either looking at opensource inference engines. vLLM supports setting the seed so if you have that you do, and maybe it's just as simple as harness setting and recording that. At least for replay this request type of operation but that does not show where in model each random numbers would have been applied.