Technology

My brother, this is not just autistic people. Everyone wants this. Except the people who make the sites, because all that noise is how they make money.

If you look at a private blog, they're usually devoid of much noise.

❌️ Adding accessibility features to make the internet usable by anyone

✅️ ~~anyone~~ other computers

Woohoo, the interests of capital have coincidentally aligned with ours in this one brief moment!

I have ASD; I made several tools that explicitly convert web sources to .md and JSON.

The shitty thing is, a lot of sites - even if they have stuff available in simple, beautiful JSON format, refuse to give public access to it. Notoriously, movie session times for local cinemas. That should be a simple look up...but no.

Oh well, at least cool shit like this still exists

https://github.com/chubin/wttr.in

https://github.com/scrapy/scrapy

don’t “reader” views in web browsers essentially accomplish the same thing?

"Prove that you're a bot by factorising this large number."

There are many flavours and extensions of markdown. For example rendering of a table is not part of the standard. How newlines are handled also differ.

to have all browsers support this, this need to be made into a web standard. And all agreed by different browser vendors (like chrome, safari and firefox).

right now this might be best handled by browser extension first.

A token is basically a linguistic unit, like a word or a phrase.

LLMs don't parse text word-by-word because it would miss a lot of idiomatic meaning and other context. "Dave shot a hole in one at the golf course" might be parsed as "{Dave} {shot} {a hole in one} {at the golf course}"

They use NLP to "tokenize" text, meaning parsing it into individual tokens, so depending on the tokenizer I suppose there could be slight variations on how a text is tokenized.

Then the LLM runs each token through layers of matrices on attention heads (basically, vectors) in order to assess the probabilistic relationships between each token, and uses that process to generate a response via next-token prediction.

It's a bit more complex than that, of course. Tensor calculus, billions of weighted parameters, layers divided by hidden sizes, also matmuls, masks, softmax, and dropout. Also the "context window" which is how many tokens it can process at a time. But it's the gist of it.

But a token is just the basic unit that gets run through those processes.

Here's an OpenAI page that allows you to enter text and see how it gets tokenized:

https://platform.openai.com/tokenizer

A token is the word for the base unit of text that an LLM works with. It’s always been that way. The LLM does not directly work with characters; they are collected together into chunks less than a word and this stream of tokens is what the LLM is processing. This is also why the LLMs have such trouble with spelling questions like “how many Rs in raspberry?” — they do not see the individual letters in the first place so they do not know.

No, the LLMs do not all tokenize the same way. Different tokenizers are (or at least were once) one of the major ways they differed from each other. A simple tokenizer might split words up into one token per syllable but I think they’ve gotten much more complicated than that, now.

My understanding is very basic and out-of-date.