User:Rongzhou/BehindScenes/Transformer is a super-charged next word prediction algorithm

I have long wanted to write an intuition for Transformers.

Transformer is a super-charged next word prediction algorithm is far from my first attempt. I tried many times previously, experimenting with various analogy: an alien trying to imitate humans without understanding a word of English, a monkey learning to type... None felt quite right, and I scraped the draft in the end every time.

In the end, while I was texting my aunt, I got bored, and I started typing randomly by clicking on auto-complete suggestions, as I used to do as a child to kill time (yes, I used to do that for hours).

The last sentence I got was this:

The expected arrival time going on is a good idea to get the border force in the week to do the first one in a row in a protestant family and I was thinking about the batteries for you to do the first half and I can do the first one then it will try to be quite happy.

However, there is something unsettlingly good about this algorithm's performance. I live in France, and I type almost entirely in French on my phone. I have often typed "famille protestante" and "arrivée", the French version of "protestant family" or "arrival", but probably never the English version. Somehow, the algorithm understood that "famille protestante" is "protestant family", and "arrivée" means exactly the same thing as "arrival".

It occurred to me immediately that some kind of ANN (artificial neural network) is at work, and quite likely, some kind of miniature Transformer.Transformers work with numbers. They rely on encoders and decoders to translate between natural language and numbers, and as such they don't care that much what language you are writing in: your words are programmatically encoded into numbers, processed by the transformer, then spit out as different numbers, which are then decoded into natural language words outside the Transformer programmatically. The cross-language intelligence the next-word prediction algorithm demonstrated, is a characteristic trait of Transformers.

Obviously, if I was looking for a working example of a Transformer, this is a terrible example. The performance is far-away from human-level, unlike the imaginary super-charged version in the Transformer is a super-charged next word prediction algorithm story. There are highly coherent sub-sentence structures consisting of 2-4 words, but the sentence as a whole makes no sense. Worse, this particular algorithm does not even attempt to predict punctuation. It goes on predicting the next word based on the few ones immediately before and how often I typed different words, until it can absolutely no longer generate useful output, then it puts a period on the sentence.

Nevertheless, I have found more than my perfect analogy: I found a working Transformer that anyone with a smartphone can relate to.

Once I got home, I started sketching out the story. The first version did not look quite right: so I once again scrapped it. I started writing a second version, and as usual I hit quite quickly a writer's block. With considerable effort, I overcame it. Soon after, I hit a second block, and decided to take a break and go get some snacks. As I was eating my favourite snack, it suddenly occured to me: what if, I turn the intuition into a live demonstration of a Transformer's capacity to continue existing content?

I quite liked the idea. So I decided to stop my story at an intersection relatively easy to pick up from, right after the first "The message came out as this:". Then I fed my half-story to a Transformer model I had at hand, which happens to be a deepseek-v4-flash with a generic system prompt, accompanied by a very simple task prompt:

Continue the story following my writing style.

And within seconds I had my first story-completion draft.

I kind of liked it, especially that it tried at the end to reveal real-world data source for machine-learning in our time: large corpus of data scrapped from everywhere, books, movie transcripts, but also real conversations held between humans, usually anonymised, but sometimes not so much so.

However, it was not good enough for me. It read like a conspiracy theory, as if Transformers trained on public data are capable of somehow "inferring out" your secret family drama that you never told it about. They cannot. Transformers, are mathematicians, not magicians. They only know as much about you as you told them, unless you are a very well-known person that everyone writes about on social media/in the newspaper, etc. Even then, they would not know about your secret family drama, unless you or someone wrote extensively about it and it somehow made into the Transformer's training data.

So I decided to follow the LLM rule-of-thumb: if you do not like the response, regenerate it. I started a new chat (to prevent context contamination), and pasted in exactly the same prompt with my half-story, and got my second response.

My Transformer attempted a new angle this time. Instead of going down the privacy concern rabbit role, it transitioned the story into a somewhat good explanation about attention mechanisms in Transformers. Although it is an interesting direction that I have never thought of taking, I decided to only take the first few lines of this version, far before it started talking about attention.

For one, the story-completion had an obvious self-contradiction:

The algorithm has now carried two entire conversations on your behalf, and nobody noticed. Not your aunt, not your date, not your boss on that one afternoon you were "in a meeting" but were actually napping.

The Transformer was at some point talking about two entire conversations, which was already not quite obvious based on the context up until that point; then it hinted in the immediate next sentence that the super-charged algorithm has, in fact, at least done four conversations on "your" behalf (1. aunt 2. date 3. boss 4. meeting).

This kind of self-contradiction is a somewhat typical problem of Transformer-generated creative writing.

In a sense, Transformers today (2026) understand the stories better than humans, as they process each word in a story with complex mathematical algorithms permitting a precise determination of the semantic relationship between any two given words and the statistically most-likely ways the story continues onto the next word, whereas us humans can seldom explain why we decided to write a sentence the way we wrote it or continue a story down one particular direction but not any other based on what we have already written up until that point. And it is for that reason Transformers today are better at grammar than most humans are, as they understand really well, via mathematics, the safest, if not the most interesting arrangements to turn words into sentences.

What keeps Transformers from maintaining a consistent flow in creative stories, is the lack of real-world understanding typical to humans. Transformers understand well how "fridge" relates to "microwave", as they observe in their training data those being listed often in parallel quite close to each other. They also quite likely understand how "fridge" and "microwave" are both subclasses of "kitchen appliances", as they observe in their training data fridges and microwaves being listed as child items of "kitchen appliances" (in appliance store catalogues, etc.). They can even describe a fridge: "An electrical appliance that refrigerates items at sub-room temperature for their conservation." However, they simply do not understand what a fridge is: they have never seen one. Even if you show them a picture, they can only see the pixels, which are just numbers arranged in a particular way. Similarly, they don't understand temperature. They know 30°C is cooler than 100°C, and humans don't like living in 100°C environments, but they have zero idea how 100°C feels like. For them, temperatures are numbers, just like distances are numbers, and the difference between the two is that they have different units.

In my case, the Transformer failed to keep a consistent track on the number of conversations the super-charged algorithm has already handled on the user's behalf. For the Transformer, there is no inherent contradiction between the algorithm having carried out two conversations and the Transformer having conversed with at least "your aunt", "your date", and "your boss" in addition to whoever that were "in a meeting." The Transformer was cheerfully ignorant of the implication of "two conversations" in this context.

Rightfully forgiving the lack of real-world understanding of my Transformer friend, I decided to give it a third opportunity, this time giving it slightly more context by pasting in also the first part of the second attempt that I liked.

This time, my Transformer decided to play it safe: it did not attempt any further transition, ending the story with the natural conclusion of the moderately uncomfortable SMS conversation with the moderately unpleasant aunt. It did, however, try to spin the focus back on to the super-charged next-word-prediction algorithm in the end, probably because it "understood" that the algorithm is the protagonist of the story, not the aunt. The very last sentence is, unfortunately, again, a self-contradiction: "(the algorithm) now also (learns) from the words that you clicked". The claim is factually accurate, as the algorithm is indeed learning from user clicks. Sadly it is written in a way to suggest that this is a new behaviour of the algorithm's part after the conclusion of the aunt conversation.

Overall, the story-completion from the third attempt is quite strong: the Transformer depicted the classical ambivalence people have towards requests/invitations from distant family members that we could have blacklisted a long time ago if they weren't part of the family. Evidently the Transformer is unable to feel this emotion itself (it does not have less than likable distant relatives anyways), but it reproduced the ambivalence expressed by human authors in the numerous stories it read in its training data quite well. So in the end I accepted most of this story-completion, making a few tweaks here-and-there.

And Transformer is a super-charged next word prediction algorithm is born.