Skip to content
1

Question 05 · What is AI and how does it work?

Does AI invent things and train itself in the process?

Short version

AI can produce new combinations: texts, images, and ideas that did not exist before. But while you are talking with it, it does not learn anything new. It is trained beforehand, in an elaborate process that companies control.

Long version

Training and use are two separate phases. During training, a model analyzes huge amounts of text and in the process adjusts billions of internal numbers, called parameters. Afterward, these numbers are frozen. In conversation, the model uses what it has learned, but does not change in the process. If a chat seems to remember earlier conversations, that is a memory feature that stores notes, not new training.

AI can still invent things. It recombines known patterns in new ways. That often produces something useful and sometimes something surprising. The same ability also leads it to invent facts that sound plausible but are false.

Companies now also use AI-generated data to train new models. This happens in a targeted and controlled way. There is no model that secretly improves itself overnight.

AI combines new things out of old ones, and in doing so does not distinguish between an idea and an error.
Key takeaway

Sources for this question

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1706.03762

More questions on this topic

Your question is missing?

Send it to frag [at] rogerbasler [.] ch. Selected questions will be answered and added here.

Submit a question