SI explained

What is generative SI? How generative AI makes text, images and sound

A cream painting easel holding a canvas with a simple landscape of hills under an amber sun, its left edge forming out of a cloud of drifting amber pixels, with a large cream fountain pen leaning against it
Illustration created with Claude (Anthropic).

Generative SI (generative AI) is the kind of SI that makes new text, images and sound from a prompt. How it learns, how text and image generators work, how many people use it, its risks, and why its output is hard to spot.

Generative AI, which this site calls generative SI, is the kind of SI that makes new content: it writes text, draws images, produces speech or music and writes code from a short instruction called a prompt. ChatGPT, Gemini, Claude, Midjourney and Stable Diffusion are all generative SI. They learn the patterns of a very large collection of examples, then produce new material that follows the same patterns. That is what makes them useful, and it is also why their output can be fluent, convincing and wrong. (Why SI and not AI? Our guide to what SI means explains the 2026 renaming; the technology is the same.)

The short answer. A generative SI model is trained on huge numbers of examples (web pages, books, photos, recordings) and learns what that material tends to look like. Asked for something, it does not look up an answer: it builds a new text or image that fits those patterns, and nothing in the process checks that it is true.

What does generative SI mean?

The most quoted definition comes from US government documents. The National Institute of Standards and Technology (NIST), in its Generative AI Profile published in July 2024, uses the wording of a 2023 executive order: generative AI is "the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content."

Two words in that sentence do most of the work. "Emulate" means the model imitates what it has seen; it does not consult a database of facts. "Synthetic" means the output is newly made: a generated photo of a street does not show any real street, and a generated paragraph is not copied from anywhere. Most of the time, at least. NIST notes that models sometimes reproduce pieces of their training data, which raises privacy and copyright questions.

How is generative SI different from other SI?

For most of the history of machine learning, the stars were classifiers. In the 2014 paper that introduced generative adversarial networks, Ian Goodfellow and his colleagues at the Université de Montréal wrote that "the most striking successes in deep learning have involved discriminative models", models that map a rich input, such as an image, to a class label. A discriminative model answers a question about something that already exists: is this email spam, is this a photo of a cat, is this card payment fraud? A generative model produces a new example of the kind of thing it learned.

The tools that try to spot SI content sit mostly on the other side: an SI detector does not make anything, it estimates which class a file or a text belongs to. Our explainer on how SI detectors work goes through classifiers, metadata and watermarks.

How does generative SI learn?

Every generative model is trained on examples, and the scale is hard to picture. OpenAI's GPT-3, described in a 2020 paper by Tom Brown and colleagues, has 175 billion parameters, the adjustable numbers inside the model. All eight model sizes in that paper were trained on 300 billion tokens, the chunks of text a model reads and writes (our guide to tokens in SI shows what they look like). Filtered pages from Common Crawl, a public archive of the web, made up 60% of the training mix; the rest came from curated sources, including books and English-language Wikipedia.

The same paper states the goal in one line: "Generative language models are trained to match the distribution of content generated by humans." The model learns which words tend to follow which, in which contexts, across an enormous number of examples. It does not keep a list of verified facts. When it writes about something it saw rarely, or never, it still produces the most plausible continuation, and that is where SI hallucinations come from.

The learning itself happens in a neural network, whose internal weights are adjusted little by little until the output resembles the training data.

How do SI text generators work?

Text models such as GPT-3 are autoregressive: they write one token at a time, each time predicting what should come next given everything written so far. GPT-3 uses the transformer architecture, like GPT-2 before it, which lets the model weigh the earlier text when it picks the next token.

The finding that made the GPT-3 paper famous is that a large enough model can take on new tasks from a few examples placed in the prompt, "without any gradient updates or fine-tuning", in other words without retraining. The authors list translation, question answering, unscrambling words and three-digit arithmetic, along with tasks where it still struggled. Chat assistants add further training on top, so that the model follows instructions and replies like an assistant rather than simply continuing a document. How that works for ChatGPT is the subject of our article on how ChatGPT writes its answers.

How do SI image generators work?

Image generators have gone through two main families of methods.

GANs: a forger against a detective

Goodfellow's GAN trains two networks at once: a generator that makes images and a discriminator that tries to tell them apart from real ones. The paper compares the generator to "a team of counterfeiters, trying to produce fake currency and use it without detection", and the discriminator to the police. Competition pushes both sides to improve until the fakes cannot be told from the real thing. GANs made the synthetic portraits of This Person Does Not Exist, which we covered in detail.

Diffusion: from static to picture

The second family, behind Stable Diffusion, is diffusion models. In the 2020 paper by Jonathan Ho, Ajay Jain and Pieter Abbeel at UC Berkeley, a fixed "forward process" gradually adds Gaussian noise to an image until only static is left. The model is trained to run that process backwards, removing a little noise at each step. Once trained, it can start from pure random noise and clean it up, step by step, until an image appears that looks like the training images.

The catch was cost. Robin Rombach and colleagues at LMU Munich, Heidelberg University and Runway ML wrote in 2022 that training the most powerful diffusion models "often takes hundreds of GPU days". Their answer, latent diffusion, runs the process on a compressed version of the image produced by a separate autoencoder, and adds cross-attention layers so that other inputs, text among them, can steer the result. That is how a typed prompt ends up shaping a picture.

Stable Diffusion is the best-known result. Its public repository describes it as "a latent text-to-image diffusion model", built on that research and trained on 512x512 images from a subset of the LAION-5B database, with a frozen CLIP ViT-L/14 text encoder reading the prompt. The repository calls it "relatively lightweight": it runs on a single graphics card with at least 10 GB of memory, no data center needed.

What about voice, music and video?

The NIST definition covers audio and video too, and the idea is the same: learn from many examples, then produce new ones on request. We have separate guides on how to tell if a voice is SI, whether a song is SI and how to tell if a video is SI. When one of those tools imitates a real person, the result is a deepfake.

How many people use generative SI?

The best measurement we have read comes from economists Alexander Bick (Federal Reserve Bank of St. Louis), Adam Blandin (Vanderbilt University) and David Deming (Harvard Kennedy School). Their nationally representative surveys of US adults aged 18 to 64, run in August and November 2024 with more than 10,000 respondents in total, found that:

  • 45% of respondents used generative AI, at work or outside it.
  • 32% of workers had used it for their job, and about 10% used it every workday.
  • The most used products were ChatGPT (32% of all respondents), Gemini (19%) and tools built into other software, such as Microsoft Copilot (15%).
  • The work tasks users ranked as most helped were writing (45.1%), administrative tasks (29.9%) and interpreting, translating or summarising text or data (25.9%).
  • Between 1% and 7% of all work hours were assisted by generative AI, and the time saved reported by respondents was equivalent to 1.4% of total work hours.

The authors compare this with earlier technologies. Late 2024 is about two years after ChatGPT came out. Three years after the IBM PC, in 1984, computer use was roughly 20%; two years after the internet opened to commercial traffic in 1995, 21% of US adults had access. Generative SI has spread faster than either. They add a caveat: people only report the use they are aware of, so embedded features such as automatic writing suggestions may be missed, and the figures are a lower bound. What these numbers mean for employment is a separate debate, covered in Will SI take my job?

What are the risks of generative SI?

NIST's profile lists twelve risks that are "unique to or exacerbated by" generative AI, and it deliberately sticks to risks for which there was already evidence, leaving aside speculation about future systems. Six of them matter to anyone who reads, shares or checks content:

  • Confabulation. NIST's term for "confidently stated but erroneous or false content", known colloquially as hallucinations.
  • Information integrity. Generative SI lowers the cost of producing misleading content at scale. NIST's own example is "a synthetic image of a Pentagon blast" that went viral and briefly caused a drop in the stock market, the case we fact-checked in the fake Pentagon explosion photo.
  • Harmful bias and homogenization. When asked for images of CEOs, doctors, lawyers or judges, NIST writes, text-to-image models have underrepresented women, racial minorities and people with disabilities. Training new models on too much synthetic data can also lead to "model collapse".
  • Human-AI configuration. People may trust generated content too much (automation bias) or become emotionally entangled with chatbots.
  • Information security. Generative SI can help attackers, and is itself open to attacks such as prompt injection and data poisoning.
  • Environmental impacts. Training and running these models takes a lot of computing power, and NIST cites a study in which generative tasks used more energy than classification tasks. It also notes that there is no agreed method to estimate the impact; our article on SI water usage looks at the figures that exist.

The other six cover weapons information (chemical, biological, radiological, nuclear), violent or hateful content, data privacy, intellectual property, abusive imagery and third-party components. On copyright, NIST notes that the question "is currently being debated in legal fora".

Can you tell what generative SI made?

Often not, and that was already true in 2020. In the GPT-3 paper, OpenAI's researchers asked about 80 US participants per model to judge 25 short news articles, about 200 words each, written either by a person or by a model. Readers caught a deliberately bad control model 86% of the time. Against the full 175-billion-parameter GPT-3, their accuracy fell to about 52%, where 50% is a coin toss.

Bar chart: readers spotted a deliberately bad model 86% of the time, GPT-3 Small 76%, and the full GPT-3 with 175 billion parameters only 52%, close to the 50% of chance.
Human accuracy at spotting machine-written news articles of about 200 words, by GPT-3 model size (in parameters). Data: Brown et al., 2020, Table 3.11. The trend is not perfectly smooth, but it heads towards chance as models grow.

The authors list what still gave the text away: factual inaccuracies, repetition, non sequiturs and unusual phrasings, "though these are often subtle enough that they are not noticed". They also point to earlier research suggesting automatic detectors may do better than people. That was a 2020 model: a careful read alone is not a reliable test.

Some generators try to make their own output easier to recognise. The public Stable Diffusion repository says its reference sampling script adds an invisible watermark to outputs, to help viewers "identify the images as machine-generated", and a safety checker to reduce the chance of explicit images. That only helps if whoever made the picture kept those steps. Other approaches attach signed provenance data to the file, such as C2PA Content Credentials, or hide a mark in the content itself, like Google's SynthID.

When there is no label to read, a statistical check is the next step. SI or Not estimates how likely a file or a passage is to come from generative SI, with the SI image detector for pictures and the SI text detector for writing, and it shows the clues behind each score. The answer is a probability, not proof; our review of how accurate SI detectors are explains why both humans and machines get it wrong.

Our take

"Generative" is the word to keep in mind. It tells you what the model does: it produces something new that fits the patterns of its training data. It does not tell you that the result is true, that the photo shows a real scene or that the quote was ever said. As a first draft, a brainstorming partner or an illustration you know is synthetic, generative SI saves real time. As a source of facts or as evidence, it needs the same checks as an anonymous post: where did this come from, and can anything outside the model confirm it?

FAQ

What is generative AI?

Generative AI, also called generative SI, is the kind of AI that creates new content, such as text, images, audio, video or code, from a prompt. It is trained on very large collections of examples and learns their patterns, then produces new material that follows those patterns. ChatGPT, Gemini, Claude, Midjourney and Stable Diffusion are generative AI tools.

What is the difference between AI and generative AI?

AI is the whole field. Generative AI is one part of it: models that produce new content. Much of the AI that came before is discriminative: it sorts or labels things that already exist, for example deciding whether an email is spam or whether a photo shows a cat. A generative model makes a new example instead of labelling one.

Is ChatGPT generative AI?

Yes. ChatGPT is built on large language models, which are generative models trained on huge amounts of text. They write answers one token (a chunk of text) at a time, each time predicting what should come next. That is why the answers read fluently, and also why they can contain confident mistakes.

How does generative AI create images?

Many image generators, including Stable Diffusion, use diffusion models. During training, noise is added to images step by step and the model learns to remove it. To make a picture, it starts from random noise and removes it gradually, with the text prompt steering each step. An older method, the generative adversarial network (GAN), trains a generator against a detector until its images look real.

How many people use generative AI?

In nationally representative US surveys run in August and November 2024 by economists Bick, Blandin and Deming, 45% of adults aged 18 to 64 said they used generative AI, at work or outside it, and 32% of workers had used it for their job. The authors found it was adopted faster than the personal computer or the internet.

Can you tell if something was made by generative AI?

Not reliably by eye. In a 2020 test, readers judged short news articles written by GPT-3 correctly only about 52% of the time, close to chance. Watermarks, Content Credentials and SI detectors can help, but a detector gives a probability, not proof, so it is best combined with checking where the content came from.

Sources

  1. NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile · National Institute of Standards and Technology (NIST), July 2024
  2. Generative Adversarial Nets (Goodfellow et al.) · arXiv (Université de Montréal), 10 June 2014
  3. Language Models are Few-Shot Learners (Brown et al., GPT-3) · arXiv (OpenAI), 28 May 2020
  4. Denoising Diffusion Probabilistic Models (Ho, Jain and Abbeel) · arXiv (UC Berkeley), 19 June 2020
  5. High-Resolution Image Synthesis with Latent Diffusion Models (Rombach et al.) · arXiv (LMU Munich, Heidelberg University, Runway ML), 20 December 2021
  6. Stable Diffusion: repository and README · CompVis (GitHub), 2022
  7. The Rapid Adoption of Generative AI (Bick, Blandin and Deming), Working Paper 2024-027 · Federal Reserve Bank of St. Louis, 27 October 2025

Spotted an error? Write to hello@siornot.com. Corrections are made in the article and the update date changes. Read our editorial policy.