How ChatGPT and Gemini make videos with diffusion, and why we took an AI-first route instead

Ask ChatGPT or Gemini for a short video today and, a minute later, you get something that looks filmed. Behind ChatGPT is OpenAI’s Sora; behind Gemini is Google DeepMind’s Veo. Both belong to the same family of models: diffusion models. They are remarkable at footage. But a lot of what people actually need is much simpler: an explainer, a product walkthrough, a logo that bounces in, a diagram that builds itself. For that kind of video, diffusion is the wrong shape of tool. This post explains why, and how we approach the problem at Vexo.

Diffusion in one paragraph

A diffusion model is trained by taking real data, adding noise to it step by step until nothing is left but static, and teaching a neural network to undo one step of that noise. To generate something new, you start from pure noise and run the network backwards many times. Each pass removes a little noise and adds a little structure, guided by your prompt, until an image appears. The model never draws a circle or writes a letter; it nudges millions of numbers until the result looks like a circle or a letter.

From images to video

Video adds time, and time is expensive. A few seconds of 1080p footage is hundreds of millions of pixel values, far too many to denoise directly. So video models share a few tricks:

Latent space. An autoencoder first compresses the video into a much smaller “latent” representation. Diffusion runs there, and a decoder turns the result back into pixels at the end.
Spacetime patches. OpenAI has described Sora as a diffusion transformer that cuts the latent video into small patches spanning both space and time, then treats them like tokens in a language model. Attention between patches is what keeps a character looking the same from one second to the next.
Text conditioning. Your prompt is encoded, often rewritten into a richer description first, and every denoising step is steered towards it.
Upsampling and audio. Results are typically generated small and refined upwards. Newer models such as Veo 3 and Sora 2 also generate a matching soundtrack alongside the frames.

The output is a grid of pixels per frame. That is the key fact for everything that follows.

Where it breaks for simple videos

For a cinematic shot of waves at sunset, pixels are exactly what you want. For a 20-second animated explainer, the same design choices turn into problems:

You can’t edit it, only re-roll it. Want the arrow a bit to the left, the title in your brand colour, or the second step to start half a second later? There is no arrow, title or step in the output, only pixels. You write a new prompt and hope the rest stays the same.
Text and exact shapes are fragile. Letters, numbers, logos and clean geometry are the hardest thing for a denoiser to hold steady across frames. They shimmer, warp or quietly change.
Timing is approximate. “Fade in the chart at 3 seconds” is a hint, not a contract. There is no timeline to adjust afterwards.
Clips are short and costly. Every second is generated on large GPUs, so length, resolution and iteration are rationed.
Output is heavy and fixed-size. You get an MP4 at one resolution. It can’t be restyled, translated or embedded on a web page as a light, crisp, responsive asset.

The AI-first approach: generate the scene, not the pixels

Most simple videos aren’t footage at all. They are motion graphics: shapes, text and images moving on a timeline. That structure is exactly what a large language model is good at producing. So in Vexo, the AI doesn’t paint frames; it builds the scene.

When you describe what you want, the assistant works through the same tools a person would use in the editor. It draws vector shapes, groups them into objects, places them on timelines, and records clips: “the train moves from here to there between 2s and 6s, easing out”. It can take a screenshot of any moment to check its own work and fix what looks wrong. The result is a real project, not a rendering of one.

Everything stays editable. Move a shape, change a colour, drag a clip on the timeline. No re-rolling.
Exact by construction. Text is text and a circle is a circle, in every frame.
Precise timing. Clips, tracks and easing live on a timeline you can see and adjust.
Light and sharp. Export animated SVG that stays crisp at any size and weighs kilobytes, or render MP4 when you need a video file.
Cheap to iterate. Language models are called to plan and edit the scene, not to compute every pixel of every frame.
Works with your AI. The same editor tools are available over MCP, so assistants such as Claude can build and edit Vexo scenes too.

The train journey on our home page was made this way: a scene of vector shapes and timelines that you could open and change, rather than a video you would have to generate again.

Which one should you use?

Use a diffusion model when you need something that looks filmed: people, places, physics, lighting. Use an AI-first animation tool when you need something you’ll keep working on: explainers, UI and product animations, diagrams, social posts, logo reveals, anything with text or a brand to respect. The two will meet in the middle, with generated imagery placed inside editable scenes, and that is where we are heading.

Vexo is coming soon. If you want early access or just want to compare notes on AI and animation, get in touch.