Models & Comparisons

Elevenlabs (topic)

Last updated 2026-09-28

What's new

2026-09-28
  • OpenAI released cheaper GPT-6 models, and Anthropic launched Claude Opus 5.5, which is 40% cheaper and performs similarly to its more expensive Claude Fable 5.1 model.
  • Claude Opus 5.5 requires less computing power, making it more cost-effective for tasks like coding and data analysis, and it's better at communicating clearly, reducing jargon and improving readability.
  • Meta announced a new AI device small enough to carry on a keychain, and there are rumors of OpenAI releasing a new AI agent platform (a tool that automates tasks) next week.
2026-09-22
  • AI can create video clips, but humans are still needed to craft meaningful stories with relatable characters (people we can connect with emotionally).
  • To keep AI-generated videos consistent, use a character reference sheet (a set of images showing a character from different angles) and follow strict visual rules.
  • Build a believable world for your story by defining rules for location, lighting, colors, objects, and camera angles, and stick to them throughout the video.
  • Use AI tools like Claude (a type of AI assistant) to organize your story ideas and create a structured narrative with clear goals and conflicts.
2026-09-19
  • Buzz (a free, open-source app) now lets you create and edit videos using AI agents like GPT-6 Astra (a video editing AI) without needing traditional editing software.
  • ElevenLabs MCP (a creative toolkit) allows AI agents to generate and control voice, music, images, and videos, making it easier to create content.
  • You can use AI agents like Zeus (a task manager AI) to create YouTube Shorts and thumbnails, then have another agent like Astra (a quality assurance AI) check the final product.
  • Buzz is self-hosted, meaning your data stays private and you can back it up or move it anywhere you like.
2026-09-16
  • UA 2 (a free, open-source music generator) can create songs in various styles, languages, and even transform existing songs into different genres, like turning a Beethoven melody into a hard rock song.
  • This tool first creates editable sheet music, allowing you to easily change lyrics, keys, or instruments, and even generate cover songs.
  • UA 2 outperforms other open-source models and some paid ones (like Sunno V6) in song quality, according to benchmarks.
  • You can connect UA 2 to AI agents (like GBT Astra or GLM, which are AI assistants) to generate and edit music using simple text commands.
2026-09-10
  • AI tools like GPT-6 Astra (a smart AI assistant) can now control video editing software like Da Vinci Resolve (a paid video editing tool) through a feature called MCP (a direct connection between AI and the software), making editing faster and more automated.
  • The AI can handle tasks like organizing media, creating edits, and even transcribing audio, all without manual input, potentially speeding up the editing process significantly.
  • Da Vinci Resolve 21.1 (the latest version) has integrated this AI assistant feature, allowing users to use AI agents like ChatGPT to control the software, reducing the need for manual cursor control.
  • While the AI shows promise, it's not perfect—it can encounter issues like timeouts during transcription, but it's a step towards more efficient video editing workflows.
2026-09-07
  • Claude Fable 5.1 is a new AI model that can work on tasks independently, using multiple tools and programs, like a smart assistant (called an agent) to achieve goals you set for it.
  • It can create detailed 3D models and designs, like a fully furnished apartment, based on a simple floor plan, showing strong spatial understanding.
  • The model also demonstrated advanced physics and lighting understanding by creating a ray tracing simulation of shapes floating in an ocean with adjustable settings, all coded from scratch.
2026-08-31
  • OpenAI's CEO, Sam Altman, predicts the company will have an internal system it considers artificial general intelligence (AGI, AI that can perform any intellectual task a human can) by the end of 2026, with their upcoming Astra model already showing advanced capabilities in research, coding, and cybersecurity.
  • China is making significant strides in AI and robotics, with Unitree unveiling a humanoid robot capable of impressive physical feats, and Bite Dance reportedly training a massive AI model with 10 trillion parameters.
  • OpenAI is developing a portable, screenless AI device called Joanie IV, and Warmwind has launched autonomous cloud AI workers that can operate normal software visually.
  • AI agents are now auditing scientific literature and finding errors that humans had previously missed, demonstrating the potential for AI to enhance productivity and accuracy in various fields.
2026-08-28
  • OpenAI's new chip, Jalapeno (a special computer part for AI tasks), is up to 100 times more efficient than Nvidia's best chips, according to their tests.
  • Anthropic (an AI company) released two new AI models briefly, which were tested and then removed, and they might be rolling out a new model, Fable 5.1, quietly.
  • Alibaba (a tech company) created a new AI model that costs much less to train than its previous one.
  • Claude (an AI assistant) fixed a frustrating issue with agents (AI tools that do tasks for you), making it easier to work with them.
2026-08-19
  • "World models" (AI systems that create real-time interactive videos) could revolutionize how we make and watch content, making video as programmable as software.
  • Unlike current AI video tools (like VO3 or CDNS 2) that generate fixed videos, world models let you change and interact with videos as they happen.
  • Real-time tech has transformed industries before, like GPS enabling Uber or digital cameras fueling Instagram and TikTok.
  • Reactor (a company building tools for world models) aims to make this tech accessible, giving creators more control over AI-generated videos.
2026-08-16
  • DeepSeek V4 Pro, a new AI model, is highlighted for its impressive performance and affordability, outperforming other models like GPT 5.6 Luna in certain aspects.
  • The video demonstrates using DeepSeek V4 Pro with Open Code (a free, open-source coding assistant) to create a SaaS (software as a service) project plan for generating stylistic thumbnails using Gemini 3.1 flash image.
  • The creator uses Convex (a backend service), Clerk (user management), Next.js (a web development framework), Open Router (for AI integration), and Stripe (for payments) to build the project quickly.
  • The process involves setting up API keys and configuring services to streamline the development, showcasing the efficiency of these tools together.
2026-08-10
  • Companies like WiseDocs (a medical claims processor) face challenges with slow AI pipelines, complex codebases, and difficulty updating systems.
  • Technical debt (bad code that causes problems later) can slow down progress, but AI advancements are helping teams work faster.
  • AI tools, like orchestrators (software that manages AI tasks), are improving, but it's important to verify claims (like those from Google and OpenAI) to avoid "AI psychosis" (believing hype over reality).
  • Modern AI models, such as Sonnet 4.6 and Opus 4.8, are making coding tasks faster and more accurate, reducing manual work.
2026-08-07
  • Traditional cloud infrastructure isn't designed for personal AI code generation, where users can customize apps with AI agents (software that helps users create code) to add features just for them.
  • Current systems, like those from Apple and Google, make it hard for users to install non-approved software, while web apps, though customizable, run on outdated cloud models that don't support user customization.
  • The speaker, Kenton Varta, argues for a new infrastructure that allows users to modify their software, drawing a parallel to Richard Stallman's advocacy for software freedom.
  • Varta, the creator of Cloudflare Workers (a serverless application hosting platform), suggests that current systems delay feature implementation and create messy codebases, which AI could help avoid.
2026-07-31
  • Morgan Stanley has developed an "auto research agent" (a computer program that automates research tasks) to improve financial predictions using AI models like Opus 4.5 and Claude Code.
  • This agent can handle tasks from start to finish, like analyzing data, creating tests, and running experiments, all based on simple instructions in plain language.
  • The system is designed to work with various AI models and improve as those models get better, aiming to maximize profits and integrate smoothly with existing tools.
  • They've released a first version called Alpha Lab 1.0, with all code open-sourced on GitHub, and are now working on improvements based on initial results and failures.

Key points

What it is

  • ElevenLabs is an AI voice platform that creates natural-sounding voice agents (AI programs that can talk like humans) for customer conversations.
  • It shifts AI from text-only to voice-driven, making interactions feel more lifelike and practical for customer service, virtual assistants, or accessibility tools.
  • Unlike simple chatbots, ElevenLabs agents can handle phone calls, in-person kiosks, or voice-activated support.
  • It allows developers to create AI that talks to users in a human-like way, making interactions less robotic and more natural.

How to use it

  • Start by creating a new voice in ElevenLabs and provide a sample recording for voice cloning (copying a person's speaking style).
  • Define your problem (e.g., needing natural-sounding narration) and prepare your script using a tool like Claude (an AI assistant).
  • Follow the step-by-step process: create a new voice, upload a 30-second audio clip, paste your script, and generate the audio.
  • For multi-step workflows, generate audio in ElevenLabs first, then import it into a video tool like HeyGen (a video creation platform).

Watch out for

  • Audio quality may drop when exporting from ElevenLabs to other tools, so always test your exported audio in its intended environment.
  • Use a multi-stage workflow and have a second AI tool review the output to catch mistakes like off-sounding voices or branding inconsistencies.
  • Ensure high-quality input data for voice cloning, as poor quality leads to poor results (garbage in, garbage out).
  • ElevenLabs has strict monetization rules, and revenue sharing may not generate significant income.

Tools named

  • ElevenLabs (AI voice platform for creating natural-sounding voice agents), Claude (AI assistant for writing scripts), HeyGen (video creation platform)

Lesson 1: What is Elevenlabs (topic) and why it matters

ElevenLabs is an AI voice platform that lets you build real-time, natural-sounding voice agents—think of it as giving AI a human-like voice for customer conversations. Unlike a simple chat bot tucked in a website corner, ElevenLabs agents can talk to people naturally, using uploaded business knowledge to answer questions or handle tasks. For AI development, this matters because voice is becoming a key interface for practical AI agents (programs that autonomously complete tasks). The platform allows developers to create customer experience-focused agents that can actually speak, not just type. This shifts AI from a text-only tool to one that can handle phone calls, in-person kiosks, or voice-activated support. In a landscape where many companies claim "AI agent" but deliver clunky chat, ElevenLabs offers a more polished, voice-driven alternative. For beginners, this means you can design an AI that talks to users in a lifelike way, making interactions feel less robotic and more natural. It moves AI development beyond coding and into real-world communication, which is critical for applications like customer service, virtual assistants, or accessibility tools. Understanding ElevenLabs helps you see how voice can make AI more practical and human-centered, a growing focus in the field.

Sources

Lesson 2: How to use Elevenlabs (topic): step-by-step

To start using ElevenLabs, go to the platform and click voices, then create a new voice. You will provide a sample of the voice you want to clone—this is called voice cloning (copying a person's speaking style). ElevenLabs is mostly audio, so you feed it a short recording, and the model learns that voice's tone and inflection.

First, define your problem: you need natural-sounding narration, but standard text-to-speech sounds robotic. ElevenLabs solves this by generating audio that sounds phenomenal compared to raw outputs from other tools.

Next, prepare your script. Use a tool like Claude to write the script and structure it with beats (key points to cover). For example, ask Claude for an intro and for help filling gaps. Then copy that script into ElevenLabs.

Step by step: 1) Go to Voices and create a new voice. 2) Upload a 30-second audio clip of the target speaker. 3) Paste your script into the text box. 4) Click generate. The model will read the script in the cloned voice.

For multi-step workflows, some creators generate audio in ElevenLabs first, then import that audio into a video tool like HeyGen, because ElevenLabs sounds better than raw generation. You can also use ElevenLabs to build voice agents (automated phone or chat voices) by configuring a system prompt and first message inside the platform.

Example: If you need a video voiceover, write the script in Claude, create the voice in ElevenLabs, generate the audio, then combine with visuals. The audio output will match the cloned voice's natural rhythm.

Sources

Lesson 3: Best practices and pitfalls

ElevenLabs has a common problem: audio that sounds great inside the platform can sound poor when exported to other tools. One creator noted that a voice "in 11 Labs, it sounds phenomenal" but after moving it to another app, the quality dropped noticeably. To avoid this pitfall, always test your exported audio in its intended environment before finalizing a project.

A best practice is to use a multi-stage workflow. Generate your initial audio in ElevenLabs, then have a second AI tool review the output. As one guide explains, "make AI check its own work before you do" — feed that feedback back into the tool to "make an even better version." This catches mistakes like off-sounding voices or branding inconsistencies, which can often be "fixed up on a second pass."

Another key mistake is ignoring the quality of your input data. When training an LLM (large language model) from scratch, ElevenLabs emphasizes that you need "very high-quality label data" and a QA (quality assurance) process because "if your QA is not good, you get fired." The same principle applies to voice cloning: garbage in, garbage out.

Finally, remember that ElevenLabs has strict monetization rules. While they offer revenue sharing, one experienced user warns "I never made more than a few hundred bucks off it" despite having one of the most popular voices on the platform for years. Do not expect significant income from sharing your voice clones.

Sources
← Back to curriculum