Explore Opportunities in This AI+X Domain
Indicate your interest to receive relevant AI+X pathway updates and future opportunities in this field.
PBL
ElevenLabs Project
AI in Speech and Audio
Build the core systems behind AI that listens and talks: speech recognition, speech synthesis, and neural audio compression, moving from signal-processing foundations and classical sequence models to the transformer-based approaches used today.
Project
ElevenLabs Project
Location
Online
Duration
8 Weeks
Upcoming Sessions
Feb. 01 - Apr. 02, 2027
Outcomes
Apply transformer architectures to speech and audio tasks
Synthesize speech from text with Tacotron and vocoders
Train GAN-based models for audio generation and codecs
Compress audio into discrete tokens with neural codecs
Evaluate speech systems with WER and quality benchmarks
You Will Get
Industry Guidance
Work directly with our project leads—experts and top researchers—who bring their real-world insights and expertise straight to your learning experience.
Research Experience
Collaborate with teammates and the project lead in a multi-week project to pursue novel questions in your research field.
Peer Networks
Engage with our PBL participants from all over the world. Collaborate with new peers and learn about their own research endeavours.
.png)
A Strong Portfolio
Put your best foot forward in the PBL with a standout project and receive a PBL Evaluation Report that can be used as a recommendation letter for employers and grad schools.
Expert Guidance
Get personalized feedback to grow your research and innovation skills.
Deliverables
Real projects, lasting connections, and new opportunities beyond your program.
Project Deliverables
The final presentation of your 8 weeks could be a poster, written report, or a slide deck, all of which can be expanded on.
Research Extension
Utilize up to 5 additional meeting times with the project lead after the project’s conclusion to build your work out for publication or conference presentation.
Industry Network
Meet peers in your projects and participate in a global talent community both online and in-person.
Industry Application
ElevenLabs builds AI voice technology that turns text into natural-sounding speech, transcribes spoken audio into text, and powers voice agents that hold real-time conversations. These products depend on the same building blocks this PBL covers: speech recognition, speech synthesis, and compact audio representations that let voice models run quickly. Students will gain hands-on experience with the core technical problems behind ElevenLabs' products.
Popular Industry Positions
Speech Recognition Engineer
Build ASR systems that transcribe speech across accents, languages, and noisy conditions.
Speech Synthesis / Voice AI Engineer
Develop text-to-speech and voice generation models for assistants, media, and accessibility.
AI Research Engineer (Audio & Speech)
Research audio representation, neural codecs, and speech language models.
Tracks
Track 1
Converting Speech to Text with Neural Models
Build a speech recognizer that turns recorded speech into text, and measure where it breaks on accents, noise, and unfamiliar languages.
Represent audio as waveforms and spectrograms with Torchaudio and Librosa
Train an RNN-based recognizer with CTC loss on LibriSpeech
Fine-tune Wav2Vec 2.0 on a custom dataset with Hugging Face
Measure word error rate across accents, noise conditions, or languages
Test whether data augmentation or fine-tuning improves robustness
Track 2
Generating Natural Speech from Text
Build a text-to-speech system that turns written text into audible speech, and measure how natural and intelligible it sounds.
Convert text into acoustic representations such as mel spectrograms
Train a Tacotron acoustic model in PyTorch on LJSpeech, VCTK, or LibriTTS
Train a GAN-based vocoder that turns spectrograms into waveforms
Evaluate the naturalness and intelligibility of the generated speech
Investigate expressive synthesis, speaker adaptation, or zero-shot voice generation
Track 3
Compressing and Reconstructing Speech with Neural Networks
Build a neural audio codec that compresses speech into compact discrete tokens and reconstructs it, then measure what quality is lost.
Build autoencoders and vector-quantized models for audio in PyTorch
Implement an EnCodec-style encoder, decoder, and quantizer
Set up adversarial training with GAN discriminators to improve reconstruction
Measure reconstruction quality against compression rate on LibriSpeech and LibriTTS
Benchmark existing codec models to compare their strengths and limits
PBL Journey
Online PBL Projects meet once a week for 8 weeks, and follow the research project format. Participants will meet the project lead, learn the conventions of the field and familiarize themselves with the tracks, then spend the middle portion of their time collaborating to develop their research.
At the end, participants will present their final project and receive feedback, with the opportunity to extend their timeline and develop the project in greater depth.
.png)
Project Team
Our Academic Team plays a vital role in your PBL journey at Blended Learning. We are dedicated to enhancing your learning experience and ensuring your academic success. Our team consists of three distinct roles, each with a specific focus to support your Research Guidance, Project Progress, and Personal Growth.
Project Lead
Providing Industry and Research Guidance
PhD Researcher in Electrical and Computer Engineering at the University of Illinois Urbana-Champaign
He focuses on automatic speech recognition and text-to-speech systems for low-resource applications. He is also a researcher affiliated with the National Center for Supercomputing Applications (NCSA), where he guides 12 students each summer through the NCSA REU program on projects that apply AI to biology, the social sciences, geospatial analysis, and medicine. As a teaching assistant for the university's major AI coursework, he supports nearly 400 students each semester. He also runs a YouTube channel on technical implementations of current AI topics and detailed mathematical derivations. He holds a master's in Statistics from UIUC and an undergraduate degree in Neuroscience and Statistics from Emory University.
Academic Advisor
Tracking Your Project Development
The Academic Advisor is dedicated to your project completion success. They manage the progress of your PBL, guiding team formation, facilitating group discussions, and resolving conflicts. Additionally, the Academic Advisor ensures team member contributions are on track and provides logistical support, including attendance tracking, hosting recitation sessions, managing research support requests, and conducting student evaluations at the end of the PBL.
From Our Students
"After a night spent debugging, I suddenly discovered the program running perfectly. In that triumphant moment, you realize your true capability and success. The exhaustion fades, replaced by the thrill of knowing your skills and persistence led to this achievement, reaffirming your potential."
Nicole Y.
National University of Singapore
B.S. Economics
FAQs
What is the learning format of a PBL?
All PBLs are offered in an 8-week online format that begins with an orientation followed by subject setup overview of the different tracks. The majority of the session time is dedicated to project development, with a final presentation at the culmination of the 8 weeks. Many PBLs are also offered bi-annually in an on-campus format that consists of daily in-person meetings.
How long does each PBL cohort last？
One round of the Online PBL cohort lasts 8 weeks, preceded bys a pre-PBL orientation week. Each On-Campus PBL usually has 8 in person meetings, with intensive classroom education and collaboration. This means the biggest difference between online and on-campus PBLs is time participants have in between meetings.
How can I be more academically prepared before the PBL starts?
Review the Blended Learning Insights sent by the Academic Advisor and familiarize yourself with the project topic and pre-learning materials. Ensure you have all necessary softwares and other resources needed for the PBL.
For each PBL cohort, will I work in teams? Are PBL team members self-selected or assigned?
Yes, you will work in teams for each round of the PBL Cohort. Each team has 3 to 6 participants, organized by the Academic Team. The Academic Advisor will organize groupings based on students' backgrounds, preferred track, and skills.
Can I work with the Project Lead on my project after the PBL ends?
Yes, with your AI + X Research Plan, you may request up to five PBL Research Extension meetings, where you work with the project lead to develop your project into a working manuscript. To schedule a PBL Research Extension meeting, talk to your Academic Advisor at the conclusion of your PBL.
What do I receive at the end of the PBL?
At the conclusion of the PBL cohort, you can request a PBL Evaluation Report which summarizes the PBL content, the hours you spent, the track you chose, and includes a recommendation letter from the Project Lead (for eligible participants who completed the project successfully).
Is attendance mandatory for PBL Live Sessions and Recitation Sessions?
Yes, attendance is mandatory for both PBL Live Sessions and Recitation Sessions. Participants with three or more unexcused absences forfeit their eligibility for a PBL Evaluation Report.
Do I need to have my camera on during online PBL Live Sessions?
Yes, you must have your camera on during online PBL Live Sessions. Participants with cameras off will be marked as absent. This is meant to encourage active engagement and participation in meetings.
Explore PBLs
ElevenLabs Project
AI in Speech and Audio
Build the core systems behind AI that listens and talks: speech recognition, speech synthesis, and neural audio compression, moving from signal-processing foundations and classical sequence models to the transformer-based approaches used today.
