Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey

Explore AI courses and programs from Stanford Online: https://stanford.io/4BYTekh

Once AI can take actions on its own, how do you stay in control? In this Stanford CS193T: Thinking with AI lecture, Senior Lecturer Cynthia Bailey previews this week's tools, Claude Code and Codex, with a three-step process for preparing, generating, and reviewing vibe-coded work. Then she looks past launch: why even a carefully built AI tool needs monitoring, how the NIST AI Risk Management Framework structures responsible AI across a product's life cycle, and what Stanford's Responsible AI guidance asks of you. The lecture closes with evaluation: what you are measuring, and why ground truth is harder to pin down than it looks.

Key questions answered in this lecture:
Q: What is vibe coding?
A: Bailey describes vibe coding as using English as a programming language with AI tools like Claude Code and Codex. Done professionally (some call it agentic engineering) it follows three steps: prepare a detailed spec with the elephant and goldfish process, hand it to a fresh coding agent, then review the result.

Q: How can I review vibe-coded work if I can't read code?
A: Bailey recommends starting another fresh 'goldfish' agent in a coding environment, giving it your scope of work and the code, and asking it to act as a severe code reviewer. Frame the code as someone else's work that may not follow the spec, to push the AI away from praise.

Q: Why do AI systems need monitoring after deployment?
A: Bailey says conditions change after deployment: distribution shift, users changing behavior once they know AI is involved (like students adding prompt-injection notes for an AI autograder), and models that update their weights. Thorough pre-deployment testing on historical data cannot catch these.

Q: What is survivorship bias in evaluating an AI model?
A: In Bailey's example, an AI predicts which employees to fire after 90 days; the retained ones do well, but there is no data on how the fired ones would have performed. She suggests firing only a random 75% of those flagged so the other 25% provide a counterfactual.

Q: What is the NIST AI Risk Management Framework?
A: Bailey describes the NIST AI RMF, released in 2023, as guidance rather than a precise standard, built on map, measure, and manage with a culture-focused governance function at the center. She stresses attending to the whole AI lifecycle, including post-deployment and decommissioning data.

Q: What is LLM as a judge in AI evaluation?
A: Bailey explains that judging hallucinations, tone, or bias at scale is subjective, so a second LLM grades each response from the first. She notes the goldfish code review in vibe coding is an example of this pattern.

Chapters:
0:00 Intro: vibe coding and controlling your AI
0:55 What is vibe coding?
3:35 Vibe coding AI out of a job
7:09 Elephant and goldfish process for vibe coding
16:32 Why post-deployment monitoring matters
18:33 Distribution shift and an AI autograder
24:50 Survivorship bias in AI evaluation
30:27 NIST AI Risk Management Framework
40:05 AI product lifecycle and decommissioning data
43:04 Stanford AI policies and data classification
46:39 The golden rule for AI and personal data
51:03 AI evals vs product metrics, LLM as judge
58:09 Ground truth and datasets of convenience
1:05:02 Case study: Twitter's image cropping bias

About the speaker:
Cynthia Bailey is a Senior Lecturer in the Computer Science Department at Stanford University. In 2023-2024, she served as an AI Policy Fellow in the United States Senate through the American Association for the Advancement of Science (AAAS). Her scholarship focuses on computer science education, AI/machine learning, and the social impacts of technology, including creating courses such as Equity and Governance for Artificial Intelligence and Race and Gender in Silicon Valley. Her teaching and research awards include the Lloyd W. Dinkelspiel Award for exceptional contributions to undergraduate education at Stanford, the Lasting Impact Award from the International Computing Education Research conference, a "Top 10 Papers of All Time" award at the 50th anniversary of the ACM SIGCSE Technical Symposium, and the Stanford Society of Women Engineers' Professor of the Year. She has consulted for Apple, NASA, law firms, and machine learning startups. She holds a Ph.D. in Computer Science from UC San Diego.

Course website: https://web.stanford.edu/class/archive/cs/cs193t/cs193t.1272/
Watch the full CS193T playlist: https://www.youtube.com/playlist?list=PLDrJO3v_iAZY

Recorded October 27, 2026 in CS193T: Thinking with AI, a Stanford University course taught by Senior Lecturer Cynthia Bailey.

#VibeCoding #ResponsibleAI #StanfordOnline Receive SMS online on sms24.me

TubeReader video aggregator is a website that collects and organizes online videos from the YouTube source. Video aggregation is done for different purposes, and TubeReader take different approaches to achieve their purpose.

Our try to collect videos of high quality or interest for visitors to view; the collection may be made by editors or may be based on community votes.

Another method is to base the collection on those videos most viewed, either at the aggregator site or at various popular video hosting sites.

TubeReader site exists to allow users to collect their own sets of videos, for personal use as well as for browsing and viewing by others; TubeReader can develop online communities around video sharing.

Our site allow users to create a personalized video playlist, for personal use as well as for browsing and viewing by others.

@YouTubeReaderBot allows you to subscribe to Youtube channels.

By using @YouTubeReaderBot Bot you agree with YouTube Terms of Service.

Use the @YouTubeReaderBot telegram bot to be the first to be notified when new videos are released on your favorite channels.

Look for new videos or channels and share them with your friends.

You can start using our bot from this video, subscribe now to Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey

What is YouTube?

YouTube is a free video sharing website that makes it easy to watch online videos. You can even create and upload your own videos to share with others. Originally created in 2005, YouTube is now one of the most popular sites on the Web, with visitors watching around 6 billion hours of video every month.