강의

챌린지

멘토링

로드맵

커뮤니티

Internal Data Chatbot in Practice — Safely Integrating RAG and Text-to-SQL with Production Databases

A demo takes a day; production is another matter. We’ll build a Text-to-SQL chatbot with local Claude Code, measure its accuracy against 50 questions with known answers, and show with real-world measurements how to protect a production database using guardrails, read-only accounts, privacy protections, and permissions.

1 learners are taking this course

Level Intermediate

Course period Unlimited

MySQL
MySQL
RAG
RAG
claude
claude
Python
Python
LLM
LLM
MySQL
MySQL
RAG
RAG
claude
claude
Python
Python
LLM
LLM

What you will gain after the course

  • Improve Text-to-SQL accuracy with schema descriptions and a business terminology glossary, then quantify the impact using a gold-standard set.

  • Create evaluation sets and regression tests that are graded based on execution results rather than SQL statements.

  • Create an operational safety guard with tests that checks SELECT-only statements, dangerous functions, EXPLAIN cost, LIMIT, and execution time.

  • Blocks access to personal information columns (including SELECT *, aliases, and WHERE clauses) and table permissions by role, and records audit logs.

  • Put it into production with follow-up questions, SQL shown in answers, retries, and model comparisons, and measure cost and response time.

A demo takes a day; production is another matter. We show, with real-world measurements, how to build a Text-to-SQL chatbot with local Claude Code, measure its accuracy using 50 questions with known answers, and protect the production DB with guardrails, read-only accounts, privacy protections, and access controls.

A Demo Takes a Day; Production Is Another Story

Put the schema in the prompt and execute SQL generated by an LLM, and you can have a demo in a day. The problem is when you connect it to the production DB.

It confidently gives incorrect numbers, reads the entire production DB, and displays personal information verbatim. This course turns that demo into a “version that won’t cause an incident even when connected to production.”

We measure every number firsthand (50 questions answered correctly, measured with Claude Sonnet)

  • Demo with only column names: 20 correct out of 50 questions

  • With schema descriptions (value lists, what one row represents, and join paths): 42 correct answers.

  • With a business glossary (definitions of revenue, active members, and repurchase rate): 48 correct (50 when including correct values)

  • Improved version after collecting failures and fixing the dictionary: 50

Accuracy was determined not by the model, but by the materials we provided. We also show actual outputs illustrating how failures happen: inventing nonexistent values, omitting payment status, misunderstanding definitions, and getting the answer format wrong.

What the Guard Does and Doesn’t Block

  • We block it in three layers: allow only a single SELECT statement, prohibit locking reads and dangerous functions, block expensive queries before execution with EXPLAIN, impose LIMIT and execution-time limits, and use a read-only account. We prove the guard works with tests.

  • But incorrect numbers pass straight through the guard. Conversely, if the limits are set too tightly, they block as many as 13 correct SQL queries.

  • Returning a rejection reason and asking for a rewrite brings the number of correct answers back from 33 to 48, but Claude added FORCE INDEX to meet the guard’s criteria, and 6 of the 7 queries whose execution times were measured actually got slower. These results were verified directly by a DBA.

Privacy, Permissions, and Operations

  • Finds personal information columns in metadata (COMMENT) and blocks leakage through SELECT *, aliases, and WHERE clauses. Creates role-based table permissions and audit logs (without recording values).

  • For ambiguous questions, ask for clarification (measured: asked for clarification on 7/7 ambiguous questions and 0/4 clear questions), and show “Calculated with this SQL” alongside the answer.

  • Embed it in web chat, measure costs and response times, and compare models (Sonnet and Haiku) against the same set of answers to make a decision. It also covers how to monitor the chatbot account on the database side.

Recommended for these people

  • Backend and data developers who have been assigned to “build a chatbot people can ask questions about our data” at work.

  • DBAs and platform engineers who need to determine whether SQL generated by an LLM is safe to run against a production database.

  • Those who have tried document RAG and now want to build something that works with a relational database.

Prerequisites and Setup Materials

Being able to read SELECT and JOIN statements is enough. Execution plans will be explained in the lecture as needed. You’ll need Docker, Python 3.11 or later, and Claude Code installed locally (and logged in).

Claude calls are billed to your account at the applicable rates. The practice materials include cached Claude answers from the course, so asking the same questions as in the course produces the same results at no cost. Running a fresh round of all 50 questions cost about $0.4–$0.9.

Materials Provided

Hands-on database (virtual shopping mall “Moa Shop,” with 100,000 members and 400,000 orders, generated with a single Docker command) · safety guard · pipeline · grader · 50 questions with answers and an evaluation runner · schema documentation and glossary · web chat · tests (normal, edge-case, and failure) · original measurement records for the numbers shown on the lecture screen · reference answers for the Lesson 33 assignment

Notice

  • All data is custom-made fictional data and is unrelated to any real companies or customers.

  • The chatbot SQL in the free course (five scenarios) is an example of a common pattern written by the instructor. The Claude output in the main course is from actual calls, and we show results that differ from the example as-is (the join duplication error did not occur in the actual results).

  • LLM answers may vary slightly each time, and the estimated row counts in EXPLAIN vary each time the practice database is created.

  • AI helped with the script, code, and slide drafts, and the narration uses an AI-generated voice.

  • 40 videos · approximately 94 minutes. Each lesson is around 2–3 minutes and focuses on the essentials; follow along with the hands-on exercises using the provided materials.

Recommended for
these people

Who is this course right for?

  • A backend and data developer tasked by their company with building a 'chatbot you can ask questions about our data'

  • DBAs and platform engineers who need to decide whether SQL generated by an LLM is safe to run against a production database.

Need to know before starting?

  • Readable SQL: SELECT·JOIN

Hello
This is dorian05202988

Career Verified

A data craftsman with over 20 years of experience in databases and IT. As a DBA and manager of a DB Platform team, I work with database platforms and specialize in MySQL, PostgreSQL, Oracle, MongoDB, SQL tuning, AWS, Terraform, and Python. Drawing on experience in the gaming, finance, and IT services industries, I explain database and cloud architecture. I have published 『Oracle Practical Tuning: Learning SQL Patterns』 and 『Oracle Pro*C Practical Project Guide』, and have presented on serverless architectures, database selection, and multi-database strategies at AWS Summit 2018 and 2019 and DataYa Nolja 2020. Recently, I have been working on slow-query analysis and tuning recommendations using Bedrock LLM, automated DB specification sizing, and query workflows through a DB metadata MCP server and Slack bot. I explain database automation for developers and AI applications for non-developers through hands-on exercises tailored to the audience and objectives of each course. Rather than applying AI suggestions as-is, I teach a decision-making framework that verifies result equivalence, execution plans, and observability conditions, and defers action when the supporting evidence is insufficient. Company operational data and credentials are not included in the course materials.

More

Curriculum

All

40 lectures ∙ (1hr 33min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

Not enough reviews.
Please write a valuable review that helps everyone!

dorian05202988's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!

Limited time deal

$61,600.00

20%

$59.40