Building an AI Judge with CrewAI and LLM
We explore how to create an AI judge by combining knowledge graphs, reasoning systems, and various agent frameworks such as CrewAI, LangChain, and AutoGen, which enable the implementation of LLM-based multi-agent technology.
14 learners are taking this course
Level Intermediate
Course period Unlimited
AI
AI
LLM
LLM
AI Agent
AI Agent
AI
AI
LLM
LLM
AI Agent
AI Agent
Notice of Revised Video Posting
# Lecture 06. Creating an AI Judge with CrewAI and LLM
## Notice Regarding New Lesson Postings
Hello, students! A total of **252** new lessons (across 34 sections) will be posted sequentially for this course. Since I am producing multiple courses simultaneously, I am proceeding by posting **a little bit of each course alternately**, rather than uploading one entire course at once. Please find the posting principles and the schedule for each section below.
> For your reference, the lessons being posted now are not entirely new content, but part of the process of creating **revised editions (2nd or 3rd editions)** that reinforce existing lectures.
## Posting Principles
- Currently, I am rotating through a total of 19 courses in order (circular method), posting one lesson per course alternately.
- Postings are made on weekdays (Mon-Fri), with a total of 5 new lessons uploaded across all courses per day.
- For this course, one next lesson will be posted each time its turn in the rotation comes around.
- While the posting speed may feel slow compared to other courses, all courses are progressing together in the same manner.
## Posting Schedule by Section (Total 34 Sections)
| Section | Number of Lessons | Start Date | End Date |
|---|---|---|---|
| Section 4. [Basic] Reasoning Systems and Logical Judgment | 8 | 2026-07-02 | 2026-08-07 |
| Section 5. [Basic] AI Judge System Design and Implementation Practice | 3 | 2026-08-13 | 2026-08-25 |
| Section 6. [Basic] AI Judge System Design Lab | 8 | 2026-08-28 | 2026-10-06 |
| Section 7. [Basic] AI Judge System Implementation Lab | 9 | 2026-10-12 | 2026-11-23 |
| Section 8. [Advanced] Professional-level Requirements Analysis and User-Centered Design | 6 | 2026-11-27 | 2026-12-24 |
| Section 9. [Advanced] Professional-level System Design using UML | 7 | 2026-12-30 | 2027-01-27 |
| Section 10. [Advanced] Applying Design Patterns and Architectural Patterns | 6 | 2027-02-01 | 2027-02-23 |
| Section 11. [Advanced] Advanced Agent Implementation Techniques | 11 | 2027-02-26 | 2027-04-07 |
| Section 12. [Advanced] Advanced Techniques for Legal Knowledge and Data Management | 10 | 2027-04-12 | 2027-05-17 |
| Section 13. [Advanced] Professional-level Quality Assurance and Testing | 7 | 2027-05-20 | 2027-06-11 |
| Section 14. [Advanced] Performance Optimization and Scalability | 5 | 2027-06-16 | 2027-06-30 |
| Section 15. [Advanced] Ethical AI and Bias Management | 9 | 2027-07-05 | 2027-08-02 |
| Section 16. [Advanced] Security and Privacy Protection | 6 | 2027-08-04 | 2027-08-18 |
| Section 17. [Advanced] Professional-level Deployment and Operations | 7 | 2027-08-20 | 2027-09-07 |
| Section 18. [Advanced] User Interface and Experience Design | 6 | 2027-09-09 | 2027-09-23 |
| Section 19. [Advanced] Project Management Strategies | 13 | 2027-09-27 | 2027-10-29 |
| Section 20. [Advanced] System Maintenance and Continuous Improvement | 5 | 2027-11-02 | 2027-11-12 |
| Section 21. [Advanced] Commercialization and Business Application | 6 | 2027-11-16 | 2027-11-30 |
| Section 22. [Advanced] Future Directions and Cutting-edge Research Topics | 10 | 2027-12-02 | 2027-12-24 |
| Section 23. [Advanced] Convergence of Cognitive Science and Legal AI | 7 | 2027-12-28 | 2028-01-11 |
| Section 24. [Advanced] Advanced Legal Language Processing and Discourse Analysis | 8 | 2028-01-12 | 2028-01-27 |
| Section 25. [Advanced] Large-scale Legal AI Systems | 6 | 2028-01-31 | 2028-02-10 |
| Section 26. [Expert] Advanced Reasoning and Learning Strategies | 10 | 2028-02-14 | 2028-03-03 |
| Section 27. [Expert] Global Legal Systems and Regulatory Compliance | 9 | 2028-03-07 | 2028-03-22 |
| Section 28. [Expert] Large-scale Legal AI Operations and Safety | 10 | 2028-03-24 | 2028-04-11 |
| Section 29. [Expert] Future Technologies and Innovation Research | 8 | 2028-04-13 | 2028-04-26 |
| Section 30. [Expert] Legal AI Agent Design from a Harness Engineering Perspective | 8 | 2028-04-28 | 2028-05-12 |
| Section 31. [Expert] Harness-Reinforced Implementation of CrewAI Legal Agents | 7 | 2028-05-15 | 2028-05-25 |
| Section 32. [Capstone 1] Project Setup and Domain Narrowing | 6 | 2028-05-29 | 2028-06-07 |
| Section 33. [Capstone 2] Building Data Pipelines | 6 | 2028-06-08 | 2028-06-19 |
| Section 34. [Capstone 3] Agent and RAG Integration | 7 | 2028-06-20 | 2028-06-28 |
| Section 35. [Capstone 4] Harness HITL Evaluation Automation | 6 | 2028-06-29 | 2028-07-06 |
| Section 36. [Capstone 5] Web UI and Container Deployment | 6 | 2028-07-07 | 2028-07-14 |
| Section 37. [Capstone 6] Retrospective and Limitation Analysis | 6 | 2028-07-17 | 2028-07-24 |
## Estimated Completion
Based on the current pace, the entire course is expected to be fully posted around **July 2028**. (The actual schedule may be moved forward or delayed depending on the progress of production.)
Comment




