[Enterprise Challenge] Mastering the Core of Arm Architecture (Armv8-A) Through Debugging

Course Objectives

The goal of this lecture is to understand 100% of the core features of the Armv8-A architecture and to start practical work based on this understanding:

• Understand the core features of the Armv8-A architecture used in actual industry practice based on the execution flow.

• You can trace and analyze code at the assembly instruction level using the TRACE32 debugger.

• You can clearly understand the execution flow when an Exception or Interrupt occurs and explain the principles of operation.

• You can analyze Linux kernel behavior from the perspective of registers and assembly.

• You can perform primary analysis to trace the causes of bring-up issues and kernel crashes.

Target Audience

Linux system software developers (entry-level and junior developers) such as the following:

• Engineers developing Linux drivers on Armv8-A based systems

• Developers who need to analyze and resolve kernel crashes, booting issues, and fault problems

• System engineers who need to understand the flow between firmware, kernel, and drivers

• Working developers who want to understand the Armv8-A architecture from an execution perspective rather than just theory.

Key Differentiators of the Lecture

• Using TRACE32, we will conduct hands-on practice with key features of the Armv8-A architecture (assembly instructions, exceptions) that are essential in the field.

• We introduce the booting sequence of Arm-based chipsets and case studies from real-world projects.

• Practice the Armv8-A architecture (exceptions) while debugging actual memory dumps and kernel logs.

Lecture Curriculum

(Day 1)

Check TRACE32 practice environment (simulator mode)

• Basic Instructions, Practice Environment

Armv8-A Architecture Core Features and Execution Model

• Assembly instructions, registers, and exception models

• AAPCS, Hypervisor, TrustZone, MMU

Exception Level (EL0~EL3) structure and execution flow

• Boot Sequence and Exception Levels

• Software stack by Exception Level

• Exception Level Practice with TRACE32

Register Structure and Key System Registers

• Registers primarily used in the debugging process

• Analysis of system registers configured in startup code

• Analyzing key registers with TRACE32

Analysis of Assembly Instructions and Key Structures

• Analysis of Key Assembly Instructions

• Try reversing assembly instructions into C code

• Assembly instruction debugging practice with TRACE32

Calling convention analysis

• Analysis of Key Registers and Instructions

• Calling convention debugging practice with TRACE32

(Day 2)

Execution flow of exceptions and interrupts

• Analysis of key execution flows and related system registers

• Exception practice using memory dumps (TRACE32)

TrustZone

• Concepts of Secure World and Non-secure World

• Debugging methods for Secure OS (Exception, Watchdog Reset)

• Analysis of Key Stacks in Arm Trusted Firmware

Arm Virtualization (Hypervisor)

• Analysis of key Virtualization instructions and hypervisor stack

• Analysis of XEN Hypervisor Exception Handlers

• Analysis of major system registers with TRACE32

Virtual Memory System

• Analysis of multi-level page translation process (VA39, VA48)

• Major system registers (TCR_EL1, TTBR1_EL0)

• Virtual address translation debugging practice with TRACE32

Introduction to Armv9-A Extension Features

• Memory Tag Extensions (MTE )

• PAC, BTI feature

Course Objectives

The primary goal is to understand the key features of the Linux kernel used in practical development to quickly develop Linux device drivers (for chipset and product developers). Another major objective is to secure the capability to independently resolve various issues that arise in real-world practice:

• Understand the Linux kernel's booting flow and the initialization process of device drivers step by step.

• You can resolve various device driver issues by effectively utilizing the various debugging features provided by Linux (debugfs, ftrace).

• Become familiar with kernel APIs by learning the core concepts of the Linux kernel, such as processes, interrupts, kernel synchronization mechanisms, and virtual memory.

• You can analyze binary dumps to examine the kernel's major data structures.

• You can independently perform primary analysis on crashes and bring-up issues based on kernel logs and memory dumps.

Target Audience

Linux system software developers (new and junior developers) as follows:

• Engineers developing Linux kernels and device drivers on Arm-based SoCs

• Engineers who need to develop Linux devices (kernel builds, device tree analysis, bootloader optimization) within a short period of time

• Engineers who want to customize device drivers within a short period of time

• Developers who need to identify Linux kernel debugging features and analyze issues such as boot failures, kernel panics, kernel crashes, and watchdog resets.

• System engineers who need to understand the entire execution flow between the firmware bootloader and kernel drivers

Key Differentiators of the Lecture

• You will directly create patches for various functions (Device Tree, Kernel Config, Device Driver) required for debugging device driver development.

• In the practice sessions, we utilize the Crash utility and TRACE32, which are directly used in the industry (by big tech companies such as Qualcomm and NVIDIA).

• Analyze the major data structures of the Linux kernel (processes, interrupts, task scheduling, virtual memory systems) through memory dump practice.

• We introduce the booting sequence of Arm-based chipsets (or RISC-V) and case studies occurring in real-world projects.

Lecture Curriculum

(Day 1)

TRACE32 practice environment (simulator mode) check + board environment

• Basic commands, practice environment

Why you should learn the Linux kernel

• Efficiently developing device drivers

• Solving various project issues

• The foundation for writing stable code

Arm architecture (based on Armv8, AArch64) required to understand the Linux kernel

• Exception Level, Exception Vector Table, Registers (General Purpose)

• (If necessary) Explanation up to RISC-V: Privilege mode and trap handlers

Key Kernel Debug Features

• printk, debugfs, ftrace, kdump

• Memory dump and kernel log analysis methods

• How to write debugging patches

• How to use the Crash utility

Core features of device drivers

• Structure of sysfs and platform devices

• Analysis of device tree structure and related core kernel APIs

• Device tree debugging methods

• Explanation of the core structures of major device drivers

Process

• Analysis of basic process concepts and the process of creation and termination

• Analysis of struct task_struct and thread_info data structures

• Analysis of process and stack structures

• Analysis of key issues derived from the process creation and termination stages

• Analysis of key data structures through memory dump analysis

(Day 2)

IRQ and Bottom Half

• Interrupt vector (Armv8 perspective) handler analysis

• Introduction to interrupt context and in_interrupt()

• Analysis of the interrupt descriptor (struct irq_desc) data structure

• Analysis of key data structures through memory dump analysis

Bottom half

• Introduction to key bottom half mechanisms (threaded irq, workqueue, soft irq)

• Debugging methods for major interrupt-related issues (IRQ Storm, major device driver issues)

• Major bottom half issues in device drivers

Dynamic Timer

• Basic concepts of jiffies and HZ

• How to write a dynamic timer handler

• How to debug the flow of execution time

System Calls and Signals

• Basic concepts and execution flow of system calls

• Execution flow of system calls and device drivers

• Basic behavior of signals

• Key signals you need to know in real-world projects

• Analyzing major data structures through memory dump analysis

Task Scheduling

• Understanding the concepts of scheduling and context switching

• Identifying priority and process status

• Data structures: runqueue, scheduler classes

• Timing of preemption and preemption enable/disable

• sched_switch, sched_wake_up: ftrace analysis

• Introduction to major performance issues related to task scheduling

• Analysis of key data structures through memory dump analysis

(Day 3)

Synchronization

• Introduction to race conditions and critical sections

• spinlock: Queued spinlock, Ticket spinlock

• Introduction to the operating principles and key data structures of mutexes

• atomic operation

• Introduction to major race issues in real-world projects

• Analyzing key data structures through memory dump analysis

VFS(Virtual File System)

• Introduction to the overall structure and execution flow of VFS

• VFS-related data structures

• Introduction to core file systems - ext4, fat

• Key practical issues related to file systems

• Analysis of key data structures through memory dump analysis

Memory Management

• Explanation of the virtual memory map and the structure of the virtual memory system

• Analysis of struct zone and node

• Main Memory Allocator (SLUB allocator)

• kmalloc and gfp_flags

• Key memory issues and debug features

• Analysis of major data structures through memory dump analysis

Case Studies of Major Device Driver Issues

• Device node not registered

• Issues with interrupt registration and interrupt storm issues

• Firmware image loading failure issue

• i2C and spi transfer failure

Student Q&A

• Debugging, problem solving, and kernel code analysis methods

• Debugging methods for hardware-related issues

• Open source community

Course Objectives

The goal of this lecture is to understand 100% of the core features of the RISC-V architecture and to start practical work based on this understanding:

• Understand the core features of the RISC-V architecture used in practical applications.

• You can debug assembly instructions using the TRACE32 debugger.

• You can accurately understand and explain the execution flow when a Trap (Exception) or Interrupt occurs.

• Can analyze kernel behavior from the perspective of registers and assembly.

• Capable of performing primary analysis on bring-up and crash issues.

Target Audience

Linux system software developers (new and junior developers) as follows:

• Engineers developing Linux drivers on RISC-V based systems

• Engineers aiming to perform system bring-up (bootloader, Linux driver) on RISC-V based systems

• Developers who need to analyze and resolve kernel crashes, booting issues, and fault problems

• System engineers who need to understand the flow between firmware, kernel, and drivers

• Working developers who want to understand the RISC-V architecture from an execution perspective rather than just theory

Key Differentiators of the Lecture

• It explains the differences between the Arm architecture and RISC-V by comparing them.

• We will conduct hands-on practice using TRACE32 (simulator mode) to cover key features of the RISC-V architecture (assembly instructions, exceptions) that are essential in the industry.

• We introduce the booting sequence of RISC-V-based chipsets and case studies from real-world projects.

• We will practice the RISC-V architecture (exceptions) by debugging actual memory dumps and kernel logs.

Lecture Curriculum

(Day 1)

Checking the TRACE32 practice environment (simulator mode)

• Basic instructions, practice environment

Introduction to Core Features and Execution Model of RISC-V Architecture

• Assembly instructions, registers, exception model

• Opensbi, Calling convention, MMU

Privilege mode structure and execution flow

• Boot Sequence and Machine Mode

• Software stack by privilege mode

• Privilege mode practice with TRACE32

Register Structure and Key CSR Registers

• Registers primarily used during the debugging process

• Analysis of CSR registers configured in the startup code

• Analysis of major registers with TRACE32

Analysis of assembly instructions and key structures

• Analysis of key assembly instructions

• Practice reversing assembly instructions into C code

• Hands-on assembly instruction debugging with TRACE32

Calling convention analysis

• Analysis of Key Registers and Instructions

• Debugging practice of calling conventions with TRACE32

(Day 2)

Execution flow of exceptions and interrupts

• Analysis of main execution flows and related CSRs

• Exception practice using memory dumps (TRACE32)

Memory synchronization

• Key assembly instructions

• Introduction to Key Case Studies

Virtual Memory System

• Analysis of multi-level page translation processes (SA39, SA48)

• Key CSR Registers (satp)

• Debugging practice for virtual address translation using TRACE32

Memory Dump Analysis Practice

• Stack Corruption Debugging, Call Stack Restoration (Using TRACE32)

• Analysis of major crash signatures