Course Overview

Welcome to CS 378, Systems for Generative AI! This course studies modern generative AI through a systems lens. Rather than treating models as isolated artifacts, we will look at the end-to-end systems that train, serve, monitor, and adapt them: accelerators, execution stacks, distributed training infrastructure, serving runtimes, retrieval pipelines, and agentic workflows. The course emphasizes how systems choices affect latency, throughput, cost, reliability, resource efficiency, and the quality of user-facing AI applications.

In this course, you will:

  • Learn how modern AI workloads stress compute, memory, storage, communication, and runtime systems
  • Understand the design choices behind training, serving, retrieval, and workflow systems
  • Reason about system metrics such as latency, throughput, utilization, cost, tail behavior, and reliability
  • Build practical intuition through hands-on programming assignments

Classes combine lectures, systems-oriented discussion, and programming projects. Whether you are interested in generative AI infrastructure, large-scale systems, or building efficient AI applications, this course will give you tools for understanding how model behavior and systems behavior interact.

Topics Covered

Throughout the semester, we will cover the following key topics:

  • Models versus systems, the AI lifecycle, and system quality
  • AI workload anatomy, including autoregressive generation, retrieval-augmented generation, and agentic workflows
  • GPU and accelerator architecture for generative AI workloads
  • Kernels, fusion, compilation, runtimes, and execution efficiency
  • Distributed training, data parallelism, model parallelism, communication, and memory efficiency
  • Serving systems, batching, scheduling, KV-cache management, and robustness
  • Post-training systems and RL-style infrastructure
  • Retrieval pipelines, vector search, and data-system tradeoffs
  • Agent and workflow systems, including tool use, state, reliability, and observability
  • Runtime protocols, control planes, operating AI systems, evaluation, adaptive systems, and future directions

Preferred Prerequisites

  • CS 439 (Principles of Computer Systems) or equivalent
  • CS 343 (Artificial Intelligence), or equivalent machine learning/AI background

Course Information

  • Course Number: CS 378
  • Semester: Fall 2026
  • Unique Number: 55595
  • Time: Tuesday & Thursday, 11:00 a.m. - 12:30 p.m.
  • Location: GDC 5.302
  • Discussion: Ed
  • Course Materials: Canvas
  • Course Staff Email: 378-sysml-f26@utlists.utexas.edu

Course Staff