Update the course for 2025, add week01 materials

Author: mryab

README.md

@@ -1,36 +1,21 @@
 # Efficient Deep Learning Systems
 This repository contains materials for the Efficient Deep Learning Systems course taught at the [Faculty of Computer Science](https://cs.hse.ru/en/) of [HSE University](https://www.hse.ru/en/) and [Yandex School of Data Analysis](https://academy.yandex.com/dataschool/).
 
-__This branch corresponds to the ongoing 2024 course. If you want to see full materials of past years, see the ["Past versions"](#past-versions) section.__
+__This branch corresponds to the ongoing 2025 course. If you want to see full materials of past years, see the ["Past versions"](#past-versions) section.__
 
 # Syllabus
 - [__Week 1:__](./week01_intro) __Introduction__
   - Lecture: Course overview and organizational details. Core concepts of the GPU architecture and CUDA API.
   - Seminar: CUDA operations in PyTorch. Introduction to benchmarking.
-- [__Week 2:__](./week02_management_and_testing) __Experiment tracking, model and data versioning, testing DL code in Python__
-  - Lecture: Experiment management basics and pipeline versioning. Configuring Python applications. Intro to regular and property-based testing.
-  - Seminar: Example DVC+Weights & Biases project walkthrough. Intro to testing with pytest.
-- [__Week 3:__](./week03_fast_pipelines) __Training optimizations, profiling DL code__
-  - Lecture: Mixed-precision training. Data storage and loading optimizations. Tools for profiling deep learning workloads. 
-  - Seminar: Automatic Mixed Precision in PyTorch. Dynamic padding for sequence data and JPEG decoding benchmarks. Basics of profiling with py-spy, PyTorch Profiler, PyTorch TensorBoard Profiler, nvprof and Nsight Systems.
-- [__Week 4:__](./week04_distributed) __Basics of distributed ML__
-  - Lecture: Introduction to distributed training. Process-based communication. Parameter Server architecture.
-  - Seminar: Multiprocessing basics. Parallel GloVe training.
-- [__Week 5:__](./week05_data_parallel) __Data-parallel training and All-Reduce__
-  - Lecture: Data-parallel training of neural networks. All-Reduce and its efficient implementations.
-  - Seminar: Introduction to PyTorch Distributed. Data-parallel training primitives.
-- [__Week 6:__](./week06_large_models) __Training large models__
-  - Lecture: Model parallelism, gradient checkpointing, offloading, sharding. 
-  - Seminar: Gradient checkpointing and tensor parallelism in practice.
-- [__Week 7:__](./week07_application_deployment) __Python web application deployment__
-  - Lecture/Seminar: Building and deployment of production-ready web services. App & web servers, Docker, Prometheus, API via HTTP and gRPC.
-- [__Week 8:__](./week08_inference_software) __LLM inference optimizations and software__
-  - Lecture: Inference speed metrics. KV caching, batch inference, continuous batching. FlashAttention with its modifications and PagedAttention. Overview of popular LLM serving frameworks.
-  - Seminar: Basics of the Triton language. Layer fusion in PyTorch and Triton. Implementation of KV caching, FlashAttention in practice.
-- [__Week 9:__](./week09_compression) __Efficient model inference__
-  - Lecture: Hardware utilization metrics for deep learning. Knowledge distillation, quantization, LLM.int8(), SmoothQuant, GPTQ. Efficient model architectures. Speculative decoding.
-  - Seminar: Measuring Memory Bandwidth Utilization in practice. Data-free quantization, GPTq, and SmoothQuant in PyTorch.
-- [__Week 10:__](./week10_invited) __MLOps, k8s, GitOps and other acronyms__ by [Gleb Vazhenin](https://github.com/punkerpunker), Bumble
+- __Week 2:__ __Experiment tracking, model and data versioning, testing DL code in Python__
+- __Week 3:__ __Training optimizations, profiling DL code__
+- __Week 4:__ __Data-parallel training and All-Reduce__
+- __Week 5:__ __Sharded data-parallel training, distributed training optimizations
+- __Week 6:__ __Training large models__
+- __Week 7:__ __Python web application deployment__
+- __Week 8:__ __LLM inference optimizations and software__
+- __Week 9:__ __Efficient model inference__
+- __Week 10:__ Guest lecture
 
 ## Grading
 There will be several home assignments (spread over multiple weeks) on the following topics:
@@ -44,11 +29,15 @@ Please refer to the course page of your institution for details.
 # Staff
 - [Max Ryabinin](https://github.com/mryab)
 - [Just Heuristic](https://github.com/justheuristic)
-- [Alexander Markovich](https://github.com/markovka17)
+- [Yaroslav Zolotarev](https://github.com/Q-c7)
+- [Gregory Leleytner](https://github.com/RunFMe)
+- [Antony Frolov](https://github.com/antony-frolov)
 - [Anton Chigin](https://github.com/achigin)
-- [Ruslan Khaidurov](https://github.com/newokaerinasai)
+- [Alexander Markovich](https://github.com/markovka17)
+- [Roman Gorb](https://github.com/rvg77)
 
 # Past versions
+- [2024](https://github.com/mryab/efficient-dl-systems/tree/2024)
 - [2023](https://github.com/mryab/efficient-dl-systems/tree/2023)
 - [2022](https://github.com/mryab/efficient-dl-systems/tree/2022)
 - [2021](https://github.com/yandexdataschool/dlatscale_draft)

week01_intro/README.md

@@ -1,12 +1,14 @@
 # Week 1: Introduction
 
 * Lecture: [link](./lecture.pdf)
-* Seminar + bonus home assignment: [link](./seminar.ipynb)
+* Seminar: [link](./seminar.ipynb)
 
 ## Further reading
 * [CUDA MODE reading group Resource Stream](https://github.com/cuda-mode/resource-stream)
 * [CUDA Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html) and [CUDA C++ Best Practices Guide](https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html)
+* [Modal GPU Glossary](https://modal.com/gpu-glossary)
 * [How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog](https://siboehm.com/articles/22/CUDA-MMM)
+* [GPU Puzzles](https://github.com/srush/GPU-Puzzles)
 * [PyTorch Performance Tuning Guide](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html)
 * [Earlier version of this guide from NVIDIA](https://tigress-web.princeton.edu/~jdh4/PyTorchPerformanceTuningGuide_GTC2021.pdf)
 * [Docs for caching memory allocation in PyTorch](https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management)