This course covers both the fundamentals and advanced topics in operating system (OS) security. Access control mechanisms (e.g., SACL/DACL), memory protections, and interprocess communications mechanisms will be studied. Students will learn the current state-of-the-art OS-level mechanisms and policies designed to help protect systems against sophisticated attacks. In addition, advanced persistent threats, including rootkits and malware, as well as various protection mechanisms designed to thwart these types of malicious activities, will be studied. Advanced kernel debugging techniques will be applied to understand the underlying protection mechanisms and analyze the malicious software. Students will learn both hardware and software mechanisms designed to protect the OS (e.g., NX/ASLR/SMEP/SMAP). The course will use virtual machines to study traditional OS environments on modern 64-bit systems (e.g., Windows, Linux, and macOS), as well as modern mobile operating systems (e.g., iOS and Android).
Familiarity with operating system concepts.