The Life of Binaries
Creator: Xeno Kovah @XenoKovah
License: Creative Commons: Attribution, Share-Alike
(http://creativecommons.org/licenses/by-sa/3.0/)
Class Prerequisites: Recommended, but not required, to have taken Introduction to x86 and Intermediate x86
Lab Requirements: Requires a Windows system with Visual C++ Express Edition, Windows DDK or WDK kernel module compilation environment, and WinDbg. Requires a Windows guest OS running in VMWare Player or VMWare Server in order to do kernel debugging with WinDbg from the host OS.
Class Textbook: None
Recommended Class Duration: 3 days (class previously taught in 2 days which was too little)
Creator Available to Teach In-Person Classes: Yes
Author Comments:
Topics include but are not limited to:
•Scanning and tokenizing source code.
•Parsing a grammar.
•Different targets for x86 assembly object files generation. (E.g. relocatable vs. position independent code).
•Linking object files together to create a well-formed binary.
•Detailed descriptions of the high level similarities and low level differences between the Windows PE and Linux ELF binary formats. (NOTE: we didn't get to this in the class where the video was recorded, but the materials are in the slides)
•How an OS loads a binary into memory and links it on the fly before executing it.
Along the way we discuss the relevance of security at different stages of a binary’s life, from the tricks that can be played by a malicious compiler, to how viruses really work, to the way which malware “packers” duplicate OS process execution functionality, to the benefit of a security-enhanced OS loader which implements address space layout randomization (ASLR).
Lab work includes:
•Using the new “Binary Scavenger Hunt” tool which creates randomized PE binaries and asks randomized questions about the material you just learned!
•Manipulating compiler options to change the type of assembly which is output
•Manipulating linker options to change the structure of binary formats
•Reading and understanding PE files with PEView
•Reading and understanding ELF files with Readelf (NOTE: we didn't get to this in the class where the video was recorded, but the materials are in the slides)
•Using WinDbg and/or GDB to watch the loader dynamically link an executable
•Using Thread Local Storage (TLS) to obfuscate control flow and serve as a basic anti-debug mechanism
•Creating a simple example virus for PE
•Analyze the changes made to the binary format when a file is packed with UPX
•Using the rootkit technique of Import Address Table (IAT) hooking to subvert the integrity of a program’s calls to external libraries, allowing files to be hidden.
Knowledge of this material is recommended, but not required, for future classes such as Rootkits, but is required for reverse engineering.
A student Q&A forum has been set up at http://www.reddit.com/r/OST_LifeOfBinaries.
To submit any suggestions, corrections, or explanations of things I didn’t know the reasons for, please email me at the address included in the slides.
This is why it’s good to have a guide ;)
Author Biography: Xeno co-founded LegbaCore in January 2015 to focus on improving firmware security. He is also the founder and lead contributor to OpenSecurityTraining.info. He has posted 9 full days of class material material on x86 assembly, architecture, binary formats (PE and ELF), and Windows rootkits. He has a Bachelors of Science in Computer Science from the University of Minnesota Twin Cities, and an Masters of Science in Information Security Technology and Management from Carnegie Mellon University. He attended CMU through the National Science Foundation Scholarship for Service (aka CyberCorps) program. His specialty area is stealth malware and its ability to hide from security software and force security software to lie. To combat such attacks he researches trusted computing systems that can provide much stronger security guarantees than normal COTS. He has presented at BlackHat USA/EUR, IEEE S&P, ACM CCS, Defcon, CanSecWest, PacSec, Hack in the Box KUL/AMS, Microsoft BlueHat, Shmoocon, Hack.lu, NoSuchCon, SummerCon, ToorCon, DeepSec, VirusBulletin, MIRCon, AusCERT, Trusted Infrastructure Workshop, NIST NICE Workshop, DOD Information Assurance Symposium, and MTEM.
2012-2013 Class Materials
Day 1 Part 1 : Introduction (17:10)
Day 1 Part 2 : Compilers (3:14) (see the 2011 version for this content)
Day 1 Part 3 : Executables Overview (10:05)
Day 1 Part 4 : PE DOS Header (2:51)
Day 1 Part 5 : PE NT Header, File Header (10:58)
Day 1 Part 6 : BinHunt Round 1 (18:29)
Day 1 Part 7 : PE NT Header, Optional Header (28:39)
Day 1 Part 8 : BinHunt Round 2 (5:02)
Day 1 Part 9 : PE Section Headers (25:28)
Day 1 Part 10 : PE File Header Pop Quiz (3:32)
Day 1 Part 11 : BinHunt Round 3 (6:00)
Day 1 Part 12 : PE Imports (34:32)
Day 1 Part 13 : BinHunt Round 4 (16:12)
Day 1 Part 14 : PE Bound Imports (23:38)
Day 1 Part 15 : PE Optional Header Pop Quiz (5:54)
Day 1 Part 16 : PE Delay Load Imports (17:24)
Day 1 Part 17 : PE Delay Load Imports In Debugger (13:58)
Day 1 Part 18 : BinHunt Round 5 (15:33)
(4:17:53 total for Day 1, not including lab time)
Day 2 Part 1 : PE Sections Slide Correction (1:08)
Day 2 Part 2 : Delay Load Imports Correction And Questions (9:07)
Day 2 Part 3 : ASLR Questions (8:20)
Day 2 Part 4 : IAT Hooking Demo (5:42)
Day 2 Part 5 : Day 1 Review (14:22)
Day 2 Part 6 : Dependency Walker (1:46)
Day 2 Part 7 : Runtime Importing (4:37)
Day 2 Part 8 : Exports (19:33)
Day 2 Part 9 : Forwarded Exports (16:12)
Day 2 Part 10 : BinHunt Round 6 (6:49)
Day 2 Part 11 : Debug Information (11:47)
Day 2 Part 12 : Relocations (9:25)
Day 2 Part 13 : Thread Local Storage (TLS) (14:16)
Day 2 Part 14 : Resources (16:20)
Day 2 Part 15 : Load Config (8:09)
Day 2 Part 16 : Code Signing (Security Directory) (3:22)
Day 2 Part 17 : PE to ELF Topic Transition (3:34)
Day 2 Part 18 : ELF Intro (12:06)
Day 2 Part 19 : ELF Header (6:49)
Day 2 Part 20 : Lab: Using readelf to View the ELF Header (4:10)
Day 2 Part 21 : ELF Program (Segment) Headers (12:33)
Day 2 Part 22 : Lab: Compile Hello and Use readelf to View Program Headers (7:40)
Day 2 Part 23 : Graphical Representation of Program (Segment) Headers in Hello (19:14)
Day 2 Part 24 : PopQuiz: ELF Segment vs. PE Section Similarities And Differences (5:32)
Day 2 Part 25 : ELF Section Headers (13:39)
Day 2 Part 26 : Lab: Using readelf to View Section Headers (6:02)
Day 2 Part 27 : PopQuiz: ELF vs. PE Sections Similarities and Differences (3:05)
Day 2 Part 28 : Procedure Linkage Table (PLT) & Dynamic Linking (9:02)
Day 2 Part 29 : Packers (8:58)
Day 2 Part 30 : UPX Packing Example (17:04)
Day 2 Part 31 : Viruses (7:49)
Day 2 Part 32 : BabysFirstPhage.c Code Walkthrough (28:06)
Day 2 Part 33 : Class Conclusion (3:15)
(5:20:33 total for Day 2, not including lab time)
2011 Class Materials
Day 1 Part 1 (1:05:28, 587 MB)
Day 1 Part 2 (1:16:42, 632 MB)
Day 1 Part 3 (48:46, 463 MB)
Day 1 Part 4 (46:56, 432 MB)
Day 1 Part 5 (1:08:26, 801 MB)
Day 1 Part 6 (1:07:34, 803 MB)
Day 2 Part 1 (59:17, 690 MB)
Day 2 Part 2 (52:49, 651 MB)
Day 2 Part 3 (1:20:23, 834 MB)
Day 2 Part 4 (1:03:02, 731 MB)
Day 2 Part 5 (56:12, 743 MB)
Day 2 Part 6 (1:12:28, 859 MB)
(12:38:03 total, sans lab time)
The videos are useful for students, but also more useful for potential instructors who would like to teach this material. By watching the video, you will better understand the intent of some slides which do not stand on their own. You are recommended to watch the largest size video so that the most possible text is visible without having to follow along in the slides.
Revision History:
06-28-2014 - Uploaded second day of videos for the 2012/2013 version of the class.
03-02-2014 - Uploaded first day of videos for the revised 2012/2013 version of the class.
03-17-2013 - Updated class slides for the version taught in fall 2012 which was updated to cover both 64 bit PE and ELF. The PE slides were split into two decks as PE is now more than 200 slides alone. Also the class was taught as a 2 day again but with skipping the compilers information, to make enough time to cover ELF. Also made the fine-grain class map more prominent. (It needs to be updated to include ELF though. This will be removed when that is done.)
04-14-2012 - Videos uploaded to Youtube
09-06-2011 - Videos uploaded to Archive.org
02-16-2011 - Initial class content upload
If you have used and modified this material, we would appreciate it if you submit your modified version for publishing here, so that all versions can benefit from your changes.
