Updated: Dec 31, 2022
I am about to embark upon probably the most technically demanding tutorial series, Reverse Engineering Malware. Before I do so, I thought I would take a few moments to explain why YOU should study and invest your time into reverse engineering. Please take a moment to read the following and then, hopefully, decide whether this discipline is worth your time to advance your career in cyber security.
What is Reverse Engineering Malware?
In this series, we will be dissecting known malware to understand how it works, its operation and its "signature". According to the Merriam-Webster dictionary, reverse engineering is defined as "disassemble or analyze in detail in order to discover concepts involved in manufacture". That is precisely my intent with this series, to analyze in detail to discover concepts involved in manufacture" of malware.
Furthermore, Wikipedia defines reverse engineering as;
the process of discovering the technological principles of a(n)....application through analysis of its structure, function and operation. That involves sometimes taking something apart and analyzing its workings in detail, usually with the intention to construct a new device or program that does the same thing without actually copying anything from the original. (my emphasis added)
We will be using a number of different tools in this analysis including virtual machines, sandboxes, unpackers, disassemblers and debuggers to do so. Wherever possible, I will use free and open source tools.
Why Reverse Engineering Malware?
#1 To Gain a Deeper and More Thorough Understanding of Applications and Operating Systems
If nothing else, by reverse engineering malware, you will gain a deeper and more thorough understanding of the operating systems and applications. Malware must use and exploit these operating systems and applications for it's own malicious purposes and by dissecting the malware and its operation, you can better understand not only how the malware works, but the functioning of the OS and apps.
#2 Train to Work in Forensic Malware Analysis
Presently, the highest paid and most in-demand sub-discipline in digital forensics is for those capable of dissecting malware and using this information for attribution. When new malware appears, it is most often those that can reverse the malware that are commissioned to attribute its source. This becomes increasingly important in the fields of cyber espionage and cyber warfare between nation states.
By reading and studying this series, Reverse Engineering Malware, you will begin your preparation for this rewarding career.
#3 Build Security Applications
Before can even begin to build security applications to protect systems, you first need to understand how the malware works. Whether working in Intrusion Detection Systems (IDS) development, AV software, firewalls or the latest Artificial Intelligence (AI) based security systems, you must have an understanding how the malware functions and, therefore, how it can be detected and neutralized.
#4 Be Better Prepared as a Forensic Analyst or Incident Response Handler
Reverse Engineering Malware will help incident responders and forensic analysts/investigators to assess quickly the severity of a breach to better plan for recovery. By studying reverse engineering of malware, the forensic investigator can establish the key indicators of a compromise and then plan for containing and recovering from an incident.
# 5 Build Your Own Zer0-Day Exploits
The "Holy Grail" of any security researcher, hacker or pentester is to develop a zer0-day exploit. Whether you are a White Hat trying to develop a proof-of-concept (POC) exploit, a Bug Bounty Hunter, or a Black hat looking to exploit the latest new app, you must understand the inner workings of the operating system, the app and probably, the previous malware that has been developed. In this series, we will explore the inner workings of some common operating systems and applications and some malware that has successfully exploited those systems. By learning how these systems have been compromised in the past, you will have a better concept of how to develop your own. In addition, like all software development, it does not make any sense to "reinvent the wheel". All software developers re-use code to save time and money. This applies equally to malware developers (that code re-use can often provide evidence towards attribution). Here, we will study some common and successful malware over the years, many of which have modules that can be re-used.
Without the ability to build your own exploits, your career as a pentester/hacker will be largely limited to running other peoples' code. To reach the highest echelons of the security/pentesting industry you will need to understand previously deployed malware and develop your own.
Some of the subjects we will address in this series include;
Assembly Language Review
Introduction to Malware Analysis
Reversing with Disassemblers
Reversing with a DeBugger
User Mode Debuggers
Reversing Win32 with IDA Pro
Reversing Stacks and Heaps
Reversing Data Structures
Structured Exception Handling
System level Reversing
Reversing Infection Vectors
Encoders and Compressors
Detecting Debuggers and Disassemblers
I hope you will decide to join me here in this journey toward becoming among the elite of our industry!
When you are ready to begin, start with my Reverse Engineering Malware, Part 1: Getting Started