Malicious Input: How Hackers Use Shellcode

It’s Black Hat 2019 this week, with Def Con following hard on its heels, and even if you’re not going to either of these events, if you have a stake in the world of cybersecurity your social media feeds are going to be filled with plenty of “hacker talk” over the next 7 days or so. Of course, you know all about hashes in cybersecurity and how to decode Base64; you’re likely also familiar with steganography, and maybe you can even recite the history of cybersecurity and the development of EDR. But how about explaining the malicious use of shellcode? You know it has nothing to do with shell scripts or shell scripting languages like Bash, but can you hold your own talking about what shellcode really is, and why it’s such a great tool for attackers?

Not sure? No problem. We’ve got just the post for you. In the next ten minutes, we’ll take you through the basics of shellcode, what it is, how it works and how hackers use it as malicious input.

What is Shellcode?

We know shellcode has nothing to do with shell scripting, so why the name? The connection with the shell is that shellcode was originally mainly used to open or ‘pop’ a shell – that is, an instance of a command line interpreter – so that an attacker could use the shell as a means to compromise the system. Imagine if you could get a user to input some seemingly innocent string to a legitimate program on their system that would magically open a reverse shell to your machine? That’s the ultimate pwning prize. It also takes very little code to spawn a new process that will give a shell, so popping shells is a very lightweight, efficient means of attack.

In order to achieve it, you’d need to find an exploitable program and fashion some malicious input string – the shellcode – containing small chunks of executable code to force the program into popping a shell. This is possible because for most programs, in order to be useful, they need the ability to receive input: to read strings and other data supplied by the user or piped in from another program.

Shellcode exploits this requirement by containing instructions telling the program to do something it otherwise wouldn’t or shouldn’t. Of course, almost no program is going to easily misinterpret data as code without a bit of persuasion, and the primary name of the game when it comes to persuading programs to engage in this kind of undefined behavior is another hacking conversation favorite: the buffer overflow.

What is a Buffer Overflow?

A buffer overflow occurs when a program writes data into memory that is larger than the area of memory, the buffer, the program has reserved for it. This is a programming error, as code should always check first that the length of any input data will not exceed the size of the buffer that’s been allocated. When this happens the program may crash, but maliciously-crafted input may instead allow an attacker to execute their own code when it overflows into an area of executable memory. Here’s a simple example of a buffer overflow waiting to happen.

The program reserves 16 bytes of memory for the input, but the size of the input is never checked. If the user enters a string longer than 16 bytes, the data will overwrite adjacent memory – a buffer overflow. The image below shows what happens if you try to execute the above program and supply it with input greater than 16 bytes:

However, just causing a buffer overflow in a program isn’t on its own much use to attackers, unless all they want to do is bring the application to a crashing halt. While that would represent a win of sorts for attackers whose objective is some kind of denial of service attack, the greater prize in most cases is not just causing the overflow but using it as a means to take control of execution. Once this is achieved, the target device can fall under the hacker’s complete control.

Controlling Code Execution

When we create a buffer overflow, the aim is to write a sufficiently large amount of data into the program’s memory so that two things happen. First, we fill up the allocated buffer, and second we supply enough extra data so that we overwrite the address that will be executed next with our own code.

This isn’t simple, but it might sound harder to do than it actually is. Because of the nature of how program memory is mapped out, when any function is called, there’s always a pointer held in memory to the address of the next function that should be executed after the currently executing one; this pointer is known as the Instruction Pointer, sometimes referred to as EIP (32 bit) or RIP (64 bit).

By reverse engineering a particular program and with a lot of fuzzing and experimenting, we can determine both whether a given program contains any functions that are vulnerable to a buffer overflow and, if so, the address of the Instruction Pointer when that vulnerable function has finished calling.

Knowing the offset – the memory address – of the Instruction Pointer at that point in code means we can determine precisely how much extra data we need to overflow the buffer and insert our own code at the address of the Instruction Pointer. When we do that, the program will try to execute the code at the address we’ve written to the RIP register. If that code is junk, like in the example above, the program will crash, but if it isn’t – if it’s a valid address, things start to get more interesting.

From Buffer Overflow to Shellcode

Having achieved a buffer overflow and mapped out the memory addresses and offsets, the attacker has two options. Either the malicious input could write the address of another function of the program to RIP or the attacker could try to jump to an address in which attack code has already been inserted.

The first case may often times be all that is needed. Suppose the program contains a function that is normally only reached if the user is authorized to take that action. In such a case, the attacker can use the buffer overflow to write the address of that function directly into the Instruction Pointer, bypassing any earlier function that would have checked for authorization.

The second scenario is far more tricky. Suppose I want the program to do something it’s not programmed to do, like open up a shell? In that case, I need to overflow the buffer not just with junk or an address to jump to, but with executable code – shellcode – as well as the address of where that code begins.

How To Create Shellcode

Now that we understand the mechanism that shellcode exploits, how do we go about creating it? As we saw already, we need to send executable code in the input data, but we can’t just write a bunch of C or C++ instructions into the input.

If we want the program to write executable instructions into memory, then we need to send it raw assembly code in our string. However, there’s a difficulty. We must ensure that our shellcode does not contain any character codes that would translate into a string control character that the program may interpret as, for example, a new line or the end of input.

In order to achieve this and create the shellcode, the process we need to follow involves several steps.

1. Create the program we want to execute in a high-level language like C. As it has to fit in a small amount of memory – the size of the buffer plus the offset to RIP – it should be as concise as possible. The shellcode used in this example, which spawns a shell via execve, is a mere 23 bytes.

2. After compiling the code and checking that it does what we expect, use a disassembler to view the raw assembly.

3. Optimize the assembly, such as replacing any 00 null-bytes with other instructions. At line 8, notice a couple of things. First, the hexadecimal 48 31 f6, which represents the instructions or opcodes.

xor %rsi, %rsi

Second, note the use of XOR. This is an example of sidestepping the restriction on not being able to use the hex 00 (null-bytes). We want to push 0 onto the stack by first loading 0 into the %rsi register. The natural way to do that would be:

mov $0x0, %rsi

But that would produce a null-bytes string to represent 0x0. We can get around that by using XOR on the same value, %rsi, which returns 0 as a result but doesn’t give us a 00 opcode, since the opcode for XOR is 0x31 in Intel 32-bit and 64-bit architectures.

4. Next, we need to extract the opcodes and create a shellcode string. The second column in the assembly above contains all the opcodes, so all we need to do to is extract these and create the shellcode string by prefixing each hex byte with x.

x48x31xf6x56x48xbfx2fx62x69x6ex2fx2fx73x68x57x54x5fx6ax3bx58x99x0fx05

5. Finally, all we need to do is feed our shell code to a vulnerable program, or create our own and convince a user to execute it, like this one.

Some other great examples of this process can be seen here.

Protecting Against Shellcode

You would think that buffer overflows, which have been known about for decades, should be becoming rarer, but in fact the opposite is true. Statistics from the CVE database at NIST show that vulnerabilities caused by buffer overflows increased dramatically during 2017 and 2018. The number known for this year is already higher than every year from 2010 to 2016, and we still have almost 5 months of the year left to go.

Clearly, there’s a lot of unsafe code out there, and the only real way you can protect yourself from exploits that inject shellcode into vulnerable programs is with a multi-layered security solution that can not only use Firewall or Device controls to protect your software stack from unwanted connections, but also that uses static and behavioral AI to catch malicious activity both before and on execution. With a comprehensive security solution that uses machine learning to identify malicious behavior, attacks by shellcode are seen just like any other attack and stopped before they can do any damage.

Conclusion

In this post, we’ve taken a look at what shellcode is and how hackers can use it as malicious input to exploit vulnerabilities in legitimate programs. Despite the long history of the dangers of buffer overflows, even today we see an increasing number of CVEs being attributed to this vector. Looking on the bright side, attacks that utilize shellcode can be stopped with a good security solution. On top of that, if you find yourself in the midst of a thread or a chat concerning shellcode and malicious input, you should now be able to participate and see what more you can learn from, or share with, others!

---

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security