Lab 1 - Command Line Debugging Tools

Due: Wednesday by noon

The purpose of this lab is to become familiar with standard command-line debugging tools that can be used in Unix/Linux systems (with equivalents on other operating systems). These tools allow you to investigate a piece of binary code, even if you do not have the source code available.

Part 1 - gdb Debugger

The gdb debugger can be used to investigate an executable created from many different source programming languages, such as C, C++, Java, Objective C, and so on.

First refresh your basic gdb skills by working through my lab from CMPS 223 for using gdb to debug a core dump and to step through an executable: gdb lab

For this lab, let's focus on different ways to debug a core dump from a C/C++ program. Grab the file lab1.cpp from the CMPS 223 lab for the rest of this lab. Don't forget to use the command 

ulimit -c unlimited
to allow core dump files.

There are several ways to compile a program. If the programmer is debugger-friendly, they will compile with debugging information, which is the -g option for gcc/g++. A program with debugging information has access to the source code, the symbol table, and other pieces of information that makes it very easy to trace through the code line-by-line.

Using the -g option is what you should have done in the review from Lab 1 from CMPS 223.

The next way a program could be compiled is the "don't care" or default mode, which is to omit the debugging info option from the compiler command line, but to also omit any post-processing of the executable. For example, the command 

g++ lab1.cpp
would create an executable named a.out that does not have the debugging information available.

Run that code to get the core dump and then use gdb to investigate the core file. You'll notice that the commands you could use from CMPS 223 Lab 1 no longer work because the executable does not have debugging info available.

But also notice that backtrace does give some information about the executable. You can see the information about the library call (frame 0) and the address for the main() function (frame 1).

In fact, you can do assembly debugging from within gdb, assuming you know the register names for your architecture. The most important register to start with is the instruction pointer. On Sleipnir, we are using the Intel x86-64 architecture, so the instruction pointer is stored in $rip.

To see all the registers in use by your core file, use the gdb command 

info registers

To print out the assembly instruction at the instruction pointer, use the following gdb command: 

x/i $rip
This tells gdb to translate (the "x" portion of the gdb command) the information at memory address $rip into an instruction (the "i" portion of the gdb command).

As long as you are still in frame 0, you should see that the assembly command mov $eax,0($rbp) was where the segmentation fault occurred. If you use the following commands, you can see that $rbp (which should have the base address of the array) is the null pointer: 

print $eax
print $rbp
You can use the x/i <address> command to investigate other assembly instructions.

You can use the command 

disassemble
to display the assembly for that frame.

There are even instruction-level equivalents to the step and next gdb commands, named appropriately stepi and nexti.

The final way a program could be compiled is to intentionally remove all source code information from the file by using the strip command. For example, you could do the following: 

g++ -o lab1 lab1.cpp
strip lab1

The stripped executable can be debugged with gdb in a similar fashion to the "don't care" executable.

If you have an unknown executable, you can get information about its architecture and whether or not it has been stripped using the command: 

file <filename>

Take Away: gdb - It's not just for source code.

Part 2 - Other Ways to Debug an Executable from the Command Line

Depending on the information stored in the executable, there are several ways beyond gdb to go about investigating the executable. This lab is not going to be an exhaustive investigation of these tools, but rather a brief overview.

The simplest way to start is to see what human-readable information is stored in the executable using the strings command: 

strings <filename> | less
This may or may not reveal much information beyond what file revealed. For example, on our executable from Part 1, it really doesn't give you much of an insight beyond what libraries the executable uses and the prompts the program has.

The real "old school" way of debugging an executable is to fire up a command line hex reader or hex editor and break out your binary to assembly manual. While this can be a tiresome task for debugging a large program, it is still useful to know how to view a binary file in hexadecimal (or binary) format.

To simply dump the contents of a file in hex, use the following: 

hexdump <filename>
This can be rather wordy, so another common technique is to invoke the xxd command from within the vi editor to display the file in hex (with a sidebar of ASCII). Open the binary file in vi and then issue the following vi command to convert it to hex: 
:%!xxd
You can then use standard vi commands to move around (and change, although I don't recommend trying to make changes at this point) the binary file. When you are done, you can issue the following vi command to convert back to binary: 
:%!xxd -r
Or if you've made no changes, you can just use :q! to quit vi without saving.

What to Turn In

This is a skills-based lab, so you can either call me over during class to look at your terminal buffer to see that you've tried all portions of the lab (instant grading! highly recommended!).... OR .... you can upload a log file of your terminal session to Moodle (and I don't guarantee the speediness of grading).