Embedded developers can use a plethora of debugging technologies to solve their problems. But for the most difficult of bugs, there is one option in particular that many seasoned developers turn to - ETM instruction tracing. This is the "heavy guns" of debugging.
Read this blog post to learn more on ETM instruction tracing.
With instruction tracing, the processor emits information on what it is doing to a trace-enabled debug probe. The debugger can then extract the trace data and visualize to the developer what the processor has been doing during a previous execution run.
In effect, instruction tracing display the execution history of a processor. You can therefore work out what was going on when it was malfunctioning, even if it was not stopped at the time of the bug. All the clues are there to be found if you mine the trace data.
Additionally, instruction tracing may be useful in situations where execution cannot stop on a breakpoint. Motor control is such an application, where stopping the software execution would physically damage the motor. Instruction tracing comes to help - you can stop execution in a controlled manner a bit later, but still investigate what the processor was doing at the time of the bug.
But there are two problems with instruction tracing:
- Instruction tracing requires expensive trace-enabled debug probes. For Cortex-M devices, this often means a SEGGER J-Trace debug probe. These typically have a large onboard trace buffer, although the latest models now have real-time streaming capabilities, thus leveraging the PC hard drive as the trace buffer.
- Instruction tracing produces massive amounts of recorded trace data, and so finding the right information in all the data becomes a major problem. In short, you are often finding yourself looking for a nail in a haystack. Trigger conditions and filtering is necessary to find the right information in oceans of data.
An instruction trace can be 200 times larger in human readable form compared to in compressed binary form. For example, the 16MB trace buffer in a SEGGER J-Trace expands to 2-3 GB when converted to human readable format in .CSV or .TXT files.
To use ETM instruction tracing, you need to connect an instruction-trace enabled JTAG probe to the electronic board and the developer PC, and enable ETM tracing in the debugger. Furthermore, you ought to configure triggering conditions that only start recording in certain situations to avoid being drenched in data.
Trace triggers can either be used to start or stop tracing. Ideally, you should set trace start and stop triggers such that you record as little information as possible, right where the bug is located.
Trace trigger actions start or stop the trace when the condition is triggered. Different types of memory accesses can trigger the action:
- PC: Trigger when execution reaches an address
- Data read: Trigger when data is read from an address
- Data write: Trigger when data is written to an address
- Data Read/Write: Trigger when data is read or written to an address
When you later on suspend the execution, the recorded trace log is uploaded to the trace view. In this view, you can see the sequential flow of execution that was recorded in the trace buffer. You can thus analyze each and every C or assembler instruction that was executed - step by step, and in great detail.
Additionally, the trace log view can show an execution flow branch icon on each C/assembler line, further indicating the nature of this code. Various icons exist for different execution flow branches:
- Call a new function
- Return from a function
- Jump up in the code
- Jump down in the code
- Iterate on the same instruction
- A conditional branch was not taken
An "End of trace marker" can sometimes be displayed at the end of the trace log. This indicates the buffer has overflowed, a situation where some trace data has most likely been lost. Beware of this situation, as data might be missing and can thus not be trusted.
The trace log view can be configured in different visualization modes, which effectively acts as different "zoom" levels. The visualization modes - or "zoom levels" - of the Trace log view is:
- Function call tracing
- C line tracing
- C/assembler mixed mode tracing
- Assembler tracing
- Raw trace packet log
Because the data volumes can easily become enormous even with well-defined start and stop triggers, the trace log view contains a Search function. You can thus, for example, search for an erroneous value that a variable sometimes "magically" get.
It is also possible to export the trace to log .CSV or .TXT formats. This enable you to analyze the trace log offline, possibly using a script utility you create for the purpose of mining the trace log data.
To learn more on ETM instruction tracing, read this whitepaper:
Read the superset blog post that covers the entire topic of Cortex-M debugging:
You can also get a good overview of Serial Wire Viewer real-time tracing on Cortex-M cores by reading this white paper on the subject: