Contents
otrace is an object-oriented debugger for nonlinear tracing of asynchronous or multithreaded interactive programs. It addresses some of the limitations of sequential debugging techniques which do not work well with server programs, where multiple requests are handled in parallel. For example, instrumenting web servers with print/logging statements can often result in voluminous log output with interleaved streams of messages.
otrace takes a different approach to debugging that relies less on sequential operations. Its features including taking "snapshots" of variables for tracing, "tagging" objects for tracking across different method invocations, and modifying live code ("monkey patching") to insert print statements etc.
otrace maps all the objects in the running program, as well as the "snapshot" objects, to a virtual filesystem mounted under /osh. It provides a shell-like interface, oshell, with commands like cd, ls etc. that can be used to browse classes, methods, and instance variables in the virtual filesystem. Tab completion and simple wildcarding are supported.
If you wish to install otrace without the sample programs, the easy_install otrace command should be sufficient (provided the setuptools module is installed).
The latest archive of otrace including sample programs may be downloaded from the Github tags page. The unzipped archive should contain the following files (and some more):
hello_trace.py ordereddict.py otrace.py README.rst setup.py ...
All the code for the otrace module is contained in a single file, otrace.py. (For python 2.6 or earlier, you will also need ordereddict.py.) To use otrace without installing it, just ensure that these files are present in the module load path. If you wish to install otrace, use:
python setup.py install
This README file provides a brief introduction to otrace. A tutorial using a demo program is available in docs/GettingStarted.rst Additional documentation will be made available on the project home page, info.mindmeldr.com/code/otrace.
Report bugs and other problems using Github Issue Tracker.
otrace does not consume any resources until some tracing action is initiated. So it can be included in production code without any performance penalty. otrace works well with detached server processes (daemons) via the GNU screen utility that emulates a terminal.
otrace is meant to be used in conjunction with an event loop, which is usually present in programs that interact with users such as web servers or GUI applications. otrace takes control of the terminal, and would not work very well with programs that read user input directly from the terminal (or standard input).
To use otrace, simply import otrace and instantiate the class otrace.OShell, which provides a unix-like shell interface to interact with a running program via the terminal.
Here is a simple server example:
import BaseHTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
from otrace import OShell, traceassert
http_server = BaseHTTPServer.HTTPServer(("", 8888), SimpleHTTPRequestHandler)
oshell = OShell(locals_dict=locals(), globals_dict=globals(),
new_thread=True, allow_unsafe=True, init_file="server.trc")
try:
oshell.loop()
http_server.serve_forever() # Main event loop
except KeyboardInterrupt:
oshell.shutdown()
Usage notes:
If you run in oshell in its own daemon thread as shown above, use the ^C sequence to abort the main thread, and call OShell.shutdown from main thread to cleanup terminal I/O etc.
If you run oshell in the main thread and the event loop in a separate thread, ^C will abort and cleanup oshell. You may need to shutdown the event loop cleanly after that.
Install the python readline module (easy_install readline) to enable TAB command completion.
- To start a detached server (daemon) process, use the command:
screen -d -m -S <screen_name> <executable> <argument1> ...
- To attach a terminal to this process, use:
screen -r <screen_name>
By default, otrace logs to the logging module. Subclass TraceCallback, overriding the methods callback and returnback to implement your own logging (see DefaultCallback for a simple example)
otrace uses a Virtual Directory Shell Interface which maps all the objects in a a running python program to a virtual filesystem mounted in the directory /osh (sort of like the unix /proc filesystem, if you are familiar with it). Each module, class, method, function, and variable in the global namespace is mapped to a virtual file within this directory. For example, a class TestClass in the globals() dictionary can be accessed as:
/osh/globals/TestClass
and a method test_method can be accessed as:
/osh/globals/TestClass/test_method
and so on.
otrace provides a unix shell-like interface, oshell, with commands such as cd, ls, view, and edit that can be used navigate, view, and edit the virtual files. Editing a function or method "monkey patches" it, allowing the insertion of print statements etc. in the running program.
The trace command allows dynamic tracing of function or method invocations, return values, and exceptions. This is accomplished by dynamically decorating (or wrapping) the function to be traced. When a trace condition is satisfied, the function-wrapper saves context information, such as arguments and return values, in a newly created virtual directory in:
/osh/recent/*
These trace context directories can be navigated just like /osh/globals/*. (If there are too many trace contexts, the oldest ones are deleted, unless they have been explicitly saved.)
oshell allows standard unix shell commands to be interspersed with oshell-specific commands. The path of the "current working directory" determines which of the these two types of commands will be executed. If the current working directory is not in /osh/*, the command is treated as a standard unix shell command (except for cd, which is always handled by oshell.)
oshell supports the following commands ([..] denotes optional parameters; | denotes alternatives):
alias name cmd <arg\*> <arg\1>... # Define alias for command cd [pathname] # change directory to "pathname", which may be omitted, "..", or "/" or a path cdls [pathname] # cd to "pathname" and list "files" (cd+ls) del [trace_id1..] # Delete trace context dn # Command alias to move one level down in stack frames in a trace context (to a newer frame) edit [-f] (filename|class[.method]) [< readfile] # Edit/patch file/method/function exec python_code # Execute python code (also !<python_code>) help [command|*] # Display help information lock # Lock terminal until password is entered ls [-acflmtv] [-(.|..|.baseclass)] [pathname1|*] # List pathname values (or all pathnames in current "directory") pr python_expression # Print value of expression (DEFAULT COMMAND) pwd # Print current working "directory" quit # Quit shell repeat command # Repeat command till new input is received resume [trace_id1..] # Resume from breakpoint rm [-r] [pathname1..] # Delete entities corresponding to pathnames (if supported) save [trace_id1..] # Save current or specified trace context set [parameter [value]] # Set (or display) parameter source filename # Read input lines from file tag [(object|.) [tag_str]] # Tag object for tracing trace [-a (break|ipdb|pdb|hold|tag)] [-c call|return|all|tag|comma_sep_arg_match_conditions] [-n +/-count] ([class.][method]|db_key|*) # Enable tracing for class/method/key on matching condition unpatch class[.method]|* [> savefile] # Unpatch method (and save patch to file) untag [object|.] # untag object untrace ([class.][method]|*|all) # Disable tracing for class/method up # Command alias to move one level up in stack frames in a trace context (to an older frame) view [-d] [-i] [class/method/file] # Display source/doc for objects/traces/files
The default command is pr, which evaluates an expression. So you can simply type a python variable to print out its value. You can also insert otrace.traceassert(<condition>,label=..,action=..) to trace assertions.
otrace.py and the demo program hello_trace.py work with Python 3, after porting using the 2to3 tool. Further testing remains to be done.
Reliability: This software has not been subject to extensive testing. Use at your own risk.
Thread safety: In principle, otrace should thread-safe, but more testing is needed to confirm this in practice.
Memory leaks: The trace contexts saved by otrace could potentially lead to increased memory usage. Again, only experience will tell.
Platforms: otrace is pure-python, but with some OS-specific calls for file, shell, and terminal-related operations. It has been tested only on Linux and Mac OS X so far, although the demo program works with the Windows console as well.
- Current limitations:
- Decorated methods cannot be patched.
- TAB command completion is a work in progress.
- Spaces and other special characters in command arguments need to be handled better.
otrace was developed as part of the Mindmeldr project, which is aimed at improving classroom interaction.
otrace is distributed as open source under the BSD-license.