Welcome back… and with a new project!
Ok so this project is not that new, and not so unrelated to the previous one.
So, let’s catch up! for last year’s GSoC, we managed to:
- complete the new debugger functionality (to match the old debugger ones)
- add support for filtering the stack (so that you can hide the contexts you’re not interested in) with a flexible and extensible API.
- all these changes are integrated in Pharo 3.0 image 😀
Buuuut… (yes, there’s always a but) there were two parts of the project we didn’t get to work in (because of lacking of time and platform migration issues): a) the smart breakpoints API and b) the Object Centric Debugger. So, maybe you have guessed, my current project is actually to solve that “technical debt” 🙂
Well, done with the catching up thing. Now, what’s going on?
Debugging in Pharo
Debugging process in Pharo is still a little bit not straightforward. For example, for inserting a breakpoint in the code, you have to explicitly write it. So what? writing “self halt” is not that hard… But adding even such a little line like self halt to your code generates a dirty package in Monticello… and you are not going to commit that line once you fix your bug. And, on the other hand, you don’t want to change your code by putting information that has nothing to do with your application. So in an ideal world you want to be able to add debugging information, but without modifying the source code.
And then an interesting question is which kind of information would be nice to add? The most popular IDEs (i.e. Eclipse, IntelliJ IDEA) provide debuggers that allow you to insert several specialized breakpoints: by line, by condition, by exception, to name some. Also they provide expression watchers, to be able to follow the value of a variable dynamically during the debugging process. Currently, even tough Pharo’s reflective capabilities are huge, there’s is no support for such kind of features, which will greatly improve the debugging process.
The heart of the tool: Reflectivity
As I have already mentioned, Pharo’s reflective capabilities are huge. You can introspect and modify objects in runtime, and most parts of the execution process are already reified: execution context, message sends, sender, receiver, etc; so you can inspect and even adapt the method lookup, the compiling process, and anything you like. So, how does this help to solve our current dilemma? Let’s imagine that you have a tool that can intercept, for example, a message send that is of your interest, add some extra information, and then continue the normal execution process. This looks exactly like what we need, doesn’t it? And then let’s go an step further, what if this tool did all that stuff by using Pharo’s reflective mechanisms (so that you can virtually add information anywhere and anywhen you want)? Well, such a tool already exists, and it’s called Reflectivity. A little more in detail: Reflectivity introduces a new abstraction: the metalink. A metalink is an object that reifies the information that we want to add, and when we want to add it; among other things, it knows:
- which object is in charge to perform the operation we want to add, the metaobject
- which concrete action (that is, which message we have to send to the metaobject)
- when to insert the code: a control object (so for example you can insert information before, after, or instead a certain piece of code).
But of course just having the link doesn’t imply anything by itself, we need a way to intercept the execution that is of our interest and add this link. Reflectivity does that by traversing the AST generated in the compilation process, and adding a new node which represents a message send to the metaobject. Since this code is generated and added in runtime, the original source code is not affected; thus providing a mechanism to implement several tools, such as profilers, debuggers, etc… in any granularity we want.
A little example
Let’s suppose we have a class Bar, with an instance-side method #foo. For this example is not very much important what the method does, so let’s keep it simple:
Now let’s suppose we have a Counter class, with a single message, #inc, which of course increments a counter 😉
#Counter>>inc count := count + 1
And we want to count how many times #foo is called. Now, you can imagine that I’m going to use a metalink to achieve this, but how? First we identify the method we want to intercept, in this case #foo. Then, we need a metaobject, and a message: we have this counter object, so we can use it 🙂 . After that, we specify when in the execution we want to call the counter: by default, this is done before the first statement; so we’re going to leave it as is. So, the actual code for creating such a link looks something like this:
counterLink := RFLink metaobject: Counter new selector: #inc
So now we have the link, then we must install it. This is done by generating a method wrapper for the original code, which contains the message send introduced by the link. The implementation details are not much important (and probably may create some confusion), but once you have the wrapper, the message #run:with:in: is sent (which has a very informative comment):
run: selector with: arguments in: receiver "Called when the wrapper is installed in the method dictionary instead of the compiled method. Generates a new compiled method , install it and call it with the same arguments" self generateCompiledMethod. self installCompiledMethod. self flushCache. ^ receiver perform: selector withArguments: arguments
What happens then, is that the code generated by the wrapper is executed reflectively (by sending #perform:withArguments: ) with the original arguments. Then the wrapper is uninstalled, so the original method is still there. Of course, I’m omitting a lot of the details, but to prove this works, I can inspect the counter to see if it has increased after sending #foo to an instance of Bar class:
And a more fine grained example: if I just generate and install the wrapper, and I have a collection of instances of #Bar:
The initial state of the counter and the collection, if we inspect them:
if now I use #collect: to send the message #foo to the elements of the collection, the counter should be increased to 3, but #foo should still just return 2:
Of course this approach is not fool proof: there are some implementation issues (such as what happens if the message you want to intercept is sent not only at base level, but in the meta-level as well? This generates an infinite recursion that we should handle), and also all this information should be hidden for the end-user (this is just a little explanation of how it works, but you won’t have to explicitly create a link, nor install it yourself).
But it provides a simple yet powerful way to introduce meta-level information in runtime; which comes in handy, for example, for interrupting the execution without changing the application code (which is basically what a breakpoint does 😉 )
State of the art
What it’s done:
- Currently there are two implementations of Reflectivity, so the first step was to “merge” them. My approach is to try and reuse most of them as possible, and then write the most simple version I can (so that we can build on top of it later, adding whatever it is necessary).
- Writing tests. But I should write more 🙂
Future work (aka the Roadmap):
- Write an API for Smart Breakpoints. We need a model that is flexible (to allow different applications and frameworks to customise them as needed) as well as a simple way to integrate it to the current debugger (ideally, the end user shouldn’t be aware of link installation/uninstallation, but instead just say something like “break here”, and the framework will do the magic)
- Add support for setting execution watchers, to follow how the value of a variable/expression changes during the execution
- Highlight hot methods on source: provide a way to identify which methods are highly likely to break everything if you change them.
- Heimdall: the implementation of the Object Centric Debugger on top of Reflectivity. Sadly it is not possible to reuse Bifröst implementation because after several tries we weren’t able to migrate it from old versions of Pharo image.
TL:DR (you wrote so much and I’m too lazy to read everything…)
Long story made short: My project right now is to provide a simple way to set up breakpoints in the debugger, without changing the application code. Furthermore, we want these breakpoints to be Smart: the ability to break when a condition is met, to create custom breakpoints, and to create them not only from the UI, but programatically as well, is desired. After that, we want to be able to use the debugger but scoped to one specific object, and with dynamic-based abstractions (the objects are dynamic, but the concepts used in debugging tools are rather statical and more related to the source code, and not to the live objects). And we will use Reflectivity, a framework to insert meta-level information into the AST nodes, to build such tools.
Aaaaand… that’s it. Thanks for reading 🙂 Stay tuned!