To halt or not to halt: that’s the question!

Hi all!

The reason why I (lamely, yeah) paraphrase Shakespeare on this post’s title, is because I want to introduce you to conditional breakpoints.

So basically I have added the possibility of interrupting the execution when a certain condition is met. And what is a condition, then? It is simply a block or an statement with a boolean value (it will fail with anything other than those).
Once you set the condition, the link that is inserted by the breakpoint API is triggered only when that condition is met. Sweet, uh?

A very simple example:

Let’s suppose I want to halt when the value of a counter is non zero. We can use the counter from our past experiments:

Screen Shot 2014-08-28 at 13.57.32

Now, we set the breakpoint:

Screen Shot 2014-08-28 at 14.08.48

We select “halt on condition” from the following menu:

Screen Shot 2014-08-28 at 14.08.58

And we write down the condition, when prompted:

Screen Shot 2014-08-28 at 14.09.38

Now if we execute the following workspace:

Screen Shot 2014-08-28 at 14.28.31

Nothing happens! And that’s cool, because our counter is still zero… Now, if I increment the counter:

Screen Shot 2014-08-28 at 14.32.20

And we execute the workspace again, we get our halt:

Screen Shot 2014-08-28 at 14.37.33

Quite simple, but imagine that instead of checking out a counter, you inspect the current context… That enables a lot of possibilities, and I have to keep on experimenting in that regard. Also it may be cool to change the menus so to make them simpler to use…

And that’s it for now. As always, stay tuned, there are more interesting things on the way!

Clari

By clariallende

SmartBreakpoints: alpha release 1.0

Hello!
I am super proud and happy to announce the first release of SmartBreakpoints 🙂

Features:
– One-shot AST based breakpoints
– Persistent breakpoints
– Integration to SmartSuggestions menu (check out the previous post)
– halts only at base level (for now this is by default, later on it would be nice to have the ability to break in system methods as well).
– Works in Pharo 4.0 (with posibility of being backported to 3.0 eventually)

This is an experimental feature: finally I succeeded on being able to insert breakpoints everywhere (even in system methods) but it is not thoroughly tested. So, now it would be nice if you could use it, test it and let me know if it breaks, and where (it’s likely to break somewhere, but I cannot prevent all possible use cases) 🙂

So, how do you get it? Run this Script:

Gofer it
	smalltalkhubUser: 'ClaraAllende' project: 'SmartBreakpoints';
	configurationOf: 'SmartBreakpoints';
	loadStable .
By clariallende

Break it all!

Salut à tous!

In case you don’t know the song that gives title to this post:

Today I’m going to tell you a little more on those fancy breakpoints we want to get 🙂

Now that Reflectivity is more or less ported and almost stable (I keep on doing small refactorings and improvements, and fixing bugs), I have started to work in a small API for inserting breakpoints. After thinking of it for a while (and thanks to Martin Dias for the brainstorming), I decided to integrate them to SmartSuggestions framework, which provides suggestions (duh!) based in the selected text or the cursor position.Currently there are suggestions for:

  • Temporary/Instance/Class Variable
  • Class
  • Method (when you are in the selector)
  • Source (multiple lines)
  • Message

What I did was just adding a new suggestion which triggers the creation of a new breakpoint. In particular, I am interested in methods and message sends.

So, let’s check out how it looks like:

Screen Shot 2014-07-11 at 14.05.17

 

“Break here” triggers the creation of a one shot breakpoint, that inserts a halt using Reflectivity.

Eventually I am thinking of having either a submenu to choose which kind of breakpoint you want to insert or some sort of pop-up menu. Currently, you can insert two kind of breakpoints: persistent ones, and one-shot. All of them are available at message-send and method level, because they are AST based.

For now this is super alfa: still cannot insert breakpoints anywhere you want (you have the suggestion, but in some cases the image still crashes). But I wanted to do some show off 😛

And then, the roadmap would be:

  • Be able to insert breakpoint everywhere (even in the methods that reflect on the system itself, this is related to the image crashes);
  • Have other kinds of breakpoints: by condition, by exception, etc;
  • Have a nice way to uninstall breakpoints;
  • Have a nice way to pick which kind of breakpoint you want to use.

The first item is the most important to get fixed to make SmartBreakpoints “usable”. Once that is done,  I will create a configuration and start asking for feedback (because the only way of full proving the tool will be getting people to use it).

So, stay tuned! I will have more cool stuff to show soon, I hope! 😀

Clari

By clariallende

Show must go on!

Welcome back… and with a new project!

Ok so this project is not that new, and not so unrelated to the previous one.
So, let’s catch up! for last year’s GSoC, we managed to:

  • complete the new debugger functionality (to match the old debugger ones)
  • add support for filtering the stack (so that you can hide the contexts you’re not interested in) with a flexible and extensible API.
  • all these changes are integrated in Pharo 3.0 image 😀

Buuuut… (yes, there’s always a but) there were two parts of the project we didn’t get to work in (because of lacking of time and platform migration issues): a) the smart breakpoints API and b) the Object Centric Debugger. So, maybe you have guessed, my current project is actually to solve that “technical debt” 🙂

Well, done with the catching up thing. Now, what’s going on?

Debugging in Pharo

Debugging process in Pharo is still a little bit not straightforward. For example, for inserting a breakpoint in the code, you have to explicitly write it. So what? writing “self halt” is not that hard… But adding even such a little line like self halt to your code generates a dirty package in Monticello… and you are not going to commit that line once you fix your bug. And, on the other hand, you don’t want to change your code by putting information that has nothing to do with your application.  So in an ideal world you want to be able to add debugging information, but without modifying the source code.

And then an interesting question is which kind of information would be nice to add? The most popular IDEs (i.e. Eclipse, IntelliJ IDEA) provide debuggers that allow you to insert several specialized breakpoints: by line, by condition, by exception, to name some. Also they provide expression watchers, to be able to follow the value of a variable dynamically during the debugging process. Currently, even tough Pharo’s reflective capabilities are huge, there’s is no support for such kind of features, which will greatly improve the debugging process.

The heart of the tool: Reflectivity

As I have already mentioned, Pharo’s reflective capabilities are huge. You can introspect and modify objects in runtime, and most parts of the execution process are already reified: execution context, message sends, sender, receiver, etc; so you can inspect and even adapt the method lookup, the compiling process, and anything you like. So, how does this help to solve our current dilemma? Let’s imagine that you have a tool that can intercept,  for example, a message send that is of your interest, add some extra information, and then continue the normal execution process. This looks exactly like what we need, doesn’t it?  And then let’s go an step further, what if this tool did all that stuff by using Pharo’s reflective mechanisms (so that you can virtually add information anywhere and anywhen you want)? Well, such a tool already exists, and it’s called Reflectivity. A little more in detail: Reflectivity introduces a new abstraction: the metalink. A metalink is an object that reifies the information that we want to add, and when we want to add it; among other things, it knows:

  • which object is in charge to perform the operation we want to add, the metaobject
  • which concrete action (that is, which message we have to send to the metaobject)
  • when to insert the code: a control object (so for example you can insert information before, after, or instead a certain piece of code).

Screen Shot 2014-06-20 at 18.32.07

But of course just having the link doesn’t imply anything by itself, we need a way to intercept the execution that is of our interest and add this link. Reflectivity does that by traversing the AST generated in the compilation process, and adding a new node which represents a message send to the metaobject. Since this code is generated and added in runtime, the original source code is not affected; thus providing a mechanism to implement several tools, such as profilers, debuggers, etc… in any granularity we want.

A little example

Let’s suppose we have a class Bar, with an instance-side method #foo. For this example is not very much important what the method does, so let’s keep it simple:


 #Bar>>foo
           ^2.

Now let’s suppose we have a Counter class, with a single message, #inc, which of course increments a counter 😉


#Counter>>inc
count := count + 1

And we want to count how many times #foo is called. Now, you can imagine that I’m going to use a metalink to achieve this, but how? First we identify the method we want to intercept, in this case  #foo. Then, we need a metaobject, and a message: we have this counter object, so we can use it 🙂 . After that, we specify when in the execution we want to call the counter: by default, this is done before the first statement; so we’re going to leave it as is. So, the actual code for creating such a link looks something like this:


counterLink := RFLink metaobject: Counter new selector: #inc

So now we have the link, then we must install it. This is done by generating a method wrapper for the original code, which contains the message send introduced by the link. The implementation details are not much important (and probably may create some confusion), but once you have the wrapper, the message #run:with:in: is sent (which has a very informative comment):


run: selector with: arguments in: receiver
"Called when the wrapper is installed in the method dictionary instead of the compiled method. Generates a new compiled method , install it and call it with the same arguments"
self generateCompiledMethod.
self installCompiledMethod.
self flushCache.
^ receiver perform: selector withArguments: arguments

What happens then, is that the code generated by the wrapper is executed reflectively (by sending #perform:withArguments: ) with the original arguments. Then the wrapper is uninstalled, so the original method is still there. Of course, I’m omitting a lot of the details, but to prove this works, I can inspect the counter to see if it has increased after sending #foo to an instance of Bar class:

Screen Shot 2014-06-20 at 20.03.17

And a more fine grained example: if I just generate and install the wrapper, and I have a collection of instances of #Bar:

Screen Shot 2014-06-20 at 20.14.17

The initial state of the counter and the collection, if we inspect them:

Screen Shot 2014-06-20 at 20.16.40       Screen Shot 2014-06-20 at 20.18.03

if now I use #collect: to send the message #foo to the elements of the collection, the counter should be increased to 3, but #foo should still just return 2:

Screen Shot 2014-06-20 at 20.22.19         Screen Shot 2014-06-20 at 20.22.09

Of course this approach is not fool proof: there are some implementation issues (such as what happens if the message you want to intercept is sent not only at base level, but in the meta-level as well? This generates an infinite recursion that we should handle), and also all this information should be hidden for the end-user (this is just a little explanation of how it works, but you won’t have to explicitly create a link, nor install it yourself).

But it provides a simple yet powerful way to introduce meta-level information in runtime; which comes in handy, for example, for interrupting the execution without changing the application code (which is basically what a breakpoint does 😉 )

State of the art

What it’s done:

  • Currently there are two implementations of Reflectivity, so the first step was to “merge” them. My approach is to try and reuse most of them as possible, and then write the most simple version I can (so that we can build on top of it later, adding whatever it is necessary).
  • Writing tests. But I should write more 🙂

Future work (aka the Roadmap):

  • Write an API for Smart Breakpoints. We need a model that is flexible (to allow different applications and frameworks to customise them as needed) as well as a simple way to integrate it to the current debugger (ideally, the end user shouldn’t be aware of link installation/uninstallation, but instead just say something like “break here”, and the framework will do the magic)
  • Add support for setting execution watchers, to follow how the value of a variable/expression changes during the execution
  • Highlight hot methods on source: provide a way to identify which methods are highly likely to break everything if you change them.
  • Heimdall: the implementation of the Object Centric Debugger on top of Reflectivity. Sadly it is not possible to reuse Bifröst implementation because after several tries we weren’t able to migrate it from old versions of Pharo image.

TL:DR (you wrote so much and I’m too lazy to read everything…)

Long story made short: My project right now is to provide a simple way to set up breakpoints in the debugger, without changing the application code. Furthermore, we want these breakpoints to be Smart: the ability to break when a condition is met, to create custom breakpoints, and to create them not only from the UI, but programatically as well, is desired. After that, we want to be able to use the debugger but scoped to one specific object, and with dynamic-based abstractions (the objects are dynamic, but the concepts used in debugging tools are rather statical and more related to the source code, and not to the live objects). And we will use Reflectivity, a framework to insert meta-level information into the AST nodes, to build such tools.

Aaaaand… that’s it. Thanks for reading 🙂 Stay tuned!

By clariallende

Filtering the stack – Episode II: Refactor strikes back.

So, last post I introduced you to a new filter API. After some iterations, we have arrived to a nice, flexible filter implementation 🙂 As we wanted filters to be rich objects, able to combine themselves to produce new filters, we designed a hierarchy: Image

  • StackFilter defines methods #and: and #or: , which allow filter combination. Both messages return a BooleanFilter
  • BooleanFilter is a filter subclass, which know how to actually combine two filters.
  • SelectorFilter is a filter for an specific selector or collection of selectors.
  • KernelClassesFilter filters out common classes message sends (like those from Boolean, the Collection Hierarchy, etc)
  • BlockFilter provides a way to create filters from any (boolean) block.

Filters handle one context at a time. Their most important responsibility is to know whether a given context should be displayed or not, through #shouldDisplay: message. For instance:

#BlockFilter>>shouldDisplay: aContext

^self block value: aContext.

And then you can combine a couple of filters, like this:

fromBlockFilter := [:ctx | ctx isNotNil] asFilter.
doItFilter:= SelectorFilter forSelector: #doIt.

doItFilter and: fromBlockFilter.

Which in turn return a BooleanFilter that can be combined again, and so on. Then, the responsibility of filtering the stack relies in the debugger:

filterStack
^self class filterCommonMessageSends ifTrue:[self stack  reject: [:aContext | (self enabledFilters anySatisfy:[:aFilter | aFilter shouldDisplay: aContext])and: [aContext = self currentContext ] ]

But then we realized that we needed not filtering the top context, because we usually want to know where we actually are; and also the first n contexts prior to the top one. So finally the code looks like this:

filterStack
^self class filterCommonMessageSends ifTrue:[
activeFilters := self enabledFilters.
newStack := OrderedCollection new.
activeFilters do: [ :aFilter |
"first loop: add to the newStack those contexts that should not be displayed"
self stack do: [ :aContext | [aFilter shouldDisplay: aContext ] whileFalse: [ newStack add: aContext. ctx := aContext ] ].
"second loop: add the top contexts to keep track of where in the execution we are"
(self stack dropWhile: [:each | each ~= ctx ]) do: [ :context | (aFilter shouldDisplay: context) whileTrue: [ newStack add: context ]]]
newStack
 

I think the most interesting filter is BooleanFilter, which represents the combination of two filters, by a boolean operator (namely #and: and #or: messages). At first I thought of having #AndFilter and #OrFilter, but the I realized that the only difference between them was the boolean operation; so I decided to generalize that behavior by using reflection:

#BooleanFilter>>shouldDisplay: aContext
 ^ (self filters first shouldDisplay: aContext) perform: booleanOperator with: [self filters last shouldDisplay: aContext]

That code could easily be extended to handle more than two filter combination, by folding #shouldDisplay: message send results.

And then, the next challenge was to expose the filter API to the user. We wanted to have default filters (kernel classes, do it, nil messages filters) as global configuration, but we wanted to let the users create new filters and publish them as global configuration if they wished to. So we decided to create a dictionary, whose keys would be the filter classes and booleans as values (meaning if the given filter is enabled or not). So, for global configuration, you’ll see something like this on the Settings Browser:

PharoScreenshot.1

So finally, to avoid having lots of class variables in SpecDebugger to toggle filters, we decided to create a new widget, whose responsibility would be to handle filtering actions. But for the moment, that responsibility still remains on the debugger.

And that’s pretty much it.

Now, a few words, not so unrelated. This year’s Google Summer of Code is ending, and even tough i didn’t make it to the final goal (remember I told you we’d call Heimdall to open the Bifröst?) due to some technical restrictions, I want to say that I’ve learnt a lot, and that I am really thankful to my mentors and to all the Pharo community for the support they’ve given me through this months. I will keep on working on the debugger, but since it won’t be for gsoc anymore… thanks.

So, thank you for reading, and stay tuned!

Cheers!

Clari

By clariallende

Filtering the stack (Episode 1: A new filter)

Hi there! Before telling you anything of what I am doing right now, I want to tell you some great news: the new debugger has been integrated to Pharo3.0 build 🙂 So now we have a prettier and more extensible debugger, and every change and improvement we make it’s going to be integrated faster.

So now, back to business. Now most of the actions available in the old debugger are present in the new one, except for filters.  So the new challenge is to implement that functionality.

In the old debugger, all the logic for filtering the contexts of the stack was all together in one method…  In the new debugger, on the other hand, all the debugging actions are done on the full stack. So my first approach was to create a new filter class, and parametrize the filtering criteria:

#StackFilter
   >>filterWith: aCriteria
           stack select: aCriteria

Here stack is a reference to the stack, and aCriteria is one (or more than one, that’s the intention) of the filters we want to apply on the stack. The first decision I made was to model the criteria as blocks. So, for example, you could use it to filter common messages like this:

context := [Set new] asContext.

process:= Process
forContext: context
priority: Processor userInterruptPriority.
session := DebugSession process: process context: context
filter := StackFilter new stack: session stack.
filter filterWith: filter commonSelectorsFilter.

This code snippet produces a new collection with those contexts which aren’t common selectors.

Now, what about filtering by more than one criteria? Well, my first approach, in a workspace, was something like this:

filter := StackFilter new stack: session stack.
filtered := #(#doItSelectorFilter #commonSelectorsFilter #kernelClassesFilter) inject: Set new into: [:acc :symbol | acc union: (filter filterWith: (filter perform: symbol))].

But after discussing it with my mentors, we arrived to certain conclusions:

  • We’d rather pass the stack as an argument instead of holding it as internal state (this is, filters should be stateless).
  • Flattening the resulting collection isn’t good since it’s more flexible to work with different stacks for each filter, instead of folding filter applications.
  • Most filtering operations are done by intersection rather than union of filtered stacks. Also #union:  wouldn’t preserve the order of the collection.

Considering this, I did a refactor:

context := [ (Set with: 1 with: 2) collect:[:e | e*2]. self halt. ] asContext.
process := Process
           forContext: context
           priority: Processor userInterruptPriority.
session := DebugSession process: process context: context.

filter := StackFilter new.
filtered := #(#doItSelectorFilter #commonSelectorsFilter #kernelClassesFilter) collect: [ :symbol |  filter filter: session stack with: (filter perform: symbol)].

Which returns

an Array(an OrderedCollection() an OrderedCollection([ :e | e * 2 ] in UndefinedObject>>DoIt) an OrderedCollection())

This is, instead of folding, I just mapped filter application, and return an array of filtered stacks 🙂 Then we can decide if I want to intersect each one of those, or leave them separated. This would depend on how we want to plug in/out the filter from the debugger window (I mean,  the decision is closely related to how are the users going to activate/deactivate the filters, if they’re to be set as global options or in context menu, or both; etc).

Still, I am not sure if this is the way I want to combine the filters. I felt that I could still refine the idea, and Nico Passerini suggested me to change the filters so that instead of receiving the stack and returning a new, filtered one, they receive a message, and return a boolean… Something like


blocks := #(#doItSelectorFilter #commonSelectorsFilter #kernelClassesFilter) collect: [:symbol | filter perform: symbol]
filter filter: session stack with: [:message | filters allSatisfy: [:block | block value: message]

This way it’s simpler to combine filters using and/or, and then generate one resulting stack. Using and/or as combinators is a more powerful approach, but perhaps it’s not that common… So I chose to have a collection: enabling/disabling a filter is just a matter of adding/removing an element from this collection, and if you want combined filters, you could add a filter created by using and/or combinators.

And I am most content with the plot twist 🙂 Now I just have to refactor it again, and see what comes up of it.

I mentioned before that we have yet to decide how we want to expose the filters settings to the users. That is an important part of the discussion, and I think it deserves that I write another post to explain it 🙂

So, stay tuned!!

Cheers!

By clariallende

Debugger Model Overview

Hello there! In this post I will do my best to sum up the new debugger essentials. So let’s get to it!

In the old debugger all the actions and UI logic were all together. This makes it difficult to maintain and scale the debugger, and don’t even think of having new debuggers. So the new debugger came up to do some serious cleanup.

The intention is to separate the model from the UI, so any UI implementation, let’s say Glamour, Morphic, whatever, can be attached to the model, and use it. This is, the model does not know of how the stack is represented (a list, a table, a widget) , nor of the existence of the inspector, the selection handling, or the widgets that are being used.  The model only knows of processes and contexts.

So let’s see a little class diagram…

Debugger Model - New Page

The cool thing of this approach is that now you could take away everything above the dotted line (namely the UI components) and plug in any other implementation you may like. The default is one made in Spec.

Some considerations:

  • The SpecDebuggerProxy right now just adds the SpecDebugger button to the PreDebugWindow. That’s why it knows the old Debugger. This component should be removed once the new debugger is made the default for Pharo 3.0
  • A DebugSession has all the information of the current debugging process, this includes the interrupted process, the current context, and the possible actions for them.
  • A DebugContext right now just has helper methods to handle contexts. In the future, we should return a stack made of DebugContext, which would have more information (top context, active program counter, for instance) and more intelligence than a plain context.

So now about the model… A DebugSession is created for a process in a method context. The following code illustrates the creation of an interrupted process with a newly created context as top context:


context := [Set new] asContext

process:= Process
          forContext: context
          priority: Processor userInterruptPriority

And, to create a Debug session

session:= DebugSession process: process context: context

So now you have a new DebugSession, to which you can ask things like #stepOver or #stepInto to move between contexts and the stack.

And that’s pretty much it. Stay tuned in!

Cheers!

By clariallende

The Hammer of the Gods…

The hammer of the gods will drive our ships to new lands,
To fight the horde, singing and crying: Valhalla, I am coming!

Led Zeppelin – Immigrant song

Well there, welcome.

I am here to tell you a little bit of my experience during my Google Summer of Code project.

And what is it about? The not despicable process of improving Pharo‘s Debugger. There’s a lot to be done, but me and my mentors think that it’s a very important task, being the debugger a tool to introspect the system and which provides (or would be able to provide) so much information… our true Hammer of the Gods.

So, first things first: the actual state of the art of the debugger.

There are already some things done: the old debugger was (is) quite a mess,  there wasn’t a separation between de UI and the model. So Pharo guys started a new debugger, with a new and fresh UI, separated from the model. The idea is to provide a robust, smart debugger model that can be attached to any UI you can think of; being the default the one made in Spec.

And here is where I came in 🙂  The first step towards a better debugger is, as you might realize, to continue cleaning up the code. As a part of it, there are also some features the old debugger had that are still missing in the new one, so they have to be added. For instance, at the moment I’ve added tests, and I’m adding support for handling post-mortem contexts (contexts that have no longer a process attached).

In that regard, I had to get to know the internals of the debugger. It’s proving to be an interesting challenge for me to think of processes and contexts and getting my hands into them, since I’m much more used to work with other abstractions, closer to the “bussiness logic”. Also I have to get to be friends with Spec, since my little incursions into UI design in smalltalk have all been done in Morphic.

And then, once we get the new debugger as good as the current one, comes the fun(nier) part: calling Heimdall to open the Bifröst.

But that’s another post’s tale 🙂

By clariallende