Documenting Exploratory Testing

Exploratory testing is important, whether it’s a new feature that we’re introducing or a bug fix that addresses a long standing issue. Doing a deep dive into our product to find any issues (new or longstanding) is a rewarding (and sometimes humbling) experience. It can be interesting to just take a fresh approach from time to time, and go through some of the happy paths our customers experience.

One of the key parts of exploratory testing is having a way to document your findings: all of the bugs, questions, notes and ideas that hit you while you’re clicking around the app. If you find a bug, but didn’t keep track of how you got to where you are, how useful is that to anyone? Isn’t it better to jot down an idea as you think of it?

I spent some time a short while ago searching for the *perfect tool* to record my exploratory testing sessions. It would be great if there was something out there that you could start and stop, would record what you’re doing in your browser, and allow you to annotate those activities with notes and observations.

I couldn’t find a tool that fit the bill perfectly. You can accomplish all of that manually by running a screen recording and keeping a set of notes. Just note the timecode on the recording when you’re making a note. That’s a bit cumbersome though.

Instead, I compromised somewhat and found a nice Chrome Extension that will allow me to jot down notes and bugs, and take screenshots as I go.

exploratory testing extension in action
Exploratory Testing Extension

When you’re finished testing, you can export a report of your testing session to CSV, JSON or view it in HTML (which you can then print to a PDF). It’s come in handy a couple of times in sharing my results back with the product team.

Mind Maps

One of my goals for the year was to slow down and think about/plan out the testing that I’m doing ahead of time. I’ve found that I can get too involved in the details of a task and miss seeing where the task might touch other areas. Asking questions like “will this change affect the API?”, “will this change impact the apps?” or even just “what are all of the areas that accept text input?” can come in handy.

I’m not a huge fan of generating a lot of documentation that will never get read again, but I am a visual thinker — so in order to flesh out ideas I need to write things down. In order to strike a balance here, I’ve started taking a few minutes before starting testing to sit down and put together a quick mind map of what might need testing.

My process for this is pretty simple. I’ll start clicking around that area in the app and brainstorming all of the parts that might be touched by this particular change:

mind map grouping of invoice form updates

I use an app called OmniGraffle. I’ll start dragging out squares for anything and everything. If there’s any possibility that it’s touched by this work, it gets a box. I’ll try to be as specific as possible at this point, but I often just use a shorthand to describe something (it’s for my own reference, so if I understand it, good enough!)

The next step is to group related bits together. It could be a grouping by function, or by location in the app — again, this is just for my own purposes to try to help give me some structure to my thinking. I’ll often notice other things while grouping and add them in (or break down bigger ideas).

Then it’s time to get going. If it’s an office-hours type of task, there may or may not be test cases. If there aren’t, and it’s a bit more complex feature, I may translate the mind map into a checklist so that I can keep track of where I am in testing things.