Reflections on BBST Foundations Workbook – Chapter/Lesson 1

(This post is from my previous blog, but introduces another blog series that I will be working on around the BBST Foundations book – having blog series help provide some framework to keep me blogging and generating additional ideas)
I just finished the first lesson/chapter in the BBST Foundations book (and more information/background about the book can be found here – I enjoyed it, especially the reflections at the end. Given that the project (BBST) has been around for a while, there were some questions I had while taking the course that I found the reflections help to clear up, like the reference to the AST dictionary project.  I also found the Testing Circus – February 2014 edition (BBST special) to provide some great background about the courses and their evolution. It really helped me understand why I ran into some of the issues in the AST Online courses that I did and the current connection (or lack of connection beyond use of materials) between Dr. Kaner and the AST BBST courses.
As far as the content of the first lesson goes, I found the treatment of the levels of testing to be valuable. It seems pretty fundamental I know, but I can get locked into a particular way of testing from time to time and this was a reminder that there are multiple levels at which an application can (and probably should) be tested, at times with different desired outcomes for those types of testing. One of the definitions that I think could have been edited and improved was around acceptance testing. I believe I understand where the content is coming from in relation to acceptance testing (customer-focused and executed vs. some agreed upon definition or set of tests that can be executed by a test team before the new software is “accepted), however the wide-spread adoption of agile processes, acceptance criteria for user stories and subsequent tests defined to help prove out that acceptance criteria are things that many (I believe most) will run into. I believe that this definition would benefit from some extra treatment that address this.
The issues related to quizzes and instruction were familiar to me (I used to teach in the K-12 space and have a degree in education) and I generally agree with what is explained about quizzes and using them to drive additional learning instead of merely assessment of a user’s progress in the course. Finally, the treatment of the concept of quality is very valuable. I believe many people generally consider that one of the core mission of testing activities are to help contribute to better quality in software, so understanding what quality means is important, as is the realization that the concept of quality might mean something different for someone else (like your manager or the person for whom we are making the software).

Webinar Review: Tips for Writing Better Charters for Exploratory Testing Sessions

by Michael D. Kelly

Webinar URL:
Webinar Summary
Michael’s webinar focuses on one of the more challenging parts of charter-based exploratory testing – writing effective charters to help drive the exploratory testing process. He defines exploratory testing as simultaneous learning, test design and test execution (a James Bach original I believe) and focuses his ideas within the Session-based Test Management (SBTM) framework. He scaffolds this idea with an outline of the elements of SBTM along with some explanation about each element to help provide some context for those who are unfamiliar with SBTM. Here the following elements he includes as a part of SBTM:
     -session notes
     -team prioritization – talk about risks
     -ad-hoc test documentation – document what comes up as useful to be documented
     -ad-hoc test automation – automate what you encounter as useful to automate
     -dynamic metrics and reporting
Michael goes on to talk about the three essential elements of a charter: Defining Risk, Coverage and Timeframe. The idea being that a charter should be written in a way so that a tester can understand the why I am testing this (Risk), the what I am testing (Coverage) and how long it needs to be tested for.
The remaining part of the webinar covers Michael’s ten suggestions for creating better charters:
List specific risk and coverage targets
Mnemonics for risk and coverage ideas
Risk and Coverage Knowledge Base
Compare missions (i.e. charters)
Charter Template
Charter for smaller sessions – then possibly use affinity maps to create larger sessions
Thumb Vote
Testing Polarities
Let Charter Emerge Over Time
Tracking Metrics
The Q&A session was also helpful in several areas, especially in talking further about metrics and session debriefs.
My Take
The webinar is first-rate, full of concrete ideas that are abstracted from specific practices enough so as to be easy to apply or adapt in a variety of work environments or teams. In many ways I feel like these are details that I was missing to help guide my exploratory testing efforts to be more effective. Michael covers his ten suggestions with good detail and I found most of the ideas to very helpful, even though some of the ideas would require a team to be able to implement (since they involve ways of comparing or filtering through charters to help determine the priority or acceptability of charters). Probably my favorite suggestion for creating better charters was the template idea that Michael shared for beginning to write effective charters. As with any template, in sharing the idea one runs the risk of it being considered “the” way to write a charter, which is obviously not the case here, there are many ways to structure or write a charter. For a tester who is a novice in charter-based exploratory testing, this structure can be really beneficial in jump-starting your use of charters and exploratory testing. As my English teachers liked to tell me, “Once you learn the rule well enough to never break it, you can have the freedom to break it whenever is needed.” That’s kind of how I feel about his template idea.
As a tester who wants to improve my skills, I found Michael’s discussion around debriefing sessions and metrics to be helpful as well. The metrics may be different than some of the traditional software testing metrics, but they are robust enough to help tell the “story” about the testing that you are doing, though I can imagine it takes some time to tell that story well and help the stakeholders involved to feel comfortable with them.
In summary, I cannot say enough good things about the webinar. It was simply full of practical ideas and useful explanations to help anyone improve the charters they write to help drive their exploratory testing efforts.
My Rating
Well Worth Watching

Webinar Review: Co-Leadership – How being a leader is within your reach

By Bram Bronneberg

Part of my webinar review series on the EuroSTAR webinars

Webinar URL

Webinar Summary
Bram starts off by defining Co-leadership as shared leadership, horizontal leadership, collective leadership or distributed leadership. The concept here is that we all have a leadership role, even if it isn’t a formal role. He goes on to define what he saw as the three main responsibilities of a leader:
  • Inspire
  • Give Direction
  • Incite Change
He then proceeds to use Bob the tester, a fictional person, and follows his story through starting out as a new tester, who then experiences some challenges with his colleagues and falls “down the mountain”, with the mountain being a metaphor for one’s outlook or confidence in his team and his abilities. Bram defines four phases of this move down the mountain with confident and contributing being the attributes of those atop the mountain, moving downward one finds doubt, fault-finding followed by lack of caring as the element that characterizes being at the bottom of the mountain.
The final section of the webinar discuss Bob the tester’s experience and considers it from two perspectives. How Bob can avoid moving down the mountain and the responsibility that Bob (and other team members) have for helping other members of the team move up the mountain. He also makes an important point that those who help the team succeed, those who assist with the leading, cannot effectively do it from the bottom of the mountain.
Some of the steps or suggestions he provides including “patting people on the back” (i.e. complimenting or encouraging them), providing feedback when they look for it and confronting them when they have given up (he even mentioned insulting them to motivate them if needed, though I disagree with the characterization of insulting and would use “challenge” them instead). These specific ideas coincide with the big ideas of what leaders do (inspire, give direction, incite change).
He closed his portion of the webinar presentation (there was some Q&A afterwards) by highlighting some additional points for co-leadership to work among teams:
  • Know where you and your colleagues are on the mountain
  • Help each other back to the top
  • Step up to the plate personally
My Take
The concept of co-leadership was an interesting one to me and I was intrigued by the story he was going to use as the backdrop of his presentation (Bob the new tester) because I felt like it might pertain to my situation. Unfortunately, I found that using a fictional representation to drive the narrative and main ideas behind the webinar made it feel too theoretical. There were not enough real experiences to allow me to connect with and understand how his ideas occur or are implemented in “the real world”.
I still find the idea of shared leadership/responsibility to be valuable and the idea feels like it should be very applicable to agile teams especially. The part of this co-leadership idea that resonated most with me was the concept that I have a responsibility to know where I and my colleagues are on this “mountain” he speaks of. I think this idea (this responsibility really) is one that can be useful to me moving, however the rest on the webinar just did not have enough concrete examples to be as valuable as it could have been.
I have also decided to use the following scale to provide a final rating/summary of the webinar. The scale is composed of the following options: Well Worth Watching, Take it or Leave it, and Skip It. Well Worth Watching means I think a tester should definitely view the webinar. Take it or Leave it means that it might have some ideas you would find compelling, but I’m on the fence about it. Skip It means I think you can get more value out of your learning time by viewing another webinar.

My Rating: 
Skip It

EuroSTAR Webinar Review Series

(This was a blog entry from my previous software testing blog that I have migrated here. I’m going to be bringing that content over a little at a time but I am also going to use the idea in my original blog post and pick up the webinar review series to complement my blog series on the software testing learning mindmap. In that spirit, I have slightly edited the post from its original)

I look for learning opportunities wherever I can get them and I’m generally of the opinion that we can learn from a diverse group of people, including those we may not always agree with. That is one of the reasons I’m looking forward to doing a review series on the webinars available on EuroSTAR’s website as a series of blog posts over the next couple of months. They have 74 listed on the site, dating back to 2008, and seem to come in bunches throughout the year, with an occasional large gap of time between them. At first glance, they look to be a valuable resource for a software tester’s learning plan and cover a wider array of topics, ranging from process, technique, strategy, documentation, management and more. I’ve perused the list and culled it down, somewhat, to the 45 I’m going to highlight in my webinar review series.

The impetus behind this project was really Erik Brickarp‘s session at CAST 2013 (see his blog here) about his experience in developing as a tester over the past year. One of my big takeaways from his session was the need to have some concrete goals that I am committed to in order to focus and create momentum for my learning and improvement process as a tester. I’m still formulating some additional goals for the upcoming year and thinking about how to incorporate another of my takeaways from Erik’s session, which is to step outside of your comfort zone to accelerate the learning and improvement process, but this is my first step. This goes hand-in-hand with the goal of starting a testing blog I have had since I entered the software testing profession just under 2 years ago. I found my experience in the education profession enriched greatly by my blogging efforts and blogging regularly helped to encourage my passion for the profession, as well as helping me to articulate and reinforce the learning and ideas I had. I felt like blogging about my software testing experience would benefit similarly, but haven’t really been able to get going with it. I realized as I encountered the EuroSTAR webinar series and attended Erik’s session at CAST (as well as reading Michael Larsen‘s on-going blog series based on the 99 Things You Can Do to be a Better Tester, for which he needs a blog post that is a table of contents for the series), that this combination of things could be the motivation and structure I needed to get blogging again and make the learning experience from the EuroSTAR webinars more meaningful. So with that backstory, you can always view the posts in the series by clicking on the category to the right.

Session-based Test Management using Evernote

(This is a post from my previous testing blog that had a short run. It was published on Aug 21, 2013 originally, but I wanted to highlight some of my original blog content as a possible contrast even to some of my current thinking)

I find using Session-based Test Management (SBTM) principles to help focus my testing work to be helpful. It is a helpful way to organize my testing activities and produce a testing artifact that can explain how and what I was testing. I don’t do context-driven testing or exploratory testing particularly well I believe, but it is a skill I am working on developing. As with anything, to really improve at something, we need to do it thoughtfully and reflectively, and I find that SBTM helps me to do that. I have always found creating and storing the session sheets to be a management chore and something that got in the way of my eagerness to embrace this as a method for organizing my testing activities. I have tried several different tools, most of them found from Paul Carvalho’s excellent page about Lessons Learned in Session-based Test Management, and mostly used the original tool developed by James and Jon Bach. In the back of my mind, however, has always been this lingering idea that implementing the session sheets in Evernote made a lot of sense. One of the limitations in Evernote for doing this (and other cool things) has always been the way it handles (or really does not handle) templates (i.e. notes with consistent data needed in new notes).

Recently, I decided to get serious about solving the problem of templating in Evernote, in hope that it would reduce some of the management overhead of duplicating session sheets, remembering the date formatting on the note filename, what notebook to store it in so it could be separated by project, and other default values that need to be included in the text of the session document. My work in this case was aided greatly by Thought Asylum‘s posts about templating in Evernote. I found the following posts to be invaluable in setting up a template system in Evernote that worked for me:

I found the first three to be very helpful in getting my batch script setup. While the daily journal post might appear a little out of place, it shows how to dynamically create a template file that can be imported into Evernote. This is helpful for inserting a custom title (one that has the date in it for example), as well as date/time specific parts of the SBTM session sheet. The other evernote articles just helped me work through some batch scripting (Enscript is the command line interface into Evernote on Windows at least), as well as some of the querying needed to trigger a search in Evernote once the import is completed so that the newly created note from the template is selected and shows up in the Evernote application without requiring the user to go find it and select it. The PS menu third party add-on mentioned is also helpful for exposing those batch scripts in the system tray in a quick and easy way to access them.

Here is the batch script that I ended up with for a session sheet template. I have added some comments to the script in bold to try and explain what each section is doing. Once the script below is saved as a batch file, it can just be clicked on and ran and functions properly (provided you have update the sections that set the location for where some files are or where they will be created). You can follow the steps at one of the Thought Asylum blog posts for getting up and running with PS Menu. The end result of all of this is that it has facilitated creating session sheets at the click of a button, which has significant reduced some of the overhead for recording my testing activities in an organized way. While I have plenty of room for improvement (identifying someone to “debrief” with, improving my identification of appropriate charters, notetaking), I’m able to embed relevant files in the session sheet as needed. The next big (and I mean BIG) project would be to look at what is required to parse the session sheets inside of of a particular notebook to generate some of the TBS metrics that can help me to evaluate what I am spending my time on. The nice thing about keeping the session sheets in Evernote is that I get great support for search and sharing of notebooks if I can ever expand the SBTM practices to other testers in my group.

Batch Script


REM Calls the GET_TIMESTAMP function at the bottom of the script, which generates some variables needed as the script progresses, when the function is finished it returns to the line below the function call
set QueryString=created:%TimeStamp%

REM Sets the location of different tools (like ENScript.exe) and my evernote database, as well as the location of text files that contain some elements of the evernote template (which is an xml based document that Evernote uses to store its notes in)
set ENscriptLocation="C:Program Files (x86)EvernoteEvernoteENScript.exe"
set "C:UsershintbwAppDataLocalEvernoteEvernoteDatabaseshintbw.exb"
set FilePart1="D:Skydrive - BrettSkyDriveDocumentsEvernote Templatessession-template-part-1.txt"
set FilePart2="D:Skydrive - BrettSkyDriveDocumentsEvernote Templatessession-template-part-2.txt"
set FilePart3="D:Skydrive - BrettSkyDriveDocumentsEvernote Templatessession-template-part-3.txt"

REM Sets the location for the temporary .enex file that is created for import into Evernote
set WorkingFile="D:Skydrive - BrettSkyDriveDocumentsEvernote Templatestemplate1.enex"

REM Sets the default notebook to import the template into, some alternatives to the script can include passing in a parameter when the batch script is called that would override this variable
set DefaultNotebook="Sessions-ProjectName"

REM Build the template file by inserting each part of the XML document. Note the two echo statements which are inserting the custom date data that I needed in different sections of my session sheet
type %FilePart1% > %WorkingFile%
echo %NoteTitle% >> %WorkingFile%
type %FilePart2% >> %WorkingFile%
echo %StartTime% >> %WorkingFile%
type %FilePart3% >> %WorkingFile%

REM Sets the query string that will be used to cause Evernote to display the note after it is created from the template
set QueryString=notebook:%DefaultNotebook% %QueryString%

REM This imports a new note based on the template file created and places it in the default notebook that was specified
%ENscriptLocation% importNotes /s %WorkingFile% /n %DefaultNotebook% /d %EvernoteDatabaseLocation%

REM This command causes the Evernote application to show any notes created after this batch script is run, which generally will be just the note that you created above
%ENscriptLocation% showNotes /q "%QueryString%" /d %EvernoteDatabaseLocation%

REM This section deletes any of the variables that were set, including, most importantly, the temporary working file that had been created on the file system
del %WorkingFile%
set ENscriptLocation=
set EvernoteDatabaseLocation=
set JournalNotebook=
set FilePart1=
set FilePart2=
set FilePart3=
set WorkingFile=
goto :EOF

REM This section sets time and date values I need to build titles and add date related information in my template
for /f "tokens=1,2,3 delims=:" %%a in ("%time%") do ^
set hour=%%a& ^
set minute=%%b& ^
set second=%%c

for /f "tokens=1,2 delims=." %%a in ("%second%") do ^
set second=%%a& ^
set remainder=%%b

REM This section may not be needed for some locales, but it eliminates the weekday information on the front of the date information that was messing up the query to show the note in Evernote after importing the template
for /f "tokens=1,2 delims= " %%a in ("%date%") do ^
set weekday=%%a& ^
set usabledate=%%b for /f "tokens=1,2,3 delims=/" %%a in ("%usabledate%") do ^
set day=%%b& ^
set month=%%a& ^

set year=%%c

set NoteTitle="<title>et-bwh-%year%%month%%day%-</title>":
set TimeStamp=%year%%Month%%day%T%hour%%minute%%second%
set StartTime=%month%/%day%/%year% %hour%:%minute%
goto :EOF

My Software Testing Origin Story

This is admittedly a glorified About page blog post (and will probably actually function as my About page I think) but it is the first post in the Software Testing Learning Mindmap series and covers the Intro section on Dan Ashby’s mindmap.

Story of Becoming A Tester

I used the graphic above in a career presentation I gave to my son’s middle school (6th – 8th grade students) because I think it can aptly describe a career path that many take to become a software tester. While a degree in computer science or information systems is still a pretty common (perhaps most common) pathway, at my current employer 3 of the 5 testers have come into the profession from a pathway that doesn’t involve one of those degrees.

I graduated from college in History Teaching and began teaching middle school history in Arizona, USA. I spent 3 years me in the classroom (two of those as the computers teacher) including building a PHP application I ran from one of the Macs in my computer lab to help teachers with one of the hourly processes we had to do. After that I moved through educational technology positions at the district and state level before grant funding eliminated my role at the state department of education. At that point, I decided it was time to consider a career path change. I had done some website development and consulting throughout those ten years and began looking for an opportunity that would allow me to jump into software development full-time. A QA job with an organization I admired came available and so I applied and started my first job as a software tester.

Workplace Experiences

My first software testing job provided a great opportunity to immerse myself in software development and the software testing discipline. I jumped into learning all I could (and I had – and still have – a lot to learn), including with the support of my employer attending a conference each year (STARWest, CAST, and STPCon) as well as taking all of the BBST courses offered by the Association for Software Testing as well as a great (and extremely hard) Domain Testing class by Rebecca Fieldler and Dr. Cem Kaner. I became familiar with the different “schools” of testing and felt like much of my learning and experience aligned me more with the Context-Driven School of testing. The two teams of developers I supported operated semi-Agile’ly’ and we weren’t doing much in the way of unit or integration testing, so I did much of my work as manual testing (yes I realize the “controversy” around that term) of our applications. I dabbled in Coded UI tests since our technology was .NET thick clients, but found them brittle and difficult to develop for our complex apps. I managed much of my work using session-based test management exploratory testing (more on that later) though I wished we had more testing layers so I could focus on more detailed and complex user scenarios than the basic functionality I was “checking”.

After about three years, I began looking for some more opportunities to learn and grow and joined a healthcare IT/data startup. Over the last nearly two years, I have wrestled with some data warehouse testing issues, web testing as well as continuing to test thick client apps. I have had opportunities to explore the value of layered testing from unit on up through acceptance tests and how to mitigate risks and gather information by testing at the layer closest to where the risk of failure exists. I have also begun learning about testing in a continuous delivery/deployment release cycle and the challenges that presents (regression testing in any way but automated is impractical) and the role that exploratory testing plays in that process. We are currently very much in a DevOps-type of transition and I’m loving the new testing challenges and opportunities that it is bringing, especially in spreading effective testing across teams/roles so we can confidently deploy our products at any time (at least that’s the idea).

I know that is more background than anything, but these are the experiences and pathway that has brought me where I am today in testing and it’s been a fun journey that has challenged me (and continues to do so). Next post is about some more meaty stuff, What is testing? – a topic which can include some “fighting” words depending on who you are talking to and one that certainly reveals a lot about someone involved in software development.

Software Testing Learning MindMap

Wow, this week flew by! I’ve had this post on the back-burner since late last week but just haven’t been able to get it out the door. I attended a webinar hosted by the Software Testing Club recently called How I Interview Testers that was given by Dan Ashby. It was excellent and you can view the webinar at the Dojo or view some of the questions Dan answered after the webinar (including one that I asked about interviewing testers completely new to the profession – yay!). (As a sidenote, the Ministry of Testing is pretty awesome and I love the different things they have going on – someday I hope to attend a TestBash, probably Philadelphia because it is on “this side of the pond”, but one in Manchester or Brighton would be even more cool).

Back to Dan’s webinar. He posted a mind map (see the image at the top of this article – credit is all his – check out the links above for more info on the webinar) that showed how his interviews tended to flow and the types of topics that he would ask a candidate about,as well as how he grouped them together. It looks like a great guide for interviewing testers, but I think it could also help guide a tester in identifying different topics to learn about and incorporate into their own testing efforts.

The map just resonated with me as a great place to start talking about testing. As a result I decided to use it for a series of blog posts to kick things off here at Technophile Tester and it give me an opportunity to share my thinking about those topics (and have my thoughts challenged and refined so I can improve them 🙂 ). Some of Dan’s sections obviously allow for some deeper exploration and more blog posts than others, but I’m looking forward to diving into discussing testing. So please consider this post as the table of contents to my first blog series (ever) – What Do You Know About Software Testing? (WDYKAST). Please see the category link to the right to get a current list of the posts in the series.