Titles and definitions

In an earlier post i talked a little about my thoughts on test manager and test team lead.

There is a underlying discussion here that I would like bring up.

Why is it that the career path for testers is usually to move on to become TM or similar?

And how come a TM or TTL usually gets better pay?

As I stated in the last post: to me a TM or TTL is a tester with some additional responsibilities and tasks. This usually mean that less time is left to actual testing, ie the value to the company/organisation might be less than for the regular testers within the team.

To me the most valuable person is the tester and the TTL is there to enable them to do their job.

So why is it that TM seems to be the way forward rather than becoming THE expert tester ?

Development have had a similar issue a few years ago when the path was: developer to dev team lead to project manager. Not sure if this still applies today. One can become a brilliant developer and get the recognition for it.

So why is it not the same in test?

Test as a profession might be a bit younger and has really come into its own in the last couple of years, while dev has a few decades to do the same.

It also might have something to do with the title, put manager there and the paygrade jumps.

But as said above a great tester will provide more value to the organization.

Maybe there is a need to better define Tester in order to better see the difference between A tester and THE tester.

De-focus and Re-focus

We have probably all been there one time or another where the meeting or discussion takes a turn away from the agenda and end up somewhere else and everyone loses interest.

This is what seems to have happened with the current ISO29119 debate. It started with a petition to stop the standard but is currently a yelling battle between a few persons about the personal agendas and personal reason for either creating or stopping the standard. What happened to the discussions on the facts in the standard and how this will affect the testing trade?

Its time to get back on track and focus on the fact at hand:

• Is there any validity for the standard?
• Can the standard be adopted to a non-waterfall project?
• Is it really that document heavy?
• ….and so on.

Bring these types of questions to light and help people understand what the standard will do or not do, and take the discussion for there. hopefully this will be a discussion that the “quiet masses” will have an interest in.

Sure there is no denying that there is a monetary aspect to this as well, but show me something today that do not have this. And there is a personal aspect as well, but the main focus should still be on the standard itself.

Don't let this discussion end up being another example of Godwin’s law  

What's in a Title

I have had a few assignments where my role has been Test Manager or Test Team Lead. 

Common titles, but what has the actual job description been behind these titles ?

A common description is usually: Manage a team to ensure that quality is delivered.

But what does this really mean ?

My take is that as a TM or TTL you are responsible for the team and the testing you do. Yes I mean you, not them, as to me the TM or TTL should also be involved in the testing as much as the team. If you end up just attending meetings and setting policies or work processes your title should probably be Test Strategist or even Project manager. If you are to lead a team one must know what is being tested and how, and this means getting your hands dirty and run tests alongside the team. Doing this will get you the information needed to lead the team, make good decisions and hopefully feel confident when reporting the status to the project.

For me one of the most important roles as TM/TTL is to shield the team from unnecessary discussions, especially politics, and allow them to do the work that is needed and what they excel at. This might mean that it is on you to explain why the team is working in a certain way. Or act as a mediator between the team and the project management when questions arise.

One important factor to remember when changing from different teams and/or assignments is that no two teams are the same. What worked last time might not work here. Different project methodology, different system, different people and skills etc.

This is where communication comes in. Listen to the team, discuss the expectations, talk about the way forward and so on. But DO NOT forget to have a similar discussion with the organization as well. What you and the team decides must be communicated so that managements expectations are also met.  

So what should the job description be, or contain?

Simple: Mediator - consolidate the requests from management with the needs and wishes of the team

Simpler: Gatekeeper - protect the team and let them work in peace.

Simplest:Team Member

Final comment:

Titles should not define you, it's the work you do that defines you.

Review of a review

Recently I was asked to review an article written by a colleague and afterwards it made me think about how I actually read and review text and how I approach test and testing.

The article itself was written as a discussion on the different approaches for a company to solve the webpage / mobile site issues. It was written for the general public and therefore did not contain massive amounts of technical details, rather focusing on the pros and cons of each solution.

My analysis of my own review (self-review?) is as follows, and can be split into three parts:

First i read through the article and just made some minor notes on where i got stuck some way or another. One read through, pretty much skimming the article

Next came the focused read, where I comment each part on a technical and understanding level. Do this make sense? is this actually correct? etc. Each time i got stuck or started questioning things i went back and reread the sentence or the whole chapter, and also checked statements on their validity. I also discussed some parts with the writer to make sure I understood what the aim or goal was.

Finally I once again started over and read the article as the person that the article was aimed at. So this time the focus was more on: Would I understand the text? Will this help me decide which solution to go for?

Once all this was done i cleaned up my notes and handed the article back to be updated. But the important thing here is that I made sure I was available to take questions regarding my comments.

So the conclusion ?

• For me to feel done with my review i read the article three times
• Different focus each time
• Checked and/or discussed questions with writer during review
• Active collaboration
• Written comments was also explained once the review was done
• Is my comments understood correctly?

Some notable observations are that not once did I actively check spelling, this due to the fact that I know another more suitable reviewer (teacher) would focus on the language. It took me three reads to feel done with the review. It was imperative to have the opportunity to discuss my comments directly with the writer to clarify questions and avoid misunderstandings.

How can this be translated to test then ?

In general terms my review are in three phases. First a quick smoke test, is this stable for the actual test? Secondly system test or even system integration test, do each part function as expected? Finally the acceptance test, is this what was actually ordered? The spell checking can be seen as the regression test, standard functionality that should work.

Another way of looking at it is the approach. First I read the requirements or User Story, do I have enough to start test? Secondly I execute the testing, focusing on different parts, digging into possible bugs etc. Lastly I rerun some of the test to get the big picture before finalizing my test report.

And as pointed out time and time again, communication is imperative. And this must be both with the business as well as the developers to avoid misunderstandings, and in the long run to avoid false negatives being reported as defects.

Trying something new and unknown is always good and will provide new insights, even if it just shows that you work in a similar manner regardless of situation.

Have you reviewed your work approach recently?

Give it 10 minutes and it might give you a big “oh-moment”

Analogies - Another way of reporting

During my last projects I have been using analogies when reporting status and during project team discussions. This started when the project I was working on as a test manager was running according to the tried and true waterfall model with many, many, many test cases. So why should I report the same number that everyone could find by a quick look in Jira? I did create a nice dashboard for this.

We had a discussion within the team and came up with the car analogy as a good starting point and something that everyone can relate to. The analogy took hold and shortly was the standard starting point for discussions or answer to questions.

Some examples of how this was used

• The body has been delivered but we are still missing the wheels
• Lacking deployments from supplier
• Brake pedal starts the wipers
• the integration's are faulty
• Again we have a SUV when its a compact we need
• Misunderstood requirements
• Supplier tests the airbag, buyer test drives the car
• Difference between System test and acceptance test
• Many more examples exist, and of course other analogies as well

We used this within the team but also when reporting to management as they also had full access to Jira and could get the actual numbers, so this gave them the quick status and also explained to those who did not have or needed the technical knowledge and full disclosure.

I have seen and used similar analogies in other projects and talked to others who have used this as well.

How do you report status? Discuss the latest progress? Or just talk shop within the team?

ISO29119 - And its possible future

Let’s rejoice!

Test and Quality Assurance can no longer be called a second hand job or something that just needs to pass, we have a ISO standard now to prove this.

Great news everyone! Or?

Pretty much every conference, blog and especially Twitter has had some sort of comment or information regarding ISO29119 these last months. This has ranged from analyzing the people involved in creating the standard to the content. It’s been a heated debate at some points and also a fairly one sided debate where the opponents have had the loudest voices, and the pro-ISO have been surprisingly quiet.

So what is actually going on?

I will give my views on things as far as I know right now given that information is still coming and the complete standard has not yet been released.

If we start off with the content in the standard, there is a slight problem right away. One must buy it from either ISO or IEEE, which is quite normal procedure in regards to standards. But once you have acquired it, it is yours and yours only, this according to the licensing agreement: “licensee's use only. No further reproduction or networking is permitted.” So how to discuss the content then? Below is my take on the standard.

Part 1

Part 1 is fairly straight forward it focuses on definitions and concepts of testing. If you have read or done the ISTQB Foundation course it will be a familiar read.

There are some interesting passages to note that sets the level for the coming parts.

It is recognized that software testing is a process and that the standard will present and describe a generic process. This process should be planned, monitored and controlled. The result of this process would be to provide information regarding the quality of the software under test and find issues as early as possible

A clarification might be in order here and that is that it is quite clear that the standard is based on the waterfall project model with sequential flow for the project and testing. Also the standard uses risk-based testing as the foundation for the dynamic test process.

Will cover the implications this might have later in the article, and the risk-based testing will be covered in Part 4 of the standard which is to be released late 2014

Part 2

This part of the standard covers the test process in more detail and its intended usage

Looking at the presented test process model, image below [1], it once again seems fairly standard, excuse my pun.

We have the organizational process that sets the overall setting for test across the organization. The Management process goes a bit deeper and focuses on how each project should handle testing and reporting. And finally the actual testing is covered by the dynamic process.

When looking at the overview of the process it feels simple and logical but once you continue reading the complexity increases, below is a diagram [1] on how the test plan process is described

Again at first glace this is normal work process for creating a test plan, i.e. gather and analyse information, present the analysis and get a sign off.  So no biggie then?

Well if you are to conform to the standard this is the way to go and as we are talking about a standard its the only way and you must follow all the steps regardless of the size of your project. And if changes are needed one is supposed to go through many of the steps to get the change approved.

Then comes the monitoring and control process and yet again things get complex to the point where it looks like the process will take more time than the actual testing.

As stated before the standard is focused on test cases and therefore this is described in detail, the writers acknowledges that writing test cases is a time consuming activity but also maintains that it may lead to savings over time.

The dynamic test process covers how to identify, create and review test cases, setup the environment and report incidents and much more

The complete process described is a lengthy one and involves many steps and iterations from identifying the requirements to creating and reviewing test cases and on to creating test procedure specifications before testing may start. Not to be left out is the process of reporting incidents and a closing report.

Part 3

This part of the standard covers the documentation that the processes described in Part 2 should have as output. As the processes is quite complex with multiple steps the documentation follows suite and there are a lot of documents to create. The fact that this part is 138 pages long should give you quite a good idea on the amount of document and the detail needed in each.

A detailed test policy and test strategy for the organization might be a good idea as these are normally only created when major changes happens within the organization.

The more you follow the process set in part 2 even more documents are needed or recommended. So for each test phase a new test plan should be created and then a separate test plan for each of the subsequent systems under test.

When we finally reach the actual test there are recommendations on what needs to be documented here as well, ranging from test design specifications to environment readiness reports and on to incident reports.

Much of this can and is handled by the test reporting tool one might use (QC, Jira etc.) but the standard still focuses on test cases and the process of creating, reviewing and executing these

Part three contains an appendix where examples for the documentation can be found and they have used both a waterfall project as well as an agile project, so the possibility is there to see how this can be applied to agile projects

So what will all this mean for test in general and me as a tester?

As I started the article it’s a good thing a standard has been created as this only further shows the importance of quality assurance. Today the quality is more important than time-to-market, as people are prepared to wait a bit longer for something that really works. Quality over quantity etc. and it is here that a test standard might fit.

Unfortunately I believe ISO29119 will not help our profession and especially not in its current state and with its claims. It states that this standard will help manage and perform software testing within ANY organization.

According to the standard there are different levels of conformity, either Full conformance which is as it sounds: one must follow the standard to the fullest. Or Tailored conformance which is where it’s up to the organization to justify where and why it has chosen to step away from the standard.

The tailored variant is there so that the standard can be adapted to a more agile work process. This to me is a bit of an oxymoron: create a standard that one can claim to follow as long as you justify why one has selected to not use or disregard the standard. Sure there must be some leeway even for a standard but not to the extent possible within ISO29119.

One concern regarding ISO29119 is that: Can one actually standardize an intellectual process? In my opinion this is quite difficult and will hamper the ability to actually test. this given the fact that each project and system is different, and needs different approaches depending on the situation. Now each tester and test manager must conform to a set standard and practice regardless of situation and system under test.

Another concern that I see is on a more organizational level which the standard also affects given the way it is written, with the major focus and structure on waterfall and test cases. This means that for an organization to implement and adopt the standard major organizational changes must be made.

The last concern is also the one that most likely will kill this standard as companies will not be willing to take the cost and risk needed to change and adopt to it. The unfortunate backside of this might be that it will leave a bitter aftertaste for those who has read it and then discarded it.

The standard is written and presented as yet another attempt to define a process and way-of-working for test. Had it just been another book or white paper this would have been read and in some cases used as well. But now as it is presented as a Standard with the backing of ISO/IEEE it puts things in a whole new light. Suddenly things are set in stone (maybe not, given the Tailored compliance) and this is how testing should be done. This is also what will be the standards downfall. Few or possibly no organization will be willing to take the time and cost to change its current setup to conform to the standard. The risk that I see is that new organizations or possibly government run departments will adopt this, but this will be a minor risk.

As an example just look at the ISO12207 - Systems and software engineering — Software life cycle processes this has been around since 1995 with the latest revision in 2008. In the years i have been working with SW development, support and test i have never seen or heard about this before. And that what i believe will happen to ISO29119 as well. Unfortunately before it vanishes into the annals of civilization it will give a bitter taste and give quality assurance a bad and backward striving impression.

Ref

[1] http://softwaretestingstandard.org/part2.php