How Long Will It Take You To Test…? – Part 2

“Do you have enough time to test?” 

Every time I hear this question, I can almost hear Admiral Ackbar yelling “It’s a Trap!”

As if we’re going to get more time if we need it.  Or, if the release date does actually slip to allow more testing time, inevitably scope creep comes into play and we’re back where we started; too many things to test and not enough time to test them.

“It depends” comes back into play as the “go to” response to the question.

“It depends, how much testing needs to get done?” 

This is a great opener to start the conversation on testing breadth versus depth.  Some managers or developer leads might start to get glassy-eyed as you ramble on about being able to execute dozens of test cases to ensure that one API is bullet proof but, in the time it takes to execute them, no time if left to test the 20 other APIs.  The other option is to take the same amount of time, and execute one test case against each of the 20 APIs.  This ensures that, in that one case, they all work as expected, but leaves a large risk as to how resilient the code actually is. 

Can you say that the code has been tested?

Sure!

But tested well?

Not really.

So, if you had more time, would you test more? 

I would hope so.

“It depends, is there something that doesn’t need to be tested so I can spend more time testing something else?”

Now, it’s the manager’s turn to hear Ackbar’s famous line from Return of the Jedi.  You’re asking for a pass; absolution if you will, on passing over chunks of code and functionality in favor of giving your time to some other features more deserving of your attention.  If a bug gets past you and it’s found in this “Manager Approved Pass Over Area” of the code, you can at least say “You told me not to test that.” 

Personally, I hate letting any bug get past testing and into Production, but it happens.  I’m sure to cover that topic in future postings. 

But overall, the conversation of “What are the risky areas in this build?” is one that all testers should have with their development counterparts.  It’s a conversation that I myself have almost daily. Seriously, just today, I asked this question to three different developers.  I got three different responses and cried a little inside, but the conversation was good!

Overall, letting the team know what was tested, how much testing was done (depth versus breadth),  and what was not tested or not tested as thoroughly due to time constraints, is highly recommended.  

How long will it take you to test….? - Part 1

Test Estimation can be taken as much of an art as it is a science.  In my younger, more wise-ass days (which some might say were, literally, just yesterday) I might have replied to my manager or the Dev lead; “How long will you give me?”

But I’d like to think that I’ve matured some.  That and the rolling eyes that I’ve received when I’ve given that response more than fifty times in the near past has caused me to come up with a way to give a better response.

But, honestly, what is a good response?

“It depends” is always a good one.  Of course, that response has met its fair share of rolling eyeballs as well.    

If you’re running a pre-arranged suite of tests and know that it takes about an hour to run them, you can say “About an hour.”  But what happens when your manager returns to your desk an hour later and you have no test results?  What happened?

Well, there’s always more to it than just running the test suite, right?  What makes a true estimate?  You’re going to need to make sure that you have a build first.  And that the build works and runs without crashing on startup.    And that the test suite that should take an hour to run, actually runs to completion.  You may have to run it a couple of times; restart it, make corrections, clean log files, etc.  And then, once it’s complete, you’re going to need a bit of time to look over the results.  Then, of course, you haven’t even begun to write, execute and add any new test cases and scripts to your suite to test anything that’s in this new build. 

So how long is that?

Stay away from saying “It will take as long as it takes.”  Your manager might roll his eyes as he or she walks away and finds a new tester while you’re left surfing Monster or CareerBuilder quickly before they cut off your Internet access. 

Your best bet might be to stay away from absolute times.  Rather, bring your Manager or Dev Lead in on the conversation.  Tell them that, once you’ve gotten a working build, you’ll run your suite and go over the results.  While you’re waiting on that build, you’re writing some new tests and will run them after the suite completes.  Give updates along the way.  Let them know when you’re hitting walls and road blocks.  Resetting the clock on testing time should be part of the conversation.