Methods for Bug Discovery

There are two primary methods for discovering bugs and. The first method is natural testing, and the second is systematic testing.

Be Natural

There are two perspectives we usually test from. The customer and the back end user. Both of these perspectives need to be tested extensively so we can be sure orders can be placed and all normal functions performed. So when you start testing, pick a perspective and test from it. Don’t meander around, aimlessly pushing buttons and links. While you may find bugs this way, it’s generally a waste of your time. We need to target the important processes and move down from there.

If you’re operating from the customer perspective, try to find a specific product that you want to look at, try to find the information you want, try to add to cart, checkout, etc. If you’re operating from the back end user perspective, try to do some of the same things but from the back end. Try crediting an order, etc. It’s imperative to focus on things that have to be done constantly so that we get the most important processes fixed and perfect first. After you’re sure a process is steady, move on to something less significant. The key is to operate how you’d operate if you weren’t testing, but using.

Systematic Testing

For more strenuous and detailed testing, the systematic approach is employed. When using this method, the tester will pick one function or a group of similar functions and make sure they work properly in every location across the site. This method can be tedious, but it is vital to finding rarely occurring bugs that might only affect one product or page. Systematic testing is important, but takes a lower priority than Natural testing.

Principles for Bug Discovery

Pay Attention

When you’re running processes, pay attention to what you’re doing. Try not to get into a rhythm but to test each component individually. Many times testers run a process and expect it to work because they’re in a rhythm and every similar process up to that point has worked. Don’t make this mistake. If you expect everything to be broken, you won’t miss anything. And you’ll be pleasantly surprised every time something works.

Think Stupid

Yes, think stupid. You may be very familiar with the system and feel fairly confident using it. In the case of testing from the back-end user perspective, this can be a strength. In the case of testing from the customer perspective, it’s a weakness. Since the site will be brand new (or newly tweaked), customers will not know how to use it. That means they’re not going to know how to do things you do. It also means they will not know not to do something you recognize as being “dumb.” Think less in terms of intelligence and more in terms of ignorance. When testing from the customer perspective try to behave as if you know nothing about the site.

Keep Track

As you’re testing various things, keep a mental track of what you’ve done up to that point. When/if something breaks, you’ll need to know how you did it so that you can reproduce it. The easiest way to accomplish this is to simply be aware of what you’re doing. Don’t try to multi-task while beta testing. Focus on it and pay attention to what you’re doing and what you’ve done. That way when something goes wrong, you will probably already know what caused it.

Reproduce

When you find an issue. Don’t just report it. Make sure you saw it right. And make sure you can reproduce it. Submitting an issue that isn’t really an issue is a waste of everyone’s time. Retrace your steps and find out exactly caused the error/malfunction to occur. Obviously, in the case of cosmetic issues, this won’t be as much of a factor. But in functional/system critical errors, it is absolutely essential. There could be many different reasons why something broke. Many times it is a combination of several things that causes bugs. So don’t assume. Narrow down the reason as much as possible by trying different methods to produce the same result. Chances are you won’t narrow it enough, but that’s okay. Whoever is making the correction will find out exactly why it’s breaking, but by narrowing it down, you make his job a lot easier so he can focus on fixing, not finding.

Reproducing the problem is key to the next step, documentation.