Testing Web Application

Recently I took part in our company "Tekriti White Paper Publishing Contest". There were many participants for various technologies and department. As a QA I have also taken part to this contest. Here I with my colleague Debajit Hajarika finally decided to write some content on testing methodologies and process for web based application. Finally we both work hard and conclude with a massive but useful documents and submitted it to the organizer.
Now here through my blog , I would like to share this with all of you. I hope this will really help you. I would like to say here that as testing is an art so there must be scope of improvement here with this doc. So your suggestion are most welcome.

1. Introduction

Testing process in any software organization is the key element to produce bug free, highly quality oriented software applications. This White Paper provides a great deal of information regrading testing process of web based applications.
Before we go ahead, we would like to provide an idea what Software Testing is all about.

Software Testing is an activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. It also ensures the working of the designed application as per specification. There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following rote procedure.

Mainly, we have targeted to cover the answers of the following four points:

1. Why we encounter bugs in software applications
2. What can be the best practice to prevent bugs.
3. What is the best way to get them managed.
4. What is the ideal testing process.

Software testing continues to grow more complex as the nature of web software application is getting more complicated and the vista of these application are becoming widen making it challenging for any tester to cope up with this process.

2. Why we encounter bugs in software applications:

Many of us encounter bugs in day to day life in the application that we use. Have we ever tried to give a thought why we encounter bugs in a software application. There are many factors playing a major role behind a buggy software. Most are man introduced and some are machine oriented.

If we elaborate the issue , we will have the following points:

2.1. Lack of proper communication: People associated with developing the software is not we aware of what an application should do and what it should not. In short, lack of requirements gathering.

2.2. Complexity of Software: Now a days software enters in to every domain of work. Complexity of current software system is so hard that it is very difficult to comprehend for anyone without the experience and knowledge of modern day software development. Right from Windows-type interfaces, client-server and distributed applications, data communications, enormous relational databases to sheer size of applications have all contributed to the exponential growth in software/system complexity. And the use of object-oriented techniques can complicate instead of simplify a project unless it is well-engineered.

2.3. Programing error committed by programmers: It is obvious that programmers can introduce syntax error or logical error without their knowledge, which may lead to disaster situation.

2.4. Rapid changes of requirements: Client can request any feature or changes in the mid of the development without knowing the impact of the changes. If there are many minor changes or any major changes, known and unknown dependencies among parts of the project are likely to interact and cause problems, and the complexity of keeping track of changes may result in errors. Redesign, rescheduling of engineers, effects on other projects, work already completed that may have to be redone or thrown out, hardware requirements that may cause unbalance in the developed system.

2.5. Dead Line and Time pressure: When unrealistic deadline is given, people rush in hurry without going in-depth of the product. Scheduling of software projects is difficult at best, often requiring a lot of guesswork. When deadlines predominate and the crunch comes, mistakes will be made.

2.6. Poorly Documented Code: It’s very tough to maintain and modify code that is badly written or poorly documented. That appears or results as a bug later after any changes. In many organizations management provides no incentive for programmers to document their code or write clear, understandable code. In fact, it’s usually the opposite: they get points mostly for quickly turning out code, and there’s job security if nobody else can understand it.

2.7. Lack of real Testing Environment: It is also important to consider the real environment where the software program will be implemented. It is very common that we can miss out those test conditions and scenarios to test for the lack of proper testing environment. This might be turned out as critical defect leakage at times.

2.8. Undefined Testing Process: Testing of a software system is not taken as seriously and carried out undocumented and undefined testing activities. Lack of well trained and experience QA professional also be the cause of buggy software.

3. What can be the best practice to prevent bugs:

We know that prevention is better than cure. This is the section in which we will discussion what can be done to prevent bugs before testers come to rescue. We had seen enough evidence in our company earlier that very few were interested in taking the initiative to prevent early mistakes. However, since the guide line in place, we witnessed a lot of improvement in the software developing stage.

3.1. Writing quality code: Though quality is a non tangible attribute, but programmer needs to ensure it right from the beginning.

3.2. Unit testing:
Programmer should debug the code using any Unit testing tool to insure the functionality of the developed module. They should not limit themselves to positive cases but also negative use cases.

3.3. Use Editor tools: Despite the availability of automatic error prevention tools for coding, it seems developers pay less interest in adopting such tool. Automatic error prevention tools are the good way to guarantee syntax error free code.

4. Testing Practices:

Before we go in top the details of software testing process, lets have a look on the different software product/application segment that is dealt by software companies. In general, the software industries can be segmented in various part based on nature of the project/product :

• New Product Development ( Product)

• New Application Development ( Project)

• Existing product undergoing major version release( New Release)

• Maintenance Projects

Based on the above software segments, it has the possibility that organization may adopt different testing process suitably to fit in to their projects.

Testing Process: Software Testing process is a very indispensable part of any software development process. If you are developing a web based application for your business, then the scope of testing challenges are getting increased. If the develop web site does not perform well under the stress of many customer visiting your site, you will lose business, customer and credibility. So, it's very essential to follow a software application testing process to deal with these issue and to measure the quality of it. A typical testing process starts from the initiation of Software Application. We are portraying the Software Testing Life Cycle with help of following diagram. This will let you know the different phases and its inter dependencies.

4.1. Requirement Gathering: Requirement gathering is an essential part of any project and project management. Understanding fully what a project will deliver is critical to its success. We want to be sure that the requirements specification contains all the requirements that are known about. While we know that there will be evolutionary changes and additions, we would like to restrict those changes to new requirements. We also consider the functional and non functional requirements for testing. As far as non functional requirements are concerned, it is the response by the system on a particular action. In other word, if system needs to respond to an event in 1 sec then we are talking about how the system must work; not what the system does. In this phase we refer to the following documents to collect the requirements details for testing.

• Requirement Specification documents
• Functional Specification documents
• Design Specification documents (use cases, etc)
• Use case Documents
• Test Trace-ability Matrix for identifying Test Coverage

On completion of this phase, we move out to start planning the testing process.

4.2.Test Plan: Test plan specifies process and scheduling of an application under test. QA Manager/Test lead prepares test plan document based on what to test, how to test, when to test, whom to test. It contains test plan id, one or two lines about project, estimated project testing start date, actual project testing start date, estimated project testing end date, actual project testing end date, selected test engineer names for the current project, training needs, schedule, use of automation tools, risks and mitigations, selecting test cases for regression testing. A Test Plan describe the following points:

• Introduction and Objectives

• Scope of Testing.

• Test items

• Features to be tested (who , what and when)

• Entry and Exit criteria

• Testing task

• Testing strategy and approach

• Assignment of Role and responsibilities

• Testing scheduling

• Risks and contingencies

• Staff and training needed

• Identify the areas for automation

• Automation Tool to use

• Test Deliverables

Basically, a Test Plan is systematic approach to testing a developed system. The plan typically contains a detailed understanding of what the eventual work flow will be. We draw plan to keep track of progress on the software that is being built.

4.3. Test Environment Setup: This is phase we prepare the Test bed. We need to set up the environment in terms of software/Tools and hardware according to the specification provided by client. This process involves with software and hardware installation required for testing. We also focus on network connectivity. Moreover, a good coordination is establish with vendors and maintenance departments for smooth running of testing activities.

4.4. Test Design: In this phase, we cover test design of software testing life cycle. It starts with in-depth coverage of test case design techniques for black box testing. This includes Equivalence Class Partitioning, Boundary Value Analysis, and Decision Tables. We emphasize on design activities and writing Test Design Specifications including documenting test conditions, test cases, test scripts and procedures and expected results. We pursue the following three stages in Test case developing which are described below:

• Develop test scenarios.

• Develop Test Cases.

• Review Test Cases.

• Writing/recording test script.

Developing test scenarios is crucial to successful implementation of test cases. On the basis of Test Scenario, we create test cases. Once we are completed with test case writing process, we send the test case document to Team Lead or higher level for review process.

As the test case writing is an art, and anyone can enhance the skill on writing test cases by the experience and practices. Keeping the following facts in mind, we can write effective test cases. 1. Always try to design the test cases in such a way to ensure the maximum coverage of

functionality with less number of test cases.

2. Read the specification carefully before developing test cases.

3. Focus on boundary value analysis and equivalent class partitioning techniques.

4. Make the test cases generic.

5. Try to make the test case as simple as that anyone can understand easily.

6. Classify the test cases according to the module/feature.

7. Identify those test cases which are required for Sanity Test of a build and make a

separate file for it.

Use specific test case template that covers all the necessary information which required in execution of test cases. Though every company has its own Test Case template, we have designed the following template to cover all the important information suitably.

In parallel, we always look out for the areas that can be automated so that manual testing effort can be reduced. This effort is generally made for functionality which are repeated in nature and stable. This automation scripts are written to perform sanity test and regression testing.

4.5. Test Execution and Defect Tracking: This phase begins with the execution of Test Design described in previous phase of STLC. This includes:

1. Execution of test cases manually.

2. Execution of automation script.

3. Update the test case status(pass/fail) after comparing the expected result with actual result.

4. Defect Reporting and Tracking.

5. Re-testing of defects fixes

6. Regression testing of modules.

7. Perform Sanity Testing on current build.

During this phase, we comply to the guideline that is set for us. Here

1. Execute the test cases which are meant for test to pass first.

2. Execute the test cases priority wise.

3. Execute the test case group wise which are interrelated.

4. Execute the test cases which are written to break the functionality.

Once we go through the test case execution process, there might be the case that some test cases will fail and some will pass. Now, it is of utmost important to track those fail test cases. This situation invokes the need of some defect management tools to track those fail test cases (bugs). To meet this need, we use defect tracking tools like Bugzilla, Mantis, Jira etc as per availability.

Now, you might think what happens once a bug is found. Well, once a bug is found in the system, it takes it own life cycle. A bugs goes through many stages before it get fixed. Here, we are trying to elaborate the Bug Life Cycle a bug goes through. We are depicting the Big Life Cycle with the following diagram:

Now, we are going to explore different stages of the Bug from it’s inception to closer. The life of bug starts from when it is logged and ends when it is fixed by developer, and verified & closed by Software Tester. Following are the status associated with a bug:

1- New:When a bug is found/discovered for the first time, we, the tester communicates it to Team Lead in order to confirm if that is a valid bug. After getting confirmation from the Test/Team Lead, we log the bug in defect tracker and status of the bug become ‘New’.
2- Assigned: Once a bugs is reported as “New” developers review it. If it is a valid bugs developers or team lead assigned the bug to appropriate person to fix it and the status of bug get changed to “Assigned”.
3- Resolved: Once a bug is assigned to developer, he/she make some changes in code and fix it, the state of bug becomes “Resolved”. Developers change the state to Resolved in bug tracker. In this Resolved state the bug might take various possible Resolutions which are listed below:
• Rejected: the bug is invalid as per the explanation from the development, he/she rejects the bug and marks its status as ‘Rejected’.
• Unable to reproduce: If developers can not produce the bug at their end the he/she can assign “Unable to reproduce” state to bug.
• Won't fix: Sometimes it may happens that improvement can not be fixed. In this state bug is said “Won't fix”.
• Deferred: If developers thinks that this bugs is not so important to fix and can be avoided then he/she can change the state of bug to “Deferred”. It means that the bugs will be taken care of at later release.
• Duplicate: It may happens that the same bug is filed again, then this bug can be resolved with the resolution “Duplicate”.
4- Closed: Once the bug state gets changed to Resolved, tester verified this bug to make sure that it is fixed actually. If the bug found resolved then tester close the bugs and the state becomes “Closed”.
5- Reopened: If after retesting the software for the bug opened, if the system behaves in the same way or same bug arises once again, then the tester reopens the bug and again sends it back to the developer marking its status as “Reopened”.

Many times we have encountered such a situation where developer is unable to reproduce a bug. This situation might lead to conflict between testers and developers. In this case, tester need to keep their head cool and communicate the bug effectively so that developer can understand the bug easily. We need to improve our skills in order to reproduce such issues. The following tips might work wonder in this confrontation:

• Try to capture the story around the issue and not just steps.

• Capture the complete Test Environment

• Capture all the available Test Execution Results. This includes your test data, screen shots, applications logs, system logs, server logs

• Need more patience (they won’t occur easily)

• Need more observation skills on the application behavior under test.

• Narrate the story around the issue to the stake holders and to corner the issue.

4.6. Test Report and Acceptance: This is the phase in which we, the tester, prepare test reports. The test report is the primary work deliverable from the testing phase. It passes around the information from the test execution phase that is needed for project managers, as well as the clients, to make further decisions. This reports is the summery of the test performed and the their results. The test report includes:

• Details of test cases(or test script) executed.

• Name of the software tester.

• The location of all test logs.

• Summaries of the results of each test non-conformity

• Summaries of the results of each test failure.

• The details of any code changes made during testing and the location of the modified

source code.

• Recommendations of any action to be taken as a result of the acceptance tests, where

applicable.

• Suggestion to the scope of improvement

Apart from that, one more important aspect is defects analysis. This is where we report defect information such as the number of defects found, defect severity classification, defect density, etc. We use Test Metrics to complement this section. Below are list of available metrics that can be used:

• Test defect by test phase -- Analyze test defects by test phase

• Test defect by severity level -- Analyze test defects by severity level

• Accepted vs. rejected test defects

• Defect density -- The number of defects per test cases

In general, writing the test report is important to make sure readers can make correct conclusions based on it. The output of this report will mold your readers' perception of you. The better your report, the better your reputation as tester.

Once we are done with Test Report, the User Acceptance Testing has taken place. It is the process that a client uses to verify that the delivered and installed product is ready to be put into production use. From the user's point of view, this means that every user oriented function should be thoroughly exercised. In this process what we will need to hand over to the team that will test your product is:

• User's Manual or Guide.

• An Installation Guide that tells the testing team how to install your product/application and who to contact in the event of problems.

• A link to your Project Plan which should describe the user requirements that our team committed to at the beginning of the project.

5. Conclusion: As the software testing is never ending process and its an art, obviously, we can not limit ourself to a defined process. There are lots of scope to venture and explore this field and software testing is becoming more challenging. We have written this document to share our knowledge that we have gain from experience and how we overcome the challenges we faced while testing. One thing is for sure that it is the tester who makes the life of user/customer easier. To become a good tester, one should have good understanding and listening power with an eye which can see the things from users point of view.

Before we sign off, we would like to thank our Director Mr. Manish Dhingra for his moral support and encouragement which make it possible to deploy this White Paper. Also we are thankful to all Teknokrats (Employee in Tekriti called as Teknokrat) for sharing their views and experience.

Besides, we can not ignore the author's articles to which we refer for help. Thank You. We are signing off.

Authors:

Vishal S. Sachan
Debajit Hazarika
Member of Quality Assurance Dept.
Tekriti Software Pvt Ltd, Gurgaon.
Date:19/05/2008