Test Strategy & Test Plan

Test Strategy
A Test Strategy document is a high level document and normally developed by project manager. This document defines “Testing Approach” to achieve testing objectives. The Test Strategy is normally derived from the Business Requirement Specification document.

The Test Stategy document is a static document meaning that it is not updated too often. It sets the standards for testing processes and activities and other documents such as the Test Plan draws its contents from those standards set in the Test Strategy Document.

Some companies include the “Test Approach” or “Strategy” inside the Test Plan, which is fine and it is usually the case for small projects. However, for larger projects, there is one Test Strategy document and different number of Test Plans for each phase or level of testing.

Components of the Test Strategy document

1. Scope and Objectives
2. Business issues
3. Roles and responsibilities
4. Communication and status reporting
5. Test deliverability
6. Industry standards to follow
7. Test automation and tools
8. Testing measurements and metrices
9. Risks and mitigation
10. Defect reporting and tracking
11. Change and configuration management
12. Training plan

Test Plan
The Test Plan document on the other hand, is derived from the Product Description, Software Requirement Specification SRS, or Use Case Documents.
The Test Plan document is usually prepared by the Test Lead or Test Manager and the focus of the document is to describe what to test, how to test, when to test and who will do what test.

It is not uncommon to have one Master Test Plan which is a common document for the test phases and each test phase have their own Test Plan documents.

There is much debate, as to whether the Test Plan document should also be a static document like the Test Strategy document mentioned above or should it be updated every often to reflect changes according to the direction of the project and activities.
My own personal view is that when a testing phase starts and the Test Manager is “controlling” the activities, the test plan should be updated to reflect any deviation from the original plan. After all, Planning and Control are continuous activities in the formal test process.

1. Test Plan id
2. Introduction
3. Test items
4. Features to be tested
5. Features not to be tested
6. Test techniques
7. Testing tasks
8. Suspension criteria
9. Features pass or fail criteria
10. Test environment (Entry criteria, Exit criteria)
11. Test delivarables
12. Staff and training needs
13. Responsibilities
14. Schedule

This is a standard approach to prepare test plan and test strategy documents, but things can vary company-to-company

Software Verification & Validation Model - An Introduction

A perfect software product is built when every step is taken with full consideration that ‘A right product is developed in a right manner’. ‘Software Verification & Validation’ is one such model, which helps the system designers and test engineers to confirm that a right product is build right way throughout the development process and improve the quality of the software product.

‘Verification & Validation Model’ makes it sure that, certain rules are followed at the time of development of a software product and also makes it sure that the product that is developed fulfills the required specifications. This reduces the risk associated with any software project up to certain level by helping in detection and correction of errors and mistakes, which are unknowingly done during the development process.

What is Verification?
The standard definition of Verification goes like this: "Are we building the product RIGHT?" i.e. Verification is a process that makes it sure that the software product is developed the right way. The software should confirm to its predefined specifications, as the product development goes through different stages, an analysis is done to ensure that all required specifications are met.

Methods and techniques used in the Verification and Validation shall be designed carefully, the planning of which starts right from the beginning of the development process. The Verification part of ‘Verification and Validation Model’ comes before Validation, which incorporates Software inspections, reviews, audits, walkthroughs, buddy checks etc. in each phase of verification (every phase of Verification is a phase of the Testing Life Cycle)

During the Verification, the work product (the ready part of the Software being developed and various documentations) is reviewed/examined personally by one ore more persons in order to find and point out the defects in it. This process helps in prevention of potential bugs, which may cause in failure of the project.

Few terms involved in Verification:

• Inspection:

Inspection involves a team of about 3-6 people, led by a leader, which formally reviews the documents and work product during various phases of the product development life cycle. The work product and related documents are presented in front of the inspection team, the member of which carry different interpretations of the presentation. The bugs that are detected during the inspection are communicated to the next level in order to take care of them.

• Walkthroughs:

Walkthrough can be considered same as inspection without formal preparation (of any presentation or documentations). During the walkthrough meeting, the presenter/author introduces the material to all the participants in order to make them familiar with it. Even when the walkthroughs can help in finding potential bugs, they are used for knowledge sharing or communication purpose.

• Buddy Checks:

This is the simplest type of review activity used to find out bugs in a work product during the verification. In buddy check, one person goes through the documents prepared by another person in order to find out if that person has made mistake(s) i.e. to find out bugs which the author couldn’t find previously.

The activities involved in Verification process are: Requirement Specification verification, Functional design verification, internal/system design verification and code verification (these phases can also subdivided further). Each activity makes it sure that the product is developed right way and every requirement, every specification, design code etc. is verified!

What is Validation?
Validation is a process of finding out if the product being built is right?
i.e. whatever the software product is being developed, it should do what the user expects it to do. The software product should functionally do what it is supposed to, it should satisfy all the functional requirements set by the user. Validation is done during or at the end of the development process in order to determine whether the product satisfies specified requirements.

Validation and Verification processes go hand in hand, but visibly Validation process starts after Verification process ends (after coding of the product ends). Each Verification activity (such as Requirement Specification Verification, Functional design Verification etc.) has its corresponding Validation activity (such as Functional Validation/Testing, Code Validation/Testing, System/Integration Validation etc.).

All types of testing methods are basically carried out during the Validation process. Test plan, test suits and test cases are developed, which are used during the various phases of Validation process. The phases involved in Validation process are: Code Validation/Testing, Integration Validation/Integration Testing, Functional Validation/Functional Testing, and System/User Acceptance Testing/Validation.

Terms used in Validation process:

• Code Validation/Testing:

Developers as well as testers do the code validation. Unit Code Validation or Unit Testing is a type of testing, which the developers conduct in order to find out any bug in the code unit/module developed by them. Code testing other than Unit Testing can be done by testers or developers.

• Integration Validation/Testing:

Integration testing is carried out in order to find out if different (two or more) units/modules co-ordinate properly. This test helps in finding out if there is any defect in the interface between different modules.

• Functional Validation/Testing:

This type of testing is carried out in order to find if the system meets the functional requirements. In this type of testing, the system is validated for its functional behavior. Functional testing does not deal with internal coding of the project, in stead, it checks if the system behaves as per the expectations.

• User Acceptance Testing or System Validation:

In this type of testing, the developed product is handed over to the user/paid testers in order to test it in real time scenario. The product is validated to find out if it works according to the system specifications and satisfies all the user requirements. As the user/paid testers use the software, it may happen that bugs that are yet undiscovered, come up, which are communicated to the developers to be fixed. This helps in improvement of the final product.

Software Testing - Bug Life Cycles

What is a Bug Life Cycle?

The duration or time span between the first time bug is found (‘New’) and closed successfully (status: ‘Closed’), rejected, postponed or deferred is called as ‘Bug/Error Life Cycle’.

(Right from the first time any bug is detected till the point when the bug is fixed and closed, it is assigned various statuses which are New, Open, Postpone, Pending Retest, Retest, Pending Reject, Reject, Deferred, and Closed. For more information about various statuses used for a bug during a bug life cycle, you can refer to article ‘Software Testing – Bug & Statuses Used During A Bug Life Cycle’)

There are seven different life cycles that a bug can passes through:

< I > Cycle I:
1) A tester finds a bug and reports it to Test Lead.
2) The Test lead verifies if the bug is valid or not.
3) Test lead finds that the bug is not valid and the bug is ‘Rejected’.

< II > Cycle II:
1) A tester finds a bug and reports it to Test Lead.
2) The Test lead verifies if the bug is valid or not.
3) The bug is verified and reported to development team with status as ‘New’.
4) The development leader and team verify if it is a valid bug. The bug is invalid and is marked with a status of ‘Pending Reject’ before passing it back to the testing team.
5) After getting a satisfactory reply from the development side, the test leader marks the bug as ‘Rejected’.

< III > Cycle III:
1) A tester finds a bug and reports it to Test Lead.
2) The Test lead verifies if the bug is valid or not.
3) The bug is verified and reported to development team with status as ‘New’.
4) The development leader and team verify if it is a valid bug. The bug is valid and the development leader assigns a developer to it marking the status as ‘Assigned’.
5) The developer solves the problem and marks the bug as ‘Fixed’ and passes it back to the Development leader.
6) The development leader changes the status of the bug to ‘Pending Retest’ and passes on to the testing team for retest.
7) The test leader changes the status of the bug to ‘Retest’ and passes it to a tester for retest.
8) The tester retests the bug and it is working fine, so the tester closes the bug and marks it as ‘Closed’.

< IV > Cycle IV:
1) A tester finds a bug and reports it to Test Lead.
2) The Test lead verifies if the bug is valid or not.
3) The bug is verified and reported to development team with status as ‘New’.
4) The development leader and team verify if it is a valid bug. The bug is valid and the development leader assigns a developer to it marking the status as ‘Assigned’.
5) The developer solves the problem and marks the bug as ‘Fixed’ and passes it back to the Development leader.
6) The development leader changes the status of the bug to ‘Pending Retest’ and passes on to the testing team for retest.
7) The test leader changes the status of the bug to ‘Retest’ and passes it to a tester for retest.
8) The tester retests the bug and the same problem persists, so the tester after confirmation from test leader reopens the bug and marks it with ‘Reopen’ status. And the bug is passed back to the development team for fixing.

< V > Cycle V:
1) A tester finds a bug and reports it to Test Lead.
2) The Test lead verifies if the bug is valid or not.
3) The bug is verified and reported to development team with status as ‘New’.
4) The developer tries to verify if the bug is valid but fails in replicate the same scenario as was at the time of testing, but fails in that and asks for help from testing team.
5) The tester also fails to re-generate the scenario in which the bug was found. And developer rejects the bug marking it ‘Rejected’.

< VI > Cycle VI:
1) After confirmation that the data is unavailable or certain functionality is unavailable, the solution and retest of the bug is postponed for indefinite time and it is marked as ‘Postponed’.

< VII > Cycle VII:
1) If the bug does not stand importance and can be/needed to be postponed, then it is given a status as ‘Deferred’.

This way, any bug that is found ends up with a status of Closed, Rejected, Deferred or Postponed.

  How to create a secured and locked folder in Windows XP

• Since some people were having issues with this method of hiding a folder in XP (folder being renamed incorrectly, visible via the command prompt, etc), I am adding to the bottom another way to permanently hide a folder using a program called Free Hide Folder. Please scroll down to read about it.

• If you've come across this page looking for a way to create a password protected, secure, hidden or locked folder in Windows XP for free, you've come to the right place. Unfortunately, Windows XP does not have any user-friendly built-in tools to create secure folders on computers with multiple users. Of course, you can encrypt data on your hard disk

Here are the steps to create the protected folder in Windows XP:

• First create a folder that you will use to store your confidential data. For example, I have created a folder called Fonts at the root of my D drive. Since the hidden folder will bring you to the Control Panel, it’s best to name the folder as one of the programs inside there.

•  In the same location where you created the new folder, create a new file in Notepad, copy the following below into it, replace Fonts with the name of your folder and save it as loc.bat.

ren Fonts Fonts.{21EC2020-3AEA-1069-A2DD-08002B30309D}

• To save the file as a .bat file in Notepad, just put the whole thing in quotes, like“loc.bat” and then click Save.

• Now you should have a bat file named loc in the same directory 
  as your Fonts folder, not inside the Fonts folder.

• • • • • • 
            

• Create another NotePad file and type in the following listed below and save it as “key.bat”.

ren Fonts.{21EC2020-3AEA-1069-A2DD-08002B30309D} Fonts

• Now you’ll have loc.bat and key.bat along with your folder. Go ahead and double-click on loc.bat and your folder will turn into the Control Panel and nothing inside can be viewed. Clicking on it will simply bring you to the Control Panel. You’ll notice the icon should have changed also.
  
  
  
  
  
• • 
    
• • view the data inside your folder again, click on key.bat and your folder will return to normal! Pretty simple!
Of course, keeping the key.bat file in the same folder will defeat the purpose of securing the folder, so it’s best to move the key.bat file somewhere else, or even better, put it on a USB stick or CD that only you can access.
Most people will be probably look at the folder and simply ignore it since it has the control panel icon and it links directly there. Of course, if someone knows this trick also, they can create their own key.bat file, etc and open it back up. However, if you’re dealing with someone who can do all of that, it’s best you check out more advanced 3rd party software to lock down a folder.

Protocol Use at each of the TCP/IP Model Layers

At each layer of the OSI Model there are associated protocols that are in use.
These are not fully comprehensive lists but are examples of the more common protocols that are functioning at these different levels of the OSI Model.
At the Application layer you can find many but some of the more common ones include:

• DHCP - Dynamic Host Configuration Protocol
• FTP - File Transfer Protocol
• HTTP - HyperText Transfer Protocol
• IMAP - IMAP4, Internet Message Access Protocol (version 4)
• LDAP - Lightweight Directory Access Protocol
• LPD - Line Printer Daemon Protocol
• MIME (S-MIME) - Multipurpose Internet Mail Extensions and Secure MIME
• NFS - Network File System
• NNTP - Network News Transfer Protocol
• NTP - Network Time Protocol
• POP - POP3, Post Office Protocol (version 3)
• RDP - Remote Desktop Protocol
• RPC - Remote Procedure Call
• SMTP - Simple Mail Transfer Protocol
• SNMP - Simple Network Management Protocol
• SNTP - Simple Network Time Protocol
• SSH - Secure Shell
• TELNET - Terminal Emulation Protocol of TCP/IP
• TFTP - Trivial File Transfer Protocol

At the Presentation layer you can find these common protocols:

• MIME - Multipurpose Internet Mail Extensions
• SSL - Secure Sockets Layer
• TLS - Transport Layer Security
• XDR - eXternal Data Representation

At the Session layer you can find socket driven connections and session establishment in Transmission Control Protocol (TCP), Session Initiation Protocol (SIP), and Real-time Transport Protocol (RTP).
You can also find Named Pipe sessions, a protocol in the Server Message Block (SMB) suite as well as the NetBIOS (Network Basic Input/Output System) application Programming Interface (since NetBIOS is not formally a true networking protocol).
Session Announcement Protocol (SAP) is a protocol for broadcasting multicast session information and it is also found at the Session layer.
At the Transport layer you can find these common protocols:

• SPX - Sequenced Packet Exchange
• TCP - Transmission Control Protocol
• UDP - User Datagram Protocol
• SCTP - Stream Control Transmission Protocol

At the Network Layer you can find these common protocols:

• ATP - AppleTalk Transaction Protocol
• IPv4 - Internet Protocol v4
• IPv6 - Internet Protocol v6
• IPX - Internetwork Packet Exchange
• ICMP - Internet Control Message Protocol
• IGMP - Internet Group Management Protocol
• OSPF - Open Shortest Path First

At the Data Link Layer you can find these common protocols:

• PPP - Point-to-Point Protocol
• PPTP - Point-to-Point Tunneling Protocol
• SLIP - Serial Line Internet Protocol
• L2TP - Layer 2 Tunneling Protocol

Since the Physical Layer is really for defining the physical "connection" or attachment of given media and how it is configured as well as the electrical and physical properties and the operating specifications for the devices and media in use there are no actual TCP/IP common protocols that are in use.
You can find certain combinations of media and standards at this layer such as RS-232 (Recommended Standard 232) which is the standard for data and control signals connecting between a DTE (Data Terminal Equipment) and a DCE (Data Circuit-terminating Equipment) and Digital Subscriber Line (DSL) which provides digital data transmission over local telephone lines.

Introduction to the OSI Model

{The Open System Interconnection Reference Model (OSI) is a seven layer model that was developed as part of the effort to standardize networking that was started in the late 1970's as part of the Open Systems Interconnection (OSI) initiative}


The Seven Layers of the OSI Model.,

in summary the four layers of the OSI model are broken as follows:

• The Physical Layer defines the electrical and physical properties and the operating specifications for the devices and media in use. The main job of the Physical Layer is the physical "connection" or attachment of given media and how it is configured (e.g. Token Ring cable, size of cable used, termination in place etc.). In some instances, there may be secondary responsibilities of this layer depending on the device for things such as flow control, modulation/demodulation and so forth. The protocol data unit in use at this level of the OSI model is referred to as a "bit."

• The Data Link Layer provides the practical means to transfer data between network nodes as its main job is to transfer data between network nodes in a wide area network or between nodes on the same local area network segment/subnet. It has the secondary responsibility to detect and correct errors (as permissible) that may take place at the Physical Layer. The protocol data unit in use at this level of the OSI model is referred to as a "frame.

• The Network Layer handles the forwarding and routing of data along logical paths between network connected nodes. In addition to routing and forwarding functions of this layer of the model is also performs addressing, error handling, quality of service control, congestion control and packet sequencing. The protocol data unit in use at this level of the OSI model is referred to as a "packet."

• The Transport Layer is responsible for the reliable, end to end transfer, recovery and flow control of the segments between the nodes. The protocol data unit in use at this level of the OSI model is referred to as a "segment."

• The Session Layer addresses the build up and tear down of the connection sessions between nodes on a network. The protocol data unit in use at this level (and all of the subsequent levels) of the OSI model is referred to simply as "data."

• The Presentation Layer is responsible for taking the data from applications at the application layer and breaking it down for use on the session layer as well as the reverse. It also has the task of formatting the data so that it can be sent to other nodes.

• The Application Layer handles the initial connection of a given application to the network. It is where applications and application type activities such as browsing the web, sending and receiving email and performing file transfers take place. There are applications that wholly reside at the level such as Telnet and FTP.

SOFTWARE TESTING TYPES

Software Testing Types:

Black box testing 

Internal system design is not considered in this type of testing. Tests are based on requirements and functionality.

White box testing

This testing is based on knowledge of the internal logic of an application’s code. Also known as Glass box Testing. Internal software and code working should be known for this type of testing. Tests are based on coverage of code statements, branches, paths, conditions.

Unit testing

Testing of individual software components or modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. may require developing test driver modules or test harnesses.

Incremental integration testing

Bottom up approach for testing i.e continuous testing of an application as new functionality is added; Application functionality and modules should be independent enough to test separately. done by programmers or by testers.

Integration testing

Testing of integrated modules to verify combined functionality after inte gration. Modules are typically code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.

Functional testing

This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application.

System testing 

Entire system is tested as per the requirements. Black-box type testing that is based on overall requirements specifications, covers all combined parts of a system.

End-to-end testing

Similar to system testing, involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity testing 

Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix.

Regression testing 

Testing the application as a whole for the modification in any module or functionality. Difficult to cover all the system in regression testing so typically automation tools are used for these testing types.

Acceptance testing 

Normally this type of testing is done to verify if system meets the customer specified requirements. User or customer do this testing to determine whether to accept application.

Load testing 

Its a performance testing to check system behavior under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.

Stress testing

System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.

Performance testing 

Term often used interchangeably with ‘stress’ and ‘load’ testing. To check whether system meets performance requirements. Used different performance and load tools to do this.

Usability testing

User-friendliness check. Application flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system navigation is checked in this testing.

Install/uninstall testing  

Tested for full, partial, or upgrade install/uninstall processes on different operating systems under different hardware, software environment.

Recovery testing

Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.

Security testing  

Can system be penetrated by any hacking way. Testing how well the system protects against unauthorized internal or external access. Checked if system, database is safe from external attacks.

Compatibility testing 

Testing how well software performs in a particular hardware/software/operating system/network environment and different combination s of above.

Comparison testing

Comparison of product strengths and weaknesses with previous versions or other similar products.

Alpha testing 

In house virtual user environment can be created for this type of testing. Testing is done at the end of development. Still minor design changes may be made as a result of such testing.

Beta testing 

Testing typically done by end-users or others. Final testing before releasing application for commercial purpose.

SMOKE TESTING:

• Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
• A smoke test is scripted, either using a written set of tests or an automated test
• A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
• Smoke testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with finer details. (Such as build verification).
• Smoke testing is normal health check up to a build of an application before taking it to testing in depth.

SANITY TESTING:

• A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
• A sanity test is usually unscripted.
• A Sanity test is used to determine a small section of the application is still working after a minor change.
• Sanity testing is a cursory testing, it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
• Sanity testing is to verify whether requirements are met or not, checking all features breadth-first.