System Testing
System testing is the process of testing an integrated system to verify that it meets specified
requirements".
System testing is the process of testing an integrated system to verify that it meets specified
requirements".
It is further sub-divided into
- Functional system testing
- Non-Functional system testing
System test Entrance Criteria:
- Successful execution of the Integration test cases
- No open severity 1 or 2 defects
- 75-80% of total system functionality and 90% of major functionality delivered
- System stability for 48-72 hours to start test
System Test Exit Criteria:
- Successful execution of the system test cases ,and documentation that shows
- coverage of requirements and high-risk system components
- System meets pre-defined quality goals
- 100% of total system functionality delivered
Functional Testing
Requirement based Testing
Designing tests based on objectives derived from requirements for the software component (e.g.,
tests that exercise specific functions or probe the non-functional constraints such as performance
or security)
Designing tests based on objectives derived from requirements for the software component (e.g.,
tests that exercise specific functions or probe the non-functional constraints such as performance
or security)
Requirements testing must verify that the system can perform its function correctly and that the correctness can be sustained over a continuous period of time. Unless the systemcan function correctly over an extended period of time management will not be able torely upon the system. The system can be tested for correctness throughout the lifecycle, but it is difficult to test the reliability until the program becomes operational.
Objectives:
Successfully implementing user requirements is only one aspect of requirements testing.The responsible user is normally only one of many groups having an interest in the application system.
Successfully implementing user requirements is only one aspect of requirements testing.The responsible user is normally only one of many groups having an interest in the application system.
The objectives that need to be addressed in requirements testing are:
- User requirements are implemented
- Correctness is maintained over extended processing periods.
- Application processing complies with the organization’s policies and procedures.
When to use Requirements Testing:
Every application should be requirements tested. The process should begin in therequirements phase, and continue through every phase of the life cycle into operations and maintenance. It is not a question as to whether requirements must be tested but,rather, the extent and methods used in requirements testing.
Every application should be requirements tested. The process should begin in therequirements phase, and continue through every phase of the life cycle into operations and maintenance. It is not a question as to whether requirements must be tested but,rather, the extent and methods used in requirements testing.
Business-Process based Non-F unctional Testing
Testing of those requirements that do not relate to functionality. I.e. performance, usability, etc.
Testing of those requirements that do not relate to functionality. I.e. performance, usability, etc.
Non-Functional testing types:
- Recovery Testing
- Security Testing
- Stress Testing
- Performance Testing
Recovery testingTesting aimed at verifying the system's ability to recover from varying degrees of failure.
Recovery is the ability to restart operations after the integrity of the application has been lost. The process normally involves reverting to a point where the integrity of the system is known, and then reprocessing transactions up until the point of failure. The importance of recovery will vary from application to application.
Objectives:
Recovery testing is used to ensure that operations can be continued after a disaster,
recovery testing not only verifies the recovery process, but also the effectiveness of the component parts of that process.
Recovery testing is used to ensure that operations can be continued after a disaster,
recovery testing not only verifies the recovery process, but also the effectiveness of the component parts of that process.
Specific objectives of recovery testing include:
- Adequate backup data is preserved
- Backup data is stored in a secure location
- Recovery procedure are documented
- Recovery personnel have been assigned and trained
- Recovery tools have been developed and are available
When to use Recovery Testing:
Recovery testing should be performed whenever the user of the application states that the continuity of operation of the application is essential to the proper functioning of the user area.The user should estimate the potential loss associated with inability to recover operations over various time spans. The amount of the potential loss should both determine the amount of resource to be put into disaster planning as well as recovery testing.
Recovery testing should be performed whenever the user of the application states that the continuity of operation of the application is essential to the proper functioning of the user area.The user should estimate the potential loss associated with inability to recover operations over various time spans. The amount of the potential loss should both determine the amount of resource to be put into disaster planning as well as recovery testing.
Security testing
Testing whether the system meets its specified security objectives.
Security is a protection system that is needed for both secure confidential information and for competitive purposes to assure third parties their data will be protected. Protecting the confidentiality of the information is designed to protect the resources of the organization. Security testing is designed to evaluate the adequacy of the protective procedures and countermeasures.
Objectives:
Security defects do not become as obvious as other types of defects. Therefore, the objectives of security testing are to identify defects that are very difficult to identify. Even failures in the security system operation may not be detected, resulting in a loss or compromise of information without the knowledge of that loss.
Security defects do not become as obvious as other types of defects. Therefore, the objectives of security testing are to identify defects that are very difficult to identify. Even failures in the security system operation may not be detected, resulting in a loss or compromise of information without the knowledge of that loss.
The security testing objectives include:
- Determine that adequate attention has been devoted to identifying security risks
- Determining that a realistic definition and enforcement of access to the system has been implemented
- Determining that sufficient expertise exists to perform adequate security testing
- Conducting reasonable tests to ensure that the implemented security measures function properly
When to Use security Testing:
Security testing should be used when the information and/or assets protected by the application system are of significant value to the organization. The testing should be performed both prior to the system going into an operational status and after the system is placed into an operational status. The extent of testing should depend on the security risks, and the individual assigned to conduct the test should be selected based on the estimated sophistication that might be used to penetrate security.
Security testing should be used when the information and/or assets protected by the application system are of significant value to the organization. The testing should be performed both prior to the system going into an operational status and after the system is placed into an operational status. The extent of testing should depend on the security risks, and the individual assigned to conduct the test should be selected based on the estimated sophistication that might be used to penetrate security.
Stress testing
Testing conducted to evaluate a system or component at or beyond the limits of its specified
requirements.
Testing conducted to evaluate a system or component at or beyond the limits of its specified
requirements.
Stress testing is designed to test the software with abnormal situations. Stress testing attempts to
find the limits at which the system will fail through abnormal quantity or frequency of inputs.
find the limits at which the system will fail through abnormal quantity or frequency of inputs.
For example;
- Higher rates of interrupts.
- Data rates an order of magnitude above 'normal'.
- Test cases that require maximum memory or other resources.
- Test cases that cause 'thrashing' in a virtual operating system.
- Test cases that cause excessive 'hunting' for data on disk systems.
Performance testing
Testing conducted to evaluate the compliance of a system or component with specified
performance requirements.
Performance testing is designed to test run time performance of software within the context of an integrated system. It is not until all systems elements are fully integrated and certified as free of defects the true performance of a system can be ascertained.
Testing conducted to evaluate the compliance of a system or component with specified
performance requirements.
Performance testing is designed to test run time performance of software within the context of an integrated system. It is not until all systems elements are fully integrated and certified as free of defects the true performance of a system can be ascertained.
Performance tests are often coupled with stress testing and often require both hardware and software infrastructure. That is, it is necessary to measure resource utilization in an exacting fashion. External instrumentation can monitor intervals, log events. By instrument the system, the tester can uncover situations that lead to degradation and possible system failure.
Alpha and Beta testing
Alpha testing: Simulated or actual operational testing at an in-house site not otherwise involved
with the software developers.
with the software developers.
Beta testing: Operational testing at a site not otherwise involved with the software developers.
This is testing of an operational nature once the software seems stable. It should be conducted by people who represent the software vendor's market, and who will use the product in the same way as the final version once it is released. The benefit of this type of acceptance testing is that it will bring out operational issues from potential customers prepared to comment on the software before it is officially released.
Alpha testing is conducted at the developer's site by a customer. The customer uses the software with the developer 'looking over the shoulder' and recording errors and usage problems. Alpha testing conducted in a controlled environment.
Beta testing is conducted at one or more customer sites by end users. It is 'live' testing in an environment not controlled by the developer. The customer records and reports difficulties and errors at regular intervals.
