Report White Box Testing
White box testing uses the internal mechanism of a system to create test cases, which uses calculated paths to discover unidentified bugs within the system. This type of testing is trying to enforce the quality of the software system however “white box testing” is a cost effective method and is compared very closely to “Black box testing”. The main jobs of these two functions have the same purpose however it is majorly debated which one is more efficient and effective. Black box testing concentrates mainly on the outputs of the system to identify bugs however this function waits till a malfunctioning error has occurred. As explained above White box testing is used to ensure that the code is complete and to the correct standard of the software mechanism. Statistics haves proven that by using a complete and precise systematic test design, will ensure that the majority of bugs within the system will be identified.
When looking at white box testing in more detail it involves checking and ensuring that every program statement is error free. White box testing allows:
Data Processing
Calculation correctness tests
Software Qualification tests
Maintainability tests
Reusability tests
Data Processing and Calculation Correctness tests
Within the main two concepts of Calculation Correctness tests and Data processing, when a test case has been created (path) it is then proceeded to be tested and verified which will ensure the correctness of the software program. This procedure is done in sequence for every test case created, this gives the user the opportunity to test whether the code / program has been implemented correctly to the requirements / specification.
Software Qualification tests
When going through the software qualification tests it looks directly to the software code itself. This tests works with the “coding standards” and work instructions set by the client or developer.
Maintainability tests
The maintainability tests installs special features to detect failures, module structures that support software development,extensions and improvements of the software system. The reusability tests is part of an example of this, as explained below:
Reusability tests
Reusability tests are performed on the package which examines all of the reused software that had been implemented into the software package. Addition to this function it also adapts to the program allowing the present code to be re-usable for future preferences.
When looking at the White box testing in more detail and analysing its procedures it becomes aware that when testing each test case / path maybe become unrealistic and unfeasible. We have to take into account on the amount of coverage of all the possible paths and the amount of code that is produced. There are two possible solutions for this situation:
Line Coverage – this is to test and cover over all of the program code and the coverage is calculated by the percentage of lines covered.
Path Coverage – this is to test and cover all the available paths in the system and the coverage is calculated by the percentage of paths covered.
Correctness tests and Path coverage
Path testing’s main goal is to go through the system covering all possible paths, this is tested with all condition such as IF-THEN-ELSE and DO WHILEstatements. To take white box into account it is very impractical due to the amount of resources needed for this type of testing. Path testing is used from the start of a method to when it finishes, each path shows the flow of execution. By looking at white box complete path coverage will show how extreme this type of testing is:
Example:
There is a software package that contains five conditional statements which will only for this example allow two procedures eg IF-THEN-ELSE, DO WHILE. When following this type of code, it is expected that it will contain over 512 paths (25 – 50 lines of code), therefore at least 512 test cases should be performed ensuring each path has been covered. Just by looking at this on example gives a good representation of the type of testing performed. In addition to this we can see that by having 50 modules a calculation can be made which will result in proximally 61,000 test cases needed in order to carry out this procedure.
This is very disproportionate testing however can be very beneficial as this type of testing should outline a great proportion of errors and defects within the software system. This complete path coverage will be tested on systems such as hospital related machines, emergency services system and mostly to do with high risk software systems. This will substitute the price of costing for the price of the risk of this system failing, therefore allowing for this long procedure to be carried out.
Correctness Tests and Line Coverage
The line coverage requires a different process to path coverage as explained above. There are different types of line coverage depending on the users requirements and specification of the software system. One function will be to run a full complete line coverage process which will consist of each line of code being compiled and executed at least once. This will be shown as a percentage of the lines compiled during the testing process. Line coverage may also perform a less efficient method test which consists of testing fewer test cases which will make the system more vulnerable to to bugs and system errors.
When talking about correctness tests, flow chart and a flow graph can be used to enforce the understanding of line coverage within the correctness tests.
A diamond will represent: the options covered by a conditional statement
A rectangle will represent: software sections connecting the diamonds.
The arrows / edges will represent: The flow of direction.
A flow diagram to show how car insurance can be calculated based on the users circumstances. This diagram involves:
A Car Insurance Company and calculating the users annual car insurance will involve:
If the users car is less then £10,000 or greater then,
Is the year of the car older then 5 years,
How old is the user,
Does the user have any previous driving experience,
Does the user have more then two years no claims bonus.
From this diagram is possible to come out with another of different unique paths. Overall to achieve complete path coverage an estimated 24 test cases would have to be performed due to their being 24 individual unique paths.
Examples of unique paths have been displayed below.
Complete Full Path Coverage
1= 1-2-3-5-6-8-9-11-12-17
2= 1-2-3-5-6-8-9-13-14-15-17
3= 1-2-3-5-6-8-9-11-13-14-16
4= 1-2-3-5-6-8-10-11-17
…
13= 1-2-4-5-6-8-9-11-12-17
14= 1-2-4-5-6-8-9-13-14-15-17
…
23= 1-2-4-5-7-8-10-11-13-14-15-17
24= 1-2-4-5-7-8-10-11-13-14-16-17
By looking at this scenario, if the entities and modules increased the amount of testing to be performed on this would grow rapidly. The increase in modules would then lead to a more complex system. A scientist called McCabe then come up with a theory to determine how complex a software system was and how easy it was to interoperate this type of software system.
McCabes Cyclomatic Complexity Metrics
As explained above McCabes theory was to find out how complex a system was and how much testing would be sufficient in order to complete white box testing. McCabe came up with a formula to distinguish the complexity, and this was done by measuring the number of independent paths needed to achieve full line coverage. This strategy would be based on taking the characteristics of the flow diagram and placing this information into the formula. An example of this is shown below:
McCabes formulae:
V(G) = R
V (G) = E –N + 2
V(G) = P + 1
R = The number of regions that is involved in the program flow graph. (Any enclosed area including the outer layer of the flow diagram.)
E = The number of edges
N = The number of nodes
P = The number of decisions
I have now taken McCabes theory and have applied it to our Car Insurance Scenario:
Applying McCabes theories.
Red Circles indicate the regions within our car insurance example
The amount of arrows subtracted by the number of nodes
The diamond shapes are represented as P as they are decision entities.
Our Example
V(G) = 6
V(G) = 21 -17 + 2 = 6
V(G) = 5 + 1 = 6
Resulting metrics calculations indicate that the maximum number of independent paths needed to achieve full line coverage is 6. To summarise Mccabes objective is that programs with cyclomatic complexities of less then 5 are simple and easy to understand, 20+ independent paths needed results in the complexity is perceived as high and extreme testing would have to be performed. Also any program containing 50 and above the software for practical purposes becomes known unpractical maybe even impossible.
Software Qualification Testing
Software qualification is just as important as maintenance testing, its main objective is to check whether the software being developed is to its correct structure and meets the appropriate standards. This type of testing checks whether the coding and documentation has been carried out to the correct procedures and working instructions. By enforcing this type of tests it gives the organisation a much more efficient and well structured piece of software. One of the main procedures is to check and validate whether the software has responded to procedures such as:
Does the coding style comply with the correct style procedures,
Does the coding comply with coding structures
Is the Documentation to the correct standard.
Today it is possible to buy different applications and software packages that allow a company or organisation to manage coding standards. This type of code auditors enables the user to take full control of the code and give an automatic review on the current project or development. The main coding auditor that is used in many organisations and programming projects is SSW Code Auditor.
Re-usability Testing
When looking at this method we first have to look at what is re-usability? This can be defined as a section of code that can be re-used to add new functionalities with very little or no modification. Not only is this time consuming, it is expected that this type of code has been through a number of testing procedure, eliminating any errors or bugs. This clean section of code can then be simply integrated into a new working project with the knowledge that this code is successfully working and follows all the correct standards and working instructions.
So when looking at re-usable code, the re-usability testing objective is to check and determines whether this section of code is efficient and bug free. This type of testing also involves the documentation side, as user manuals and help wizards have a great number of similarities.
Advantages of White Box Testing
Introspection – this gives the user or tester the opportunity to test internally which allows a better perspective of objects and modules within the software. This is effective when the Graphical user Interface is changing frequently.
Identification – By going through each line the test is able to reveal what test cases have been applied to what object and lists the number of lines that have not yet been executed.
Effectiveness – This type of testing gives the end user or the supplier the confidence that every path has been tested to a certain extent, which should limit the amount of bugs within an application. White box testing is the most effective testing as it the most thorough when testing an individual application.
Disadvantages of White Box Testing
A skilled and trained tester will be needed to perform this type of testing. This type of testing requires a certain level of skill, ensuring that every path and test case has been examined at least once.
This type of testing is time consuming and may be very cost effective.
Also it has been said that it is nearly impossible to look into every bit of code within the infrastructure to find all possible errors.
References:
White-box testing
- last modified on 25 November 2009
White-Box Testing report
- last modified on January 14 2006
White Box Testing
- last updated 2009-07-27
Software Testing - White Box Testing Strategy
- last updated 17 Nov 2009 07:51:54 GMT
Galin, D.(2004), Software Quality Assurance From theory to implementation, Pearson Addison Wesley.