Veterinary Thermographic Image Analysis
Project Number 7- 64878
Report Number 4878-10
Scott E Umbaugh, BSE, MSEE, PhD
Patrick Solt, BSEE, MSCS, PhD Candidate
Hari Krishna Akkineni, BSEE, MSEE Candidate
Computer Vision and Image Processing Laboratory
Southern Illinois University Edwardsville
May 28, 2010
Spring 2010
Submitted to:
Dr. Catherine Loughin
Dr. Dominic Marino, Chief of Staff
Long Island Veterinary Specialists
TABLE OF CONTENTS
Executive Summary…………………………………………………... 2
Introduction and Overview…………...………………………………. 4
Materials and Methods………………………………………………... 5
Images………………………………………………………….. 6
Software…………………………………………………………7
Mask Creation Software…….……………………………7
Pattern Classification Software....………………………..7
Texture Features………………………………………….7
Algorithm Test and Analysis Tool ……………………..7
Experimental Results and Discussion………………………………… 8
Preliminary Experiments………………………………….…… 8
Final Experiments………………………………………………8
Results from Experiment Set #1…………………………..……9
Results from Experiment Set #1 PCA Scatterplot……………. 10
Results from Experiment Set #2………………………….……11
Results from Experiment Set #2 PCA Scatterplot...…….….…12
Results from Experiment Set #3…………………………..…..13
Results from Experiment Set #3 PCA Scatterplot...…….…… 14
Results from Experiment Set #4…………………………..…..15
Results from Experiment Set #4 PCA Scatterplot...…….…… 16
Results from Experiment Set #5……………………………... 17
Results from Experiment Set #5 PCA Scatterplot...…….…… 18
Results from Experiment Set #6…………………………..…. 19
Results from Experiment Set #6 PCA Scatterplot...…….…… 20
Conclusions ………………………………………………………… 21
Current aned Future Projects Overview…………………………….. 22
Future Tasks………………………………………………………… 23
References……………………………………………………………24
Executive Summary
Research and development was continued to investigate the application of image analysis and pattern classification to new thermographic images of 105 canines. Specifically, to examine the thermographic images of canines of the breed Cavalier King Charles Spaniel to investigate the Chiari malformation, or COMS, pathology, and to compare to previous preliminary results. Pattern classification algorithms were developed for: absent, severe, moderate and mild classes of the cerebellar herniation and kinking of medula. It is believed that, in some cases, limited success may be due to the inconsistency in the classification method and this problem will be investigated in the future through the use of the new Index or Rating metric. The eventual goal for the research is to be able to differentiate normal and abnormal thermographic patterns in canines as a diagnostic and research tool.
Development of application-specific software was continued. Software from Phase I and Phase II of the research was enhanced. The capability of automatically performing leave-one-out cross-validation is being added to the pattern classification GUI. The software used the CVIPtools image libraries. Continued development and enhancement of this software will be quite useful for this research area.
Six sets of experiments were performed for a total for of 12,276 experiments. Each experiment consisted of a specific body part, specific classes and specific features. Experiments were performed with all permutations of the ten selected features on each selected body part.
The first set of experiments used the clinical images that were classified as severe and moderate classes of the cerebellar herniation. Here, the front of head (A1), and top of head (A1D) images were used. With top, 89.2% and front images, 97.3% successful classification was achieved. These results are not as good as the 100% previously achieved with smaller sets. This indicates that the class domains are more diverse, and the classes more correlated, than was found with the smaller set. Additionally, further experimentation with color normalization is warranted. However, the 97% success still indicates that there is a difference between the moderate and severe classes with the clinical images. The features texture correlation and histogram energy are the most predictive features.
The second set of experiments used the clinical images that were classified as severe, moderate and mild classes of the cerebellar herniation. Here, the front of head (A1), top of head (A1D) images were used. With top, 69.8%and front images, 67.9% successful classification was achieved. These results are inconclusive but indicate that there may be a difference between the mild, moderate and severe classes in the clinical images, but indicate that addition of the mild class confusses the classification. It is believed this may be due to the inconsistency in the classification and this problem will be investigated through the use of the new Index or Rating metric.
The third set of experiments used the clinical images that were classified as severe, moderate, mild and absent classes of the cerebellar herniation. Here, the front of head (A1), top of head (A1D) images were used. With top, 62.5% and front images, 56.45% successful classification was achieved. These results are inconclusive but indicate that there may be a difference between the absent, mild, moderate and severe classes in the clinical images. It is believed this may be due to the inconsistency in the classification and this problem will be investigated through the use of the new Index or Rating metric.
The fourth set of experiments used the clinical images that were classified as severe and moderate classes of the kinking of medulla. Here, the front of head (A1), and top of head (A1D) images were used. With top and front images, 89.5% successful classification was achieved. These results are not as good as the 100% previously achieved with smaller sets. This indicates that the class domains are more diverse, and the classes more correlated, than was found with the smaller set. Additionally, further experimentation with color normalization is warranted. However, the 90% success still indicates that there is a difference between the moderate and severe classes with the clinical images. The features histogram entropy and energy are the most predictive features.
The fifth set of experiments used the clinical images that were classified as severe, moderate and mild classes of the kinking of medulla. Here, the front of head (A1), top of head (A1D) images were used. With top, 82.8%and front images, 79.3% successful classification was achieved. These results are inconclusive but indicate that there may be a difference between the mild, moderate and severe classes in the clinical images. It is believed this may be due to the inconsistency in the classification and this problem will be investigated through the use of the new Index or Rating metric.
The sixth set of experiments used the clinical images that were classified as severe, moderate, mild and absent classes of the kinking of medulla. Here, the front of head (A1), top of head (A1D) images were used. With top, 60% and front images, 58% successful classification was achieved. These results are inconclusive but indicate that there may be a difference between the absent, mild, moderate and severe classes in the clinical images. It is believed this may be due to the inconsistency in the classification and this problem will be investigated through the use of the new Index or Rating metric.
Introduction and Overview
Thermographic images of 105 canines were analyzed using feature extraction and pattern classification tools. The purpose of this research is to examine the thermographic images of canines of the breed Cavalier King Charles Spaniel to investigate the Chiari malformation, or COMS, pathology.
Three questions were investigated and related experiments were performed:
1. Can we successfully differentiate between the classes moderate and severe in the clinical images? Here two different experiments were performed:
Ø Use all front of head images (A1)
Ø Use all top of head images (A1D)
2. Can we successfully differentiate between the classes mild, moderate and severe in the clinical images?
Ø Use all front of head images
Ø Use all top of head images
3. Can we successfully differentiate between the classes absent, mild, moderate and severe in the clinical images?
Ø Use all front of head images
Ø Use all top of head images
Materials and Methods
Images
The thermal images are from Long Island Veterinary Specialists taken with a Meditherm Med2000 IRIS. The images are in TIF file format as RGB images with 319 columns by 255 rows, 8-bits per pixel per color band. Note that a total of 18 colors are used in these images. The images are thermal images of 105 canines of the breed Cavalier King Charles Spaniel. Previous experimentation used a separate set of 34 canine images. Included in the data set are images of front of head and top of head.
Software
The CVIPtools (Computer Vision and Image Processing Tools) software was used to investigate and analyze the images. The primary tools used include Analysis->Features and Analysis->Pattern Classification. The features considered include histogram, and texture features. The histogram features include mean, standard deviation, skew, energy and entropy; and the texture features include energy, inertia, correlation, inverse difference, and entropy. The pattern classification tools utilized include various normalization methods, distance and similarity measure and three classification methods.
In the course of the image analysis it was determined that the previously developed special purpose software would be further enhanced for this phase of the research. The software is designed to ease the analysis process and enable us to perform a significantly greater number of experiments. The software that creates image masks for the various body parts was enhanced to include inclusion of specific colors in the thermographs. The application specific software previously developed has functionality to ease the mask creation task and also allows the user to load images, classify images, select features and perform pattern classification experiments (see Figure 1). The application software is being enhanced to automatically run experiments with all permutations for the feature set(s) and to perform leave-one-out cross-valation for all the experiments performed.
Figure 1. Application specific software developed for this project
Mask Creation Software
The enhanced mask creation software has been used for the project. This enhanced software facilitates the creation of a much larger number of masks and has proved useful due to the acquisition of many more images for current and future projects. The refinements to the methodology allows for the masks to be created in a more streamlined fashion, and with better accuracy.
Pattern Classification Software
The pattern classification was modified for the additional images in project. Specifically, spectral features have been explored. Next is to apply the new texture features. Additionally, the leave-one-out testing technique has been integrated into CVIPtools.
Texture Features
Research has been undertaken for new texture features and the software is complete. It will be applied to the to the next Phase of the project, to the approximately 105 new COMS images. The new texture feature extraction software includes the five features that have been found to be the most useful: energy (homogeneity), inertia (contrast), correlation (linearity), inverse difference (local homogeneity) and entropy (details are in spring 2009 report)
Algorithm Test and Analysis Tool
The development of the Algorithm Test and Analysis Tool (ATAT) should prove very useful as the number of projects expands, and the nature of the projects varies. Similar to what was done with the previous software development where we automated the experimental process so that we could run millions of experiments instead of hundreds, the ATAT software will allow for the investigation of many more permutations of image processing and feature extraction algorithms than was previously possible.
The ATAT software will allow the user to select from a set of image processing filters, segmentation methods, morphological filters, and various postprocessing imaging functions. The user will be able to specify the range and the increment to be used with each of the imaging functions’ parameters. The software will then automatically run all the possible algorithmic permutations. The success measure will be tailored to the specific application, and we are currently investigating the useful success metrics for the veterinary projects. An application manual for the current version of the software, CVIP-ATAT, is included in Appendix A of the spring 2009 report.
Experimental Results and Discussion
Preliminary Experiments:
Preliminary experiments were performed previously to determine the optimal classification method and distance metric, optimal set of features, and best normalization method for the final experiments. Under consideration were the following:
CLASSIFICATION METHODS AND DISTANCE METRIC
- Nearest Neighbor, distance metric: Euclidean
- K-Nearest Neighbor with K = two through five, distance metric: Euclidean
FEATURES
- Spectral features: sectors and rings, from 3 to 16.
- Histogram features: mean,. Standard deviation, Skew, Energy and Entropy
- Texture features: Energy, Inertia, Correlation, Inverse difference, and Entropy. The pixel distance was varied from two to sixteen.
DATA NORMALIZATION METHODS
- Euclidean
- Soft-max with r=1
Final Experiments:
After an initial set of experiments it was determined that following would be used for all the final experiments:
CLASSIFICATION METHOD AND DISTANCE METRIC
Ø K-Nearest Neighbor with K = 3, distance metric: Euclidean
FEATURES
Ø Histogram features: Mean. Standard deviation, Skew, Energy and Entropy
Ø Texture features: Energy, Inertia, Correlation, Inverse difference, and Entropy. The pixel distance was 6.
DATA NORMALIZATION METHOD
Ø Soft-max with r=1
The final experiments were performed with the 1,023 permutations of the ten features. A total of 12,276 experiments were performed.
Results from Experiment Set #1.
Images: Clinical group, the front and top images of the head.
Classes: moderate and severe, cerebellar herniation.
Note: for complete results see the Excel spreadsheet files Experiment1_A1, Experiment1_A1D.
Top five classification results for Experiment #1Features
(texture pixel dist=6) / Body Part / Number of images per class / Classification Success
Histogram energy / Top of head / Moderate: 33
Severe: 4 / 89.2%
Histogram energy
Histogram entropy / Top of head / Moderate: 33
Severe: 4 / 89.2%
Texture inertia
Texture correlation
Histogram energy / Front of head / Moderate: 33
Severe: 4 / 97.3%
Texture energy
Texture correlation
Histogram energy / Front of head / Moderate: 33
Severe: 4 / 97.3%
Texture correlation
Histogram StdDev / Front of head / Moderate: 33
Severe: 4 / 94.6%
Ø We believe that the 89% and 97% classification success indicates that the top and front of head are the most useful for differentiation of the two classes.