Unsupervised Color Correction Using Cast Removal for
Underwater Images
R. A. SALAM, A. O. H. EE
School of Computer Sciences
Universiti Sains Malaysia
11800 Penang
MALAYSIA
M. S. HITAM
Department of Computer Science
University College of Science and Technology Malaysia
Mengabang Telipot, 210300 Kuala Terengganu, Terengganu
MALAYSIA
Abstract: - Image enhancement for underwater images is very important especially for marine biologists. They required an automated tool to help them to observe underwater images. Aquatic environment generate some problems in underwater images, such as blurring of image features, clutter and lack of structure in the regions of interest due to the transmission of the medium. Underwater images have always been wet or bluish. The focus of this paper is to investigate and evaluate existing image enhancement methods and algorithms and to apply these techniques to underwater images. We proposed a framework for underwater image enhancement that starts with an unsupervised color correction using cast removal, contrast enhancement using histogram equalization and finally are the noise filtering and image sharpening techniques. These stages are very important for underwater image enhancement. As a result, an automatic tool will be developed to enhance underwater images.
Key-Words: - underwater images, image enhancement, contrast, sharpness, color correction, cast removal.
1 Introduction
In general, physical content of the scene, illumination on the scene and characteristics of the digital camera are the three issues of image taken by a camera. The physical content of the scene is the main interest in many applications. The images produced by digital cameras are often quite good, but rarely perfect. They may not be sufficient for recording a scene. We may need to edit the brightness, contrast and color of all or parts of the image.
In modern computing, various type of scientific fields applications involve in image processing, such as underwater research, medical diagnosis, forensic recognition, document images and industrial products monitoring. The most common problem in image processing applications is the occurrence of noise. Image enhancement is a process to solve computer vision problem. Image enhancement is a part of image processing that consists techniques to improve appearance of image. These techniques are used to emphasize and sharpen image features for display and analysis and also extract the important information from a poor quality image. However, image enhancement techniques are application specific. An image enhancement technique that works in some applications may not necessarily be the best approach for other applications.
Image enhancement is used as a preprocessing step in some computer vision applications to ease vision task, for example, enhancing edges of marking in inspection vision machine. Sometimes, image enhancement is used for post-processing to produce a desirable visual image. For instance, output image may lose most of its contrast after performing image restoration to eliminate image distortion. Here, image enhancement techniques can be applied to restore the image contrast. Besides, post-processing image enhancement may also improve the quality of image after a compressed image decompressed to its original image. For example, post-processing image enhancement with a smoothing filter technique will improve appearance of image that contains noise generated by standard JPEG compression algorithm [1].
For aquatic environments’ scientists, the image processing techniques for coping with image enhancement in underwater conditions is very important. Aquatic environment generate some problems in underwater images, such as blurring of image features, clutter and lack of structure in the regions of interest due to the transmission of the medium [2].
The underwater image enhancement techniques are crucial for studying archaeological sites in various oceans of the world. Most historical objects found underwater have to be analyzed directly from the image taken. Underwater photography presents a number of challenges. Apart from sky reflection, prominent blue color of ocean water is due to the selective absorption by water molecules. The water tends to filter out reddish tones. Red light reduces when the depth increases, thus producing blue to gray like images. The effect worsens at greater depths and distances. In fact all red lights are disappeared when reaching 3meter of depth [3]. In addition, underwater light is often less than ideal for picture taking. As a result, colors as seen and photographed underwater are severely affected by the enormous blue/green filter factor of ocean water [3].
Underwater images have always been wet or bluish. One of the major intrinsic problems with underwater images using traditional image processing is often exaggerated green or blue color in the image. In addition, the produced images frequently appeared blurry.
This research will look at available techniques for image enhancement. The maim aim is to investigate on how existing image enhancement techniques can be used for underwater images. A framework that combines different methods was proposed so that it could enhance the underwater images and reduce the aquatic environment effects. One of the important techniques is the color balancing method. Color balancing refers to the process of removing an overall color casts from an image.
Section 2 illustrates the problem addressed, related image enhancement methods and color balancing. Section 3, shows the proposed framework of this research on improvement of enhancement techniques and algorithms in underwater images.
2 Background
In general, image enhancement falls into two categories: spatial domain methods that are based on direct manipulation of pixels in an image and frequency domain methods that is based on modifying the Fourier transform of an image. Some image enhancement algorithms combine both the spatial and frequency domains.
2.1 Color Balancing
One of the image enhancement algorithms is color balancing. Color balancing techniques are useful for correcting color casts in images. Color balancing is a process of removing an overall color bias from an image. For example, if an underwater image appears to be too blue, it means that the underwater image has blue cast. Removing this blue cast brings the underwater image back into balance. The way original scene was illuminated, usage of film and filters, differs in film processing, or from scanning process will increase the color casts. Controlling color imbalance in an image is a challenging task. Usually, the easiest way to correct the problem is at the end of the process.
A powerful method for identifying color casts is to measure the color of pixel. In principle, the color of pixel in an image should be neutral gray. Neutrals must have equal components of red, green, and blue. If they are not equal, that indicates the presence of a color cast and color correction must be made.
Deep Submergence Laboratory [3] has developed software for underwater imaging. They used laplacian filtering and also 2-dimension Fast Fourier Transform. Another underwater project was the prototype of jelly fish tracking system [6] which can help remote operated vehicles pilot to observer deep ocean life. They used gradient and color thresholding segmentation in parallel. Statistics for each segmented region will form a Pattern Vector. Error will be calculated from a new Pattern Vector with the existing one.
2.2 Enhancement by Point Processing
Enhancement by point processing is based on direct manipulation transformation of gray values or color values into other gray values or color values. Gray values or color values are usually represented by a 2-D plot graph [4].
Gray values can also be modified to create desired shape in histogram, such as flat, where every gray value has the same probability. In point processing, each output pixel is the function of one input pixel and the transformation is implemented with a look-up table (LUT) [4]. LUT provides a flexible and powerful approach to adjusting the appearance of an image.
2.3 Enhancement by Mask Processing
Mask Processing also known as image filtering aims to change properties of an image by removing noise, sharpening contrast or highlighting contours in images. Often, the goal of designing image enhancement is to smooth the images in more uniform regions, nevertheless preserve edges. Another design goal is to sharpen the image. Due to these goals, operations that perform need neighbourhood processing, in which the output pixel is a function of some neighbourhood of the input pixels.
In general filters can be classified into two common categories that are spatial domain filters and frequency domain filters. These operations can be performed using linear operations in either the frequency or the spatial domain. However, there is a third classifier that are applied spatially from frequency domain filters, by using Fourier-transformed representation of an image [4].
Frequency domain filters can be categorized into Low Pass Filter (LPF) or High Pass Filter (HPF). LPF leaves low frequency components unchanged. LPF is used for operations like noise removal or image smoothing. Another low pass filter that is frequently used is median filter, which replaces the central pixel with the local median value. Median filter is effective in removing speckle noise. HPF acts to sharpen details in an image. The disadvantage of HPF is that it tends to enhance high-frequency noise along with details of interest in an image [5].
Filters can be expressed as a matrix representing weights applied to the central pixels neighborhood. Linear filters, also known as convolution filters, use a matrix multiplication to represent in linear algebra. A convolution is a mathematical function that replaces each pixel by a weighted sum of its neighbors. Convolution mask is a matrix that defines neighborhood of a pixel and specifies weight assigned to each of its neighbor [5].
Other technique that used spatial domain information is the adaptive quadratic filter [5]. Apart from point processing (neighbourhood processing) and mask processing, there are also global processing techniques. These methods will performs when every pixel depends on all pixels of the whole image. Histogram methods are usually a global processing, but they can also be used in neighbourhood processing.
3 Methodology
Typical underwater image enhancement properties requiring correction are color, contrast and sharpness. Fig. 1 shows our approach of a reliable framework for an automatic tools and a simple enhancing procedure that can improve overall quality of underwater images. Each module can be considered as an independent component that can be combined to form a more complex algorithm.
Fig1. Underwater image enhancement techniques
2.4 Color Correction
Color correction is the first module from the proposed framework. This can be seen in Fig. 1. This paper focuses on unsupervised color correction. The framework is supposed to automatically and reliably remove color cast (blue predominant) in underwater images. Therefore, the solution that we have designed can be utilized by simple statistics to make color correction and prevent artifacts.
Cast removal is a primary step in achieving color correction. As most of the users take underwater images from web or digital cameras, they have only little or no knowledge about digital image processing. Therefore, an automatic color adjustment tool would be most desirable. In this module, an unsupervised algorithm which adjust the intensity of color cast (color space conversion, histogram analysis and linear transformation), makes it possible to remove this overlay color (blue predominant) in underwater images. The algorithm can be easily integrated in a more complex procedures without a major loss of computational time.
Cast is usually due to different imaging device or changes in lighting conditions. The measured tristimulus values vary for different viewing conditions. Chromatic adaptation is a human ability to balance for light source color and to preserve approximately scene color [7]. These transformations cannot be done in color balancing. The measured tristimulus values must be transformed to recover original scene appearance, under various lighting and display conditions. These transformations are called chromatic adaptation models, and are the fundamental for cast removal methods. A chromatic adaptation model does not compare the appearance attributes, such as lightness, chromatic and hue [7]. The model simply provides a transformation of tristimulus values from one viewing condition to another viewing conditions.
Most of chromatic adaptation models are based on Von Kries hypothesis, which states that chromatic adaptation is an independent gain regulation of three cone signals, through three different gain coefficients [8]:
L’ = kL L
M’ = kM M
S’ = kS S
where L, M, S represent the original tristimulus values, and kL, kM , kS represent the gain coefficients that scale the original tristimulus values into post-adaptation tristimulus values, L’, M’, S’. Adaptation models depend on the gain coefficients.
The RGB channels are often considered an approximation of L, M, S retinal wavebands [8], therefore:
R’ = kR R
G’ = kG G
B’ = kB B
The most commonly used algorithms are gray world algorithm and white balance algorithm, which means average-to-gray and normalize-to-white [9]. The gray world algorithm suggests that, given an image with a sufficient amount of color variations, the mean value of R, G, B components of an image will average out to a common gray value [9]:
kR = GrayR / Rmean
kG = GrayG / Gmean
kB = GrayB / Bmean
where Rmean, Gmean, Bmean are the averages of RGB channels, and GrayR, GrayG, GrayB represent the gray value chosen.
The basic concept of white balance algorithm is to set at white a point or a region that appears realistically white in original scene. A way to perform a white balance algorithm is to set maximum value of each channel (Rmax, Gmax, Bmax) at a reference white (WhiteR, WhiteG, WhiteB) [9]:
kR = WhiteR / Rmax
kG = WhiteG / Gmax
kB = WhiteB / Bmax
Another approach sets a potential white region at a reference white [9]:
kR = WhiteR / RaverageW
kG = WhiteG / GaverageW
kB = WhiteB / BaverageW
where RaverageW, GaverageW, BaverageW represent three RGB channels over a potential white object, which is a subset of original image.
Here we used a modified cast removal method proposed by Gasparini and Schettini [10], which is based on Von Kries hypothesis. The Von Kries coefficients, kR, kG, kB are estimated by setting at white a particular region which is called the white balance region (WB region) [10]:
kR = WhiteR / RWB
kG = WhiteG / GWB
kB = WhiteB / BWB
WB region depends on the strength of cast. The WB region represents object or group of objects that will be forced to be white. Only objects with brightness over a set limit will be accepted (by setting a threshold for example 30), because the components to be whitened must not be too dark.