Deutsche Forschungsvereinigung für Meß-, Regelungs- und Systemtechnik e.V.
February 2008
Methods for selectively darken digital images
Sorin Ivascu, Axel Gräser
Friedrich Wilhelm Bessel Institute (FWBI)
Wilseder Berg Str. 5
28329 Bremen
Germany
Phone 0049 421 218 7448, Email {ivascu, ag}@iat.uni-bremen.de
Abstract
In today’s industrial processes, the use of high light gradient images is often required. The welding observation making no exception of this, is taking advantage of the state of the art high dynamic range (HDR) cameras, but nevertheless, such cameras can only provide very low contrast images. In order to improve the situation, new methods were developed, and in this respect, this paper is presenting two ways of enhancing the contrast of such HDR images.
A first approach is focusing on the image acquisition procedure, and interferes with that in the way described below.
A selective filter, consisting in a graphical Liquid Cristal Display (LCD) is placed in front of the HDR camera lenses. The pixels on the LCD are all set on the state with the higher light transmissivity. When the welding process starts, a software algorithm processes the video signal of the camera, and detecting the direction of the welding flare, it switches the state of the pixels which are overlapping on the flare, to the state with the lowest light transmissivity. The result consists in the reduction of the amount of light coming from the flare, thus enabling the HDR camera to capture an image with a lower dynamic range, which, in other words, translates into obtaining a better contrast.
The second approach deals with the software merging of two images acquired with different camera settings. The first image is taken with a lower camera offset, having the details around the welding flare better evidenced. On the other hand, the second image is displaying more details of the surrounding environment, further away from the flare, by setting the camera offset to a higher level. Both images are read pixel by pixel and summed up together, using a mathematical parameterized pondering function. The result is an image which preserves the amount of details from both source images, containing details of the flare area, while preserving a good lightening of the environment areas.
The paper presents the recent advances and the intermediate results of the mentioned methods.
Introduction
Despite the increasing performance of digital cameras, there are still industrial areas which stretch their needs of digital imaging, beyond the current state of the art in image acquisition and processing. Such an industrial domain is represented by the welding observation. Here, we encounter the need of acquiring images with a higher dynamic range, as the high level of incoming light, “blinds” the camera, limiting the observer to see either the areas very close to the welding flame in a dark environment, or a saturated – completely white – flame surrounding area in a normally lit environment (Figure 1).
a.) welding flame visible / environment not visible b.) welding flame area is saturated / environment visible
Figure 1: The camera is saturated by the high level of incoming light
The High Dynamic Range (HDR) cameras as the ones used in welding observation [1], can overcome this problem of high brightness saturating the image, by increasing the scale on which the pixels’ values may be defined on, namely, the range of values between bright and dark areas. But this comes with a cost. And that is a poor contrast achieved in the resulting image (Figure 2) and though an important number of methods were developed in order to solve this problem[2][3][4][5][6], in this paper, we will present solutions intended to improve the situation before running into such a problem.
So, now, we have two situations in the capturing of a welding scene. One consists in using regular cameras which are limited by the fact that are saturated when the bright welding arc switches on, and the second situation in which a HDR camera, provides the whole brightness scale of the scene, but with a poor contrast.
Figure 2: Though the brightness levels cover both the welding arc and the environment, the contrast is low
Selective filter
Both situations presented above, have in common the fact that the brightness level of a part of the scene - the welding arc - is very high, when compared with the rest of the scene. In this respect, it would be a promising workaround, to minimize this gap in brightness values, which may enable on one hand, the regular cameras to avoid saturation, and on the other hand, the HDR cameras to provide images with a better contrast. In order to achieve this, a selective filtering of the incoming light is needed.
Again, the problem has two sides. As we are in the digital imaging domain, the selective filter can be implemented by hardware or by software means. Therefore, to be consistent in finding and comparing solutions for both sides, in the next pages, we will present one implementation of both hardware and software filters.
Hardware Selective Filter – Graphical Liquid Cristal Display (LCD) in front of the camera
A commercial Super Twisted Nematic (STN) LCD[7], driven using an I2C [8] interface, is connected to a computer and mounted as a see - through filter in front of a HDR camera (Figure 3). As previously described, the idea behind the setup is to limit the amount of light passing towards the camera, therefore in order to achieve this, the LCD is driven as described in the following 4 steps (Figure 4).
Figure 3. LCD-I2C setup in front of of the HDR camera
Step 1. The welding arc triggers on
Step 2. High intensity light is detected by the computer (using the camera software)
Step 3. The welding arc position is calculated and after performing transformations between camera and LCD systems of coordinates (using the data from a previous initial calibration), the coordinates are sent to the I2C board
Step 4. The LCD switches its pixels to the “on” state (corresponding to black) in a form of a square overlapping the received coordinates
This process is continuously repeated as the welding arc is changing its position and state within the scene, and in this way, the LCD is refreshing permanently the position of the square.
Figure 4. The idea
Measurements have been taken, in order to observe the level of incoming light, in different points within the image captured by the camera and using up to 5 overlapping LCDs. The chosen points correspond to the corners of the image, as reference points of the environment light, and the brightest point in the scene corresponding to the welding arc. Though the results (Figure 5) are not very promising, as for example the selective shading is not highly increased with the number of LCDs used, it can be observed, that the principle works. The selective filter creates a slight darkening of the brightest spot, thus allowing the camera to take images with a better contrast. The limitations in achieving a stronger response, imposed by the internal structure of the STN LCD, can be eliminated using a Twisted Nematic (TN) [9] LCD. Such a device, can filter the light that passes through its molecules up to 3 times more compared with the STN LCD, and this represents a next step in our research, which is now in development.
Figure 5. Measurements using the STN LCD
Software Selective Filter – Pondered merging of two images
The second approach consisting in merging two images taken with different exposure settings is based on the following principle. The HDR camera is set up to capture every even image with a lower gain in brightness and every odd one with a higher gain. These two images will be called I1 and I2. A mathematical function chosen to model a pondering fraction (pf) of both I1 and I2 in composing the image resulted after the merging (I3), is implemented in software and applied for each pixel of I1 and I2. A simplified equation of the pondering process has the following form:
, where (1)
After taking into consideration several mathematical functions, we have decided that the Bezier curve is the most suitable for our pondering purpose (Figure 6). It is a parametric function of four points; two endpoints and two control points. The curve connects the endpoints, but doesn’t necessarily touch the control points. The general form of the Bezier equation, which describes each point on the curve as a function of time, is:
P(t) = w1(t)P1 + w2(t)P2 + w3(t)P3 + w4(t)P4 (2)
where P1 and P4 are the endpoints, P2 and P3 are the control points, and the w1..w4 are weighting functions, which blend the four points to produce the curve.
Figure 6. Bezier curve (in red) and a pixel value pondering example
The single parameter t represents time, and varies from 0 to 1. The curve touches each endpoint, so at t = 0 the first weighting function is 1 and all others are 0 (i.e., the initial point on the curve is the first endpoint). Likewise, at t=1, the fourth weighting function is 1 and the rest are 0. Between 0 and 1, we can use the values in the pondering formula defined above (1).
The software implementation is flexible, allowing us to change the function parameters’ values (Figure 7) and see in real time, the change reflected in the resulted image.
Figure 7. The parameters of the Bezier curve triggered through the software interface
It can be observed in Figure 8, that I3 contains more details both from the welding arc area and the environment.
I1 I2 I3
Figure 8. The source images and the result in a welding simulation
As the setup was developed in an experimental way, and the images were acquired manually, a further improvement of I3 can be achieved if both the image acquisition and pondering will be implemented and triggered in the camera software, and performed automatically.
Conclusions and overview
This paper has presented two solutions of improving the image contrast and details level, with direct applicability, and in relation, with the welding observation. While the selective filter still needs improvements, the pondered image merging, already provides a significant improvement regarding the amount of image details. The software calculus is performed quickly enough, for it to be implemented inside the camera image acquisition workflow.
Coming back to the selective filter, we are looking forward to use a TN LCD, which, considering the figures obtained in Figure 5 above, and taking into account it’s considerably higher level of shading, would lead to a significantly better result.
To conclude, a practical utility of these methods should be suggested. The most obvious among others, consists in building the state of the art augmented reality welding helms of tomorrow, which can have immediate application in welding training procedures, quality control or simply better welding scene observation in industrial welding processes.
References
1. Aiteanu D., Gräser A.: “Computer-Aided Manual Welding Using an Augmented Reality Supervisor”, Sheet Metal Conference XII, Livonia, MI, USA (May 9-12, 2006)
2. Whatmough, R: “Viewing low contrast, high dynamic range images with MUSCLE”, Digital Image Computing: Techniques and Applications, 6 - 8 December 2005, Cairns, Australia
3. Hayashi Hiroki, Saito Fumihiko: “Contrast Improvement for Displayed Color Image Based on Color Difference”, Papers of Technical Meeting on Information Processing, IEE Japan, 2007
4. Leyvi E. Dalal S. M.: “Method of improving the perceptual contrast of displayed images”, Publication number WO/2004/049293, 10.06.2004
5. Oho E., Peters K.R.: “Practical Methods for Digital Image Enhancement in SEM”, / Electron Mlcrosc 43: 299-306 (1994), Vol. 43, No. 5, 1994
6. Martyshevsky Y.V., Sharopin Y.B.: “Low contrast images segmentation based on Fourier spectrum approximation”, 5th International Conference on Actual Problems of Electronic Instrument Engineering Proceedings Volume 1, APEIE-2000
7. Inverse twisted and super-twisted nematic liquid crystal device – United States Patent 5701168, 23.12.1997
8. Philips Semiconductors: “The I2C-Bus specification”, document order number: 9398 393 40011
9. http://plc.case.edu/tutorial/enhanced/files/lcd/tn/tn.htm
Bibliography
The user might find the following resources useful for additional information:
Smith D. W.: “Bezier curve ahead”, Mac Tech, Volume 5, Issue number 1
David X. D. Yang, Abbas El Gamal, Boyd Fowler, and Hui Tian: “A 640 512 CMOS Image Sensor with Ultrawide Dynamic Range Floating-Point Pixel-Level ADC”, IEEE Journal of Solid-State Circuits, Vol. 34, No. 12, December 1999
Reinhard E., Ward G., Pattanaik S., Devebec P.: “High dynamic range imaging – acquisition, display and image-based lightning”, Morgan Kaufmann Publishers. December 2005
Hoefflinger B.: “High-Dynamic-Range (HDR) Vision”, Springer Series in Advanced Microelectronics 26, Springer Berlin Heidelberg 2007
Gonzalez R. C., Woods R. E.: “Digital Image Processing”, Prentice Hall 2002
Debevec P.E., Malik J.: “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH August 1997
1