Depth Estimationand Focus Recovery

  1. Introduction
  2. Introduction

First, we will introduce the definition of depth between photography and optics. Then, we discuss the issue on focus recovery. There exist some problems in optical image processing like aberrations of camera, structure of zoom lens, blurring model and geometric optics and so on. Later, we will make a introduction to Fourier Optics and optics analyzed by the linear canonical transform (LCT).

Usually the “depth” we know means “the distance between the observer and the real object” which is an actual length measurement. However, in the photography, “depth” means “the effective focused interval”, as in Figure 1, that is also called “depth of field” (DOF). This important information is usually used for reconstructing a 3-D shape or recovering defocus images. Therefore, we should realize how important the depth estimation is. There are two methods to estimate the depth of the object, binocular and monocular. Also, there are two major works in monocular methods, depth from focus (DFF) and depth from defocus (DFD), as in Figure 2.About our proposed method on depth estimation, instead of assuming a point spread function (PSF) which denotes a blurring response of a defocused point light, we use the Gaussian input or the step input to model the defocused response.

Figure 1“Depth of field” in photography means the effective focused interval for human eyes.

Figure 2 Categories of depth estimation.

Focus recovery is a important subject in optical image processing. We can recover focus from defocused images only after we solve the depth values from them. Our proposed method on recovering focus images is to inverse and simulates the original photographic environment based on LCT (Linear Canonical Transform). Before we enter the topics of LCT, we will introduce some problems happened to camera. These problems are aberrations, structure of zoom lens and blurring model and the geometric optics.

  1. Pleliminary Works

2.1 The Common Phenomenon in Optical Systems

There are main problems of aberrations which existing on the common optical lenses. The spherical aberration and the chromatic aberration are two main subjects. Aspherical convex lens is a solution to the spherical aberration and a combination of convex lens and the concave lens is a solution to the chromatic aberration, After considering about the aberrations of the lenses, we consequently introduce the structure of a zoom lens.

All zoom lenses have the joint property is that projects the image on a fixed image plane with a variant focal lengths. When the user changes the focal length of camera shot to enlarge or reduce the image view, there should be no re-focused action. To satisfy this action, we need a camera shot that has the basic lens structure as the three groups as Figure 3. The function for the first group is to be adjusted for different distances of subjects. The function for the second group is to change the focal length with a quantity. The function for the last group is to make sure the incident light to be parallel and this gets the effects like the telescope.

Figure 3 Three functional groups for a zoom lens.

2.2 Blurring Model and The Geometric Optics

Images are typically modeled as the result of a linear operator with akernel dependingon the optical structure of the camera. An “ideal” unblurred image convolve with a operator , andget the result which is also called the radiance image in many documents:

(2.1)

where is called the point spread function (PSF), which is the response of the camera of a point light source.R is the blurring radius of the point light source decided by the camera parameters. The imaging point may be before the sensor to form a positive value R, and there may be a negative value R as it is behind the sensor as shown in Figure 4 and Figure 5.

We can find an impotant formula for this thesis:

(2.2)

(2.3)

(2.4)

where Dis the diameter of the lens aperture, F is the focal length of the lens,s denotes the distance between the lens and the CCD (Charge Couple Device, the imaging sensor) and u is the depth of the object. In this article, we use the Gaussian distribution function as PDF.

(2.5)

where is the diffusion parameter based on the radius of diffusion and the constant depends on characteristics of the cameras. Because we have to consider the aberrations, we replace Gaussian function with constant function. That is, we always have to determine where the region of the scene is before or behind the focus plane.

Figure 4Geometric optics on blurring imaging before the screen (sensor).

Figure 5Geometric optics on blurring imaging behind the screen(sensor).

2.3Introduction to Fourier Optics

In 1665, Grimaldi is the first one who declared that once the point light source

pass through an aperture and it will not only cause geometrical progressing but also the corpuscular light spreading. We call the phenomenon diffraction of light on aperture. Because the image produced by camera includes not only lenses but also aperture. In 1882, Gustav Kirchhoff proposed two important assumptions in aperture effect:

1Across the specific surface , the field distribution and its derivative according the obstacle’s unit vector n are exactly the same. Just like there is no existence of the obstacle, as Figure 6.

2The values of and are identically zero.

Figure 6 One aperture in an obstacle. is the point on the aperture surface and is the point away from the aperture.

One important principle - Huygens-Fresnel Principle, is an approximation to represent the near-field advancing wave disturbance that include the Kirchhoff’s boundary condition, as in (2.6).

(2.6)

where k denotes the wave number that is equal to and s denotes the spatial dimensions.If :

(2.7)

,and replace by and then replace by , and note that we have the finite integral value for Kirchhoff’s boundary conditions:

(2.8)

Now consider that if we have and P0 is near from the original point, that is, and . Then, we can derive the equation as in (2.9):

(2.9)

Then we can consider one condition. That is, the direction between an is close to the direction of . Therefore, we can get a approximation as in (2.10):

(2.10)

Hence, we can view equation (2.10) as a convolution form and its transfer function connecting to the Fresnel-approximation as in (2.11):

(2.11)

If there is a condition as , we have a far-field diffraction form as in (2.12) and (2.13).

(2.12)

(2.13)

where , .

About this chapter, we derive a series of steps and use complex calculate method to get the image intensity from the aperture. If the image is a huge object, we have to calculate the value of every point of light source through the aperture. Therefore, we have to use the LCT method to approximate the result. We may get a distortion image, but the distortion is in our acceptable range. Next chapter, we will introduce you how to use LCT to estimate depth and recover the focus in detail.

3. Optics Analyzed by the Linear Canonical Transform (LCT)

3.1 Introduction to LCT

LCT is good tool for analyzing and is a scalable transform connecting to lots of important kernels as Fresnel transform and fractional Fourier transform and so on. In words, LCT (defined as (3.1) (3.2) (3.3) (3.4)) is a representation of matrix which contains 4 parameters is formed from 3 freedoms as in (3.5). Commutative character does not hold in Matrix properties and so as the LCT kernel. Because the matrix has 3 degrees of freedom, that is AD-BC=1. There may be two condition if B is equal to zero or not. (as shown in (3.6))

(3.1)

where f(u) is the density function of image, and is the operator of LCT.

(3.2)

(3.3)

(3.4)

(3.5)

(3.6)

3.2Simulation on Images by LCTs

LCT is not the only one transform to simulate images but is more convenient to

control those four parameters in matrix. It is complicated to simulate images by Fresnel transform hence the LCT is utilized to solve this approximation problem. However, the simulation on the LCT is a hard work. We use the LCT kernel which is defined as in (3.7).

(3.7)

Here, we show an example of optical system simulated by LCT. From Figure 7, it is a simplest case of optical system. Where is the distance between object (Uo) and lens, is the distance between lens and sensor (Ui), f is focal length of the lens and is wavelength of the light.

Figure 7 A common single thin lens system with distance between object and lens and distance between lens and sensor.

We can easily construct the above optical system by a series of LCT parameters, a general result shown in (3.8).

(3.8)

Here, the four parameters of LCT is decided by matrix computation as shown in(3.8). Where A is , B is , C is and D is .

We then discuss a special case that is as in Figure 8, where ==. That is, a symmetric system locating at focal length of both sides.

Figure 8Special case of an optical system – Fourier transform approximation.

We can decide the four parameters of LCT from matrix computation and get the result as in (3.10) and (3.11). The result in (3.11) shows a scaled Fourier transform by this special case. That means we can construct an optical system to represent a field intensity which denotes the Fourier transform of its source intensity. Actually, we can construct almost optical phenomena through LCTs.

(3.9)

(3.10)

(3.11)

Form (3.11), we can realize LCT can present any scaling function based on the equation (3.7). Table 1 show the relationship between some common function representation with LCT transforming parameters.

Characteristics / Function representation / Transforming parameters
Chirp multiplication / /
Chirp convolution / /
Fractional Fourier transform / /
Fourier transform / /
Scaling / /

Table 1 Optical analyzed functions approximated by the LCT

In (3.7), we found some problem at the term when B is not zero. We consider this term as a kernel of the fast Fourier transform (FFT),, because the two terms are similar. Where m is the output axis, n is the input axis and N denotes the FFT point number. However, we will encounter some problems by using the kernel .Here we list two problems when we simulate optical systems by LCT with the kernel of this specific term:

In (3.8), .B is always a tiny number in the real imaging environment. However, the role of B is as the same as N in the FFT. In the FFT, the point number N must be an integer, hence the value of B is limited. This is because that depth value , focal length f and imaging distance s can not be free decided.

Because of the small value of B, we must have higher resolution u and as in (3.9) and we will have large computation.

In order to solve these problems, we have to discuss the definition of the LCT and we find the the main problem is due to the small value of B. One of the solution is matrix decomposition as in (3.12).

(3.12)

From (3.12), we know that is LCT parameters representation of Fourier transform, so we can do the FFT on the input signal first then do the LCT with parameters as in (3.13)

(3.13)

In (3.13), we now have the value of -A to dominate the resolution problem of the LCT and the value of –A in is . Obviously, the value of is much bigger than . In this way, the resolution of u and is also not necessarilyhuge.

3.3Implementation of all kinds of Optical system

With the introduction of section 3.1, we have some basic concept of the relationship between LCT matrix and simple optical system. However, it is not enough for us to simulate all kinds of optical system. We found that three lenses optical system can generalize almost case which might happen in optical system, and let us consider the following instrument as shown in Figure 9:

Figure 9 The instrument with three lenses.

According by equation (3.8), we can know the four parameters of the LCT of this optical system are as in (3.14):

(3.14)

, the term is composed of several matrixes as in (3.15):

(3.15)

, from (3.14), we can derive the equation (3.16):

(3.16)

According to section 3.1, we know AD-BC=1, hence, we have three free degrees in the four parameters of LCT. From (3.16), we can transfer one free degree of the LCT parameters to the focal length f3, but we still have two free degrees which have to be solved. Therefore, we must derive the relationship between f1, f2, A and B. The derivation is shown below:

(3.17)

(3.18)

(3.19)

We only focus on the indices of containing A or B. Hence,

(3.20)

, and f1 is found as in (3.21)

(3.21)

, and with the same derivation, we can find f2 as in (3.22):

(3.22)

By equation (3.16), (3.21) and (3.22), we prove an optical system like the illustration as in Figure 8 can be fully satisfy the LCT parameters. That is, we can only use three lenses to simulate all possible optical systems. Then, the simulations of LCT parameters can be generalized through this analyzed method.

3.4Proposed DFD Method with LCT Blurring Models on The

Gaussian Function

We continue to discuss the Linear Canonical transform (LCT) in this chapter. First, we consider a point spread function as the following form:

(3.23)

Actually an ideal PSF should be a uniform distribution as in the geometric optics, that is, no blurring would happen in the images. However, the Gaussian form is considered of the free space distortions in the Fourier optics. Because the LCT can describe all of the wave propagations, we combine the derivation on the scalable LCT with the PSF. According to its four free parameters, we can model any situations of the PSFs by the LCTs. That is, we can get a solution similar to a Gaussian form but is with four adjustable parameters. Hence, we could use final solution with inverse parameters to recover the input source.

Here we will show the steps to solve the amplitude of the LCT of g(t) as in (3.24) to (3.27):

(3.24)

(3.25)

(3.26)

(3.27)

We have introduced DFD briefly in previous chapter. Here, we will continue to discuss how to use DFD method to find depth. The main work for DFD methods is to compare two images with different defocus degrees. By using the relation between the bluing variation and the blurring radius to discover the depth cues.

We want to apply Guassian blurring model with LCTs. Note that the key point concentrates on the comparing with two images with variations.Assume there are two point light sources, with different defocus degrees as in (3.28).

(3.28)

where

,

we take the naturallogarithm,

(3.29)

According to the optical system as in (3.8), we replace the parameters A and B :

(3.30)

where

(3.31)

While getting two different values for s and, we have the compared relations as in (3.32) and (3.33).

(3.32)

(3.33)

Note that inside the camera structure, the sensor plane is put with a distance that is equal to focal lengths. Due to this reason we generally choose . Through comparing, the depth can be solved by this LCT model:

(3.34)

Hence, through a series of derivation, we can get the value of depth and more over we can do focus recovery.

4 Fiber-optic Communication

4.1 Introduction to fiber-optic communication

Fiber-optic communication is a method of transmitting information from one place to another by sending pulses of light through an optical fiber. The light can carry information after modulated. Fromthe 1970s, fiber-optic communication systems have revolutionized the telecommunications industry and have played animportant role in the digital of the Information Age. With theadvantages of large capacity and good encryption, optical fibers have become the most common method of cable communication. We put the message which produced by sender into transmitter, modulate the message by appropriate carrier, transmit the modulated signal into receiver,and demodulate the signal to original message.

The process of communicating using fiber-optics involves the following basic steps:

  1. Creating the optical signal involving the use a transmitter
  2. Relaying the signal along the fiber, ensuring that the signal does not become too distorted or weak
  3. Receiving the optical signal, and converting it into an electrical signal.

4.2 Applications of fiber-optic communication

Optical fiber is used to transmit telephone signals, Internet communication, and cable television signalsby many telecommunications companies. Sometimes, one fiber can carry all signals mentioned above. Because of much lower attenuation and interference, optical fiber has large advantages over traditional copper wire in long-distance and high-demand applications. However, infrastructure development in cities was relatively difficult and time-consuming, and fiber-optic systems were complex and expensive to install and carry out. Due to these difficulties, in the early time fiber-optic communication systems had been installed in long-distance applications, where they can be taken good advantage of their full transmission capacity and can offset the increased cost.

Since the year 2000, the prices for fiber-optic communications have dropped dramatically. Today, the scale of fiber-based network is equal to copper-based network. Since 1990, when optical-amplification systems became commercially available, many long-distance fiber-optics communications come true. By 2002, an intercontinental network of 250,000 km of communications cable with a capacity of 2.56 Tb/s was completed, and telecommunications reports show that network capacity has increased dramatically since 2002.

4.3 Comparison with electrical transmission

For a specific communication system, there are several things to decide to use optic fiber or copper. Optical fiber is often used in long-distance and high-bandwidth application, because of its’ advantage of high transmission capacity and low interference. Another important advantage of optic fiber is the fibers can combine together even crossing a very long distance and fibers do not generate cross-talk each other, and this is on the other way of copper-based communication.

However, for short-distance and low-bandwidth communication, there are some advantages below by using electrical signals:

  • Lower material cost, where large quantities are not required
  • Lower cost of transmitters and receivers
  • Capability to carry electrical power as well as signals (in specially-designed cables)
  • Ease of operating transducers in linear mode.
  • Optical Fibers are more difficult and expensive to splice.
  • At higher optical powers, Optical Fibers are susceptible to fiber fuse wherein a bit too much light meeting with an imperfection can destroy several meters per second. The installation of fiber fuse detection circuity at the transmitter can break the circuit and halt the failure to minimize damage.

In some specific low-bandwidth case, optic fiber has its own advantage, as shown below:

  • Immunity to electromagnetic interference, including nuclear electromagnetic pulses (although fiber can be damaged by alpha and beta radiation).
  • High electrical resistance, making it safe to use near high-voltage equipment or between areas with different earth potentials.
  • Lighter weight—important, for example, in aircraft.
  • No sparks—important in flammable or explosive gas environments.
  • Not electromagnetically radiating, and difficult to tap without disrupting the signal—important in high-security environments.
  • Much smaller cable size—important where pathway is limited, such as networking an existing building, where smaller channels can be drilled and space can be saved in existing cable ducts and trays.

Conclusion