Deep Reconstruction of Least Significant Bits for Bit-Depth Expansion

Deep Reconstruction of Least Significant Bits for Bit-Depth Expansion

Abstract: Bit-depth expansion (BDE) is important for displaying a low bit-depth image in a high bit-depth monitor. Current BDE algorithms often utilize traditional methods to fill the missing least significant bits and suffer from multiple kinds of rerceivable artifacts. In this paper, we present a deep residual network-based method for BDE. Based on the different properties of flat and non-flat areas, two channels are proposed to reconstruct these two kinds of areas, respectively.

Moreover, a simple yet efficient local adaptive adjustment preprocessing is presented in the flat-area-channel. By combining the benefits of both the traditional debanding strategy and network-based reconstruction, the proposed method can further promote the subjective quality of the flat area. Experimental results on several image sets demonstrate that the proposed BDE network can obtain favorable visual quality as well as decent quantitative performance.

Existing system: In some similar image reconstruction tasks, e.g., image super-resolution or denoising, learning-based methods have achieved impressive results. Especially in recent years, deep neural network (DNN)-based methods show sufficient

superiority to solve the highly non-linear problems. Many related networks are focus on HDR or ITM tasks . In , an inverse half-toning method was proposed by means of U-Net structure, and this model is also applied for image commanding scenario with very low bit-depth, such as 2 bits, 3 bits, and 4 bits. These DNNbased methods can effectively reduce the serious banding and block artifacts, but they mainly focus on the artifacts removal of HDR tasks or inverse half-toning for very low bit-depth. In this paper, we propose a deep BDE network (BDEN), which consists of a flat-area-channel and a non-flat-area channel.

Proposed system: In, an inverse half-toning method was proposed by means of U-Net structure, and this model is also applied for image companying scenario with very low bit-depth, such as 2 bits, 3 bits, and 4 bits. These DNN-based methods can effectively reduce the serious banding and block artifacts, but they mainly focus on the artifacts removal of HDR tasks or inverse half-toning for very low bit-depth. A two-stream residual network is proposed to predict the missing bits by means of extra exemplars. Because the magnitude of differences between LBD inputs and HBD labels are very small, a modified L1-loss is presented to optimize the BDEN. Furthermore, the structure of residual block of the BDEN is also adjusted for BDE scenario. The LBD image is firstly divided into flat area and non-flat areaby means of the average local value difference, so that the two channels of the BDEN can be specially trained with corresponding flat/non-flat exemplars.

Advantages: A skip connection is used to connect the begin and the end of each block. There are several kinds of residual blocks for image reconstruction tasks . The first ReLU in each block is removed as in .

Hence, this paper not only meets the current requirement of debanding but also focuses on the future demand of recovering better high-frequency details The reconstructed results are the residuals rather than the gray Values of pixels.

Hence, the last ReLU has not been applied to preserve the negative values.

That is because the nonfat area contains abundant local differences, but the flat area are flatten and with very low local differences. The deeper network does not help much for flat area reconstruction.

Disadvantages: Moreover, current BDE algorithms mainly focus on how to address the banding artifacts, but the filled (h l)-bit LSBs may be not consistent with the ground truth values. Recovering the missing LSBs is a typical illposed problem.

Traditional BDE methods tend to reduce banding effect, but often ignore whether the adjusted LSBs are consistent with the HBD ground truth. Although debanding is the main purpose in BDE problem, reconstructing better LSBs is still necessary for recovering highly-preserving details.

At the last of this subsection, we further discuss the differences between the proposed BDEN and the inverse half toning CNN (IHT-CNN) [51]. First, these two networks focus on different sides of the BDE problem. The IHT-CNN is originally designed for reverse half-toning problem, and also tested on image companding with very low bit-depths.

Modules: Debanding Preprocessing in Flat Area: As mentioned before, banding/contouring effect in flat area is the most prominent artifact of LBD image. Hence, traditional BDE methods have paid many attentions on debanding strategies. In LBD image, the values of two neighbor pixels may seem like step response curve, and lead to visually contouring. Many effective algorithms thus firstly detect the contour pixels and then apply adaptive filtering or FBMs to smoothen local gradients. These methods can make the local gradients of neighbor pixels smoother, but also blur the texture or produce crossed artifacts on some isolated noises. In this paper, we propose a simple LAA method motivated by these works. The pixels around the contours are also firstly detected in the LAA. We then adjust the value of each contour pixel by considering the different relationship between itself and its neighbors. After the local adjustment process,

the transition of local values becomes smoother, and therefore the banding and noise can be remove.

Deep BDE Network : Traditional BDE methods tend to reduce banding effect, but often ignore whether the adjusted LSBs are consistent with the HBD ground truth. Although debanding is the main purpose in BDE problem, reconstructing better LSBs is still necessary for recovering highly-preserving details. However, the LSBs are totally unknown, and thus accurate prediction of LSBs is very difficult. Fortunately, recent deep networks have strong ability in highly non-linear regression, and also have achieved promising results in many similar ill-posed tasks. Hence, we present the BDEN based on residual network to reconstruct the LSBs. In the following, we introduce in detail the architecture of the BDEN. The two channels of the BDEN have similar structure, which consists of an input layer, several residual blocks, and an output layer. The input layer converts the input image to a series of feature maps via a convolutional process.

Bit – depth expansion: Bit-depth denotes the number of bits of a quantized pixel value in an image. Bitdepth expansion (BDE) is the process of recovering a high-bit-depth (HBD) image from a low-bit depth (LBD) image. With the rapid development of the display technology, people always prefer higher definition and subtler gray-value levels.

The definition enhancement task has been well discussed by super-resolution (SR) techniques, while only few attentions are paid on the BDE problem. Although most recent images are stored and displayed in 8- bit, human visual system can perceive over 12-bit luminance levels . It can often be found that the displayed images cannot well represent the natural scene; one of the important reasons is that the bitdepth of 8-bit image is much less than that of human perception ability. Moreover, BDE techniques have been incorporated in HEVC standard to improve the video coding, and also used for accelerating the video display on mobile SoC.

Framework of the Proposed Method :

The entire framework of the proposed BDEN is illustrated. The BDEN mainly contains two channels, i.e., the flat-area-channel and the non-flat-area-channel. For nonfat area, which does not suffer from banding effects, we try to reconstruct the LSBs so that the reproduced image is consistent with the ground truth HBD image.

We thus present a deep network with modified loss to reproduce the missing highfrequency components. For flat-area-channel, the local adaptive adjustment is utilized to previously reduce the unnatural artifacts and improve the visual quality.

The adjusted result is then fed into another residual network to recover the HBD flat area. At last, the reconstructed flat area and nonfat area are combined to obtain the artifacts-free and highly preserving final results. In the following, we firstly introduce the proposed two stream network, and then describe the LAA preprocessing.