Image compression with inpainting

Image Compression with Inpainting Veeramma Yatnalli DrK L Sudha Assistant Professor,Dept.of E & C JSS Academy of Technical Education I Bangalore,India veeramma71 @gmail.com Professor,Dept.of E & C Dayananda Sagar College of Engineering2 Bangalore,India klsudhal @rediffmail.com Abstract-In common wireless scenario, the data needs to be efficiently compressed by utilizing the channel bandwidth and thus to increase the data bit rate. The image is transmitted over the channel block by block. We can enhance the compression performance by intelligently choosing blocks from the image with non-relevant information in it. By removing these small blocks, we can reconstruct the lost data using correlation between the lost blocks and its neighbors. This is automatically done in a fast way by using the boundary information, thereby allowing to simultaneously fill-in numerous regions containing completely different structures and surrounding backgrounds. This paper addresses the issue of performing inpainting on the decompressed image to fill in the missing information. The effectiveness of this approach is demonstrated with various compression factors. Quantitative analysis of the desired compression ratio and the quality of the restored image is conducted. Keywords-Compression ,inpainting I INTRODUCTION There have been significant advancements in processing of still image, video, graphics, speech, and audio signals through digital computers in order to accomplish different application challenges. Transmission of multimedia data such as image requires considerably higher bandwidth requirement than text. An image stored in an uncompressed file format, such as the popular BMP format, can be huge. An image with a pixel resolution of 640 by 480 pixels and 24-bit color resolution will take up 640 * 480 * 24/8 = 921,600 bytes in an uncompressed format.Therefore, data storage capacity and data transmission bandwidth are the present critical requirements to handle this pervasive multimedia data. To accomplish this, it is essential that the data representation and encoding of multimedia data be standard across different platforms and applications. Data Compression is one of the technologies for each of the aspect of this multimedia revolution. As a result, development of efficient image compression techniques continues to be an important challenge to us, both in academia and in industry. Image Compression is an important component of the solutions available for creating image file sizes of manageable and transmittable dimensions. Image data compression becomes still more important because of the fact that the transfer of uncompressed graphical data requires far more bandwidth and data transfer rate. Performances are important in the selection of the compression/decompression technique to be employed. The term data compression also refers to the process of reducing the amount of data required to represent a given quantity of information. Now, a particular piece of information may contain some portion which is not important and can be comfortably removed. All such data is referred to as redundant data. High compression ratio can be achieved by eliminating the redundancies, but at the cost of some information loss. Up to now, great achievements have been made in image compression. State-of-the art methods such as JPEG and JPEG2000 efficiently exploit statistical redundancies among pixels and achieve high compression ratios. Information loss causes decrease in the quality of reconstructed images, especially at high compression ratios. We can enhance the compression performance by intelligently choosing blocks from the image with non­ relevant information in it. Thus the image is compressed efficiently to utilize the bandwidth during transmission. The compressed image is decompressed back and image's blocks are retrieved by image inpainting. In our proposed work, we have selected a portion of an image characterized by uniform gray level distribution. Any assistant information is not sent along with the image transmitted as indicated from the survey carried out. However, we have attempted to recover the original image with the surrounding information available in the received image. In our proposed approach, we have used the combination of DCT and image inpainting in order to improve the compression performance by an amount greater than the value achieved with DCT alone. For this method to be successful, we have chosen the blocks that can be easily restored. Since inpainting is employed in compression, it is desirable to drop as many regions as possible to save coding bits while still maintaining good visual quality.The results obtained from the approach are analyzed and compared with the results obtained using DCT alone.The performance parameters are tested with various compression ratios. II RELATED WORK Many new compression techniques have been developed by utilizing different features within images to achieve high coding performance. Among those, an image inpainting is a practical approach to be employed in image compression. 978-1-4673-5604-6/12/$31.00 ©2012 IEEE 000158 Inpainting, the technique of modifying an image in an undetectable form, is as ancient as art itself. The applications of inpainting are ranging from the restoration of damaged pictures to the removal/replacement of selected objects. Image inpainting is a technique that provides a means to repair the damaged regions of an image so that the image appears normal to the not familiar observers. Its applications include removing scratches in old photos, repairing lost blocks in unreliably transmitted images, removing undesired objects from an image, or even creating artistic effects. Some aspects of image inpainting were fIrst introduced by Bertalmio et al. [2]. They fIrst gathered information over the boundary of the inpainting domain. Then, image smoothness, estimated by Laplacian operator is propagated along isophote (i.e. a line whose points have the same gray value) directions with anisotropic diffusion.Since the missing or damaged areas cannot be simply classifIed objectively, the user needs to identify them. This specifIed region is called inpainting domain. Since inpainting faithfully reproduces blocks we can voluntarily remove some sub-blocks of the image containing structure and texture, prior to compression[3]. Blocks that are to be removed will be classifIed as either structure or texture depending on the characteristics of their adjacent blocks. Such blocks are intentionally removed and we compress the image using lossy JPEG. At the decoder side, each lost block will be recovered by using an appropriate inpainting method. In other words, when the image is decompressed, an inpainting algorithm is used to reconstruct the "lost" blocks.If the lost block contained structure, it is reconstructed using an image inpainting algorithm, while texture synthesis is used for the textured blocks. The switch between the two schemes is done in a fully automatic fashion based on the information available with the surrounding blocks. Inspired by this work, a compression technique[4] is proposed which aims at visual quality instead of pixel wise fIdelity. In this work, an original image is analyzed carefully at the encoder and portions of the image are intentionally skipped.An edge information is extracted from these skipped regions and sent to the decoder as assistant information in the compressed fashion. An assistant information delivered plays a key role, because it guides image inpainting algorithm to accurately restore these regions at the decoder side. This edge-based inpainting algorithm proposed for image restoration integrates pixel-wise structure propagation and patch-wise texture synthesis. Evaluations have been compared with baseline JPEG and standard MPEG-4 AVC/H.264. This method provides 33% to 44% bit saving at high visual quality in comparison with MPEG-4 AVC/H.264. But this scheme fails to restore large dropped regions because of limited correlation between available pixels and pixels of dropped region. Moreover, this technique uses predefIned image models to derive skipped information at decoder, where the variation in the image region is not considered. Instead of extracting assistant information from the image, a paper proposes a technique [5], which operates on the image patches. A subset of image patches can be well inferred from the others; therefore, they can be removed at encoder only to be restored at decoder. The assistant parameter used here is actually the difference between the removed and preserved patches. Meanwhile, assistant information is transmitted for the restoration, which actually encodes the similarity between removed and preserved patches. This approach is effective only for texture images. Structure-aware inpainting (SAl) method [6] is developed to restore the skipped structural regions by taking advantage of the available portion of the decoded image. In this scheme, a binary structure map of the selected "unknown" structural regions is extracted and compressed into the generated bit­ stream. Accordingly, a structure-aware inpainting is developed to recover those regions from the decoded structure map and the "known" regions at the decoder. Based on this, this scheme proposes a new image coding scheme in which a number of regions, which can be recovered through SAl at decoder side but are skipped in encoding to reduce the coded bit rate. III ApPROACH In our approach, the algorithm used for image compression using inpainting is consisting of the following steps: 1) Select the blocks to be removed. 2) A common framework in almost all lossy image compression methods is image transformation followed by quantization. This is accomplished by applying DCT and selecting the quantization table according to the compression ratio. 3) Fill in removed blocks using Image Inpainting Technique At the encoder side, steps 1 and 2 are followed. At the decoder side, step 3 is followed. The steps are summarized in Fig 2. The above three algorithms are described in the following order. 1. Block Removal:Our goal is to remove as many blocks as possible and still recover the image with good perceptual quality. The blocks in smooth area are removed so that they can be restored back using structure inpainting. Such blocks are not noticeable even if we simply fIll in DC value. We can apply structure inpainting if a block falls into one of the two cases: i)When a block does not contain 'strong edges', and ii)When it is not composed of fIne repetitive patterns. Sharp (strong) edges are critical when human recognize the shape of an object. 2.Discrete Cosine Transform: The widely used JPEG image compression standard use DCT (Discrete Cosine Transform). It has excellent compaction for highly correlated data.DCT has fIxed basis images. DCT gives good compromise between information packing ability and computational complexity. During a step 000159 called quantization where part of compression actually occurs, the less important frequencies are discarded, hence the use of term lossy. Then, only the most important frequencies that remain are used to retrieve the image in the decompression process. As a result, reconstructed image contain some distortion. The following is the general overview of the JPEG process using DCT as shown in Figl . i) The input image is broken into 8x8 blocks. ii) Working from left to right, top to bottom, Forward iii)Discrete cosine transform (FDCT) is applied to each block by multiplying with DCT matrix T (shown below) on the left and transpose of DCT matrix on its right. iv) Each block is compressed through quantization. The array of compressed blocks that constitute the image is stored in a drastically reduced amount of space. v) The image is reconstructed by a process that uses Inverse Discrete Cosine Transform (IDCT). propagated inside. This is automatically done (and in a fast way), thereby allowing to simultaneously fill-in numerous regions containing completely different structures and surrounding backgrounds. For this method to be successful, it is important to choose and erase the block that can be easily restored. C H image Redundant Information Removed Compre ssion A N ..... N E L Decompres sion lnpainting. Reconstructed [mage Fig 2 Compression achieved through inpainting. Fig I. The basic encoder and decoder structure used in JPEG standard The formula given below is used to in the computation of FDCT and IDCT coefficients each sub block: 1 T·· l,) = { $N if i {N -cos [(2j+l)i1T] 2N = The image processor can be looked upon as a function f as follows: f: -u that is u= feu 0) ' Now, let n denote the set of pixels (the region) of the image to be inpainted. Let an denote the one pixel wide boundary of n so that an c n (the set of pixels to be inpainted) as shown in Fig 3. 0 if i > 0 (1) For an 8x8 block it results in this matrix: Fig 3 The image, the region The formulae used to compute FDCT and IDCT are as shown respectively; D=TxXxT Here, D is the 8x8 sub-blocks containing DCT coefficients X is the 8x8 sub-block of the original image matrix . A=T'xSxT Here, A is the sub-block containing IDCT coefficients and S is the sub-block containing quantized coefficients. 3.Image inpainting :Digital Inpainting helps to perform inpainting digitally through image processing in some sense. Thereby, automating the process and reducing the interaction required by the user. The only interaction required by the user is the selection of the region of the image to be removed [7]. After the user selects the regions to be restored, the algorithm automatically fills-in these regions with information surrounding them. The fill-in is done in such a way that isophote lines arriving at the regions' boundaries are n to be inpainted and its boundary an. Partial differential equations (PDEs) are used for a large variety of image processing tasks, and recently, they have been proposed for so called inpainting techniques, which use PDE-based interpolation methods to fill in missing image data from a given inpainting mask. The following steps describe the general solution to the problem: STEP 1: SPECIFY n STEP 2: an = THE BOUNDARY OF n STEP 3: INITIALIZE n STEP 4: FOR ALL PIXELS X, YEn INPAINT X, Y IN n BASED ON INFORMATION in an The fundamental partial differentiation equations used are given from equation 2 through equation 5.Specifically every pixel (i,j) in the hole, is updated iteratively as follows: 000160 In+1Ci,j) = InCi,j) + b:.tIf(i,j), \fCi,j) En (2) where M is a time step set equal to 0.1 and [f(i,j) is the update at pixel(i,j) given by If(i,j) Ln(i,j) = = om (i,j). fiii(i,j) I;x(i,j) + I�y(i,j) (3) (4) Wi(i,j) V.L(i,j) [n(i,j) Where I is the image = 8L(x,y) = (5) (L(x + l,y) -L(x -l,y),L(x,y + 1) -L(x,y -1)) Ln(i,j) is the information that we want to propagate is the image vector in orthogonal direction [n+1(i,j)) is an improved version of [n(i,j) /:::,.t is the rate of improvement. The super index n denotes the inpainting "time" and i,j are the pixel coordinates. V (i,j) is the information that we want to propagate. Since we want the propagation to be smooth, Ln(i,j)should be an image smoothness estimator. For this purpose we may use a simple discrete implementation of the Laplacian: Ln(i,j) [�x(i,j) + [�yCi,j), where subscripts represent derivatives in this case. 8V (i,j) is a measure of the change in the information V (i,j) At steady state, that is, when the algorithm converges, [n+1(i,j) [n(i,j), we have that 8V (i,j) . Wi (i,j) 0 , meaning exactly that the information has been propagated in the desired direction. = = = IV PERFORMANCE EVALUATION MEASUREMENT OF IMAGE QUALITY: The design of an imaging system should begin with an analysis of the physical characteristics of the originals and the means through which the images may be generated. A detailed examination of some of the originals may be necessary to determine the level of detail within the original that might be meaningful for a researcher or scholar. Generally image quality is assessed from Quality Assessment Parameters. The two commonly used measures for quantifying the error between images are Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). Mean Square Error: MSE between the original image I and the reconstructed image I' is given by M MSE = PSNR = 10 [2552] iog10 MSE (7) V RESULTS V..L is orthogonal gradient N Peak Signal to Noise Ratio (PSNR): The PSNR between two 8-bit images (in dB) can be obtained using the above formula, since it is a logarithmic measure, and our brains seem to respond logarithmically to intensity. Increasing PSNR represents increasing fidelity of compression. Generally, when the PSNR is 40 dB or larger, the two images are virtually distinguishable by human observers. The sample image of cameraman is used for the simulation. An increase in the compression performance can be obtained by removing the non significant information in the image prior to compression and then performing image Inpainting at the receiver. The combination of DCT and Inapinting is applied to cameraman.tif image (512x512) with various compression ratios. In the first step of our approach, the image is compressed for various compression factors. Fig 4 illustrates the resulting reconstructed images with compression factors from 2: 1 to 32: 1. Redundant information is removed in the form of blocks in smooth area. It is shown in Fig 5.2, wherein the blocks in smooth area are removed so that they can be effectively recovered back using Inpainting at the decoder side. Fig 5.3 is a simple case of recovering the image without the DCT compression employed at the encoder side. Fig 5.4 is an image reconstructed with a compression factor of 2 which demonstrates that the image can be recovered without any error. In addition, the amount of Compression achieved by the combination of DCT and Inpainting is better than that of using DCT only. This is indicated in table 1 which illustrates that the amount of compression achieved is more than the compression value specified. Fig 5.4 through Fig 5.7 shows the results for various compression Ratios obtained with the combination of Inapinting and DCT. These figures are the results obtained with various compression ratios applied to cameraman image. However, in reconstructing the image with a compression factor of 4 and higher value, we notice that a kind of distortion appearing in the missing region as shown in Fig 5.6 due to higher compression factors. Table 1 illustrates the various quality assessment parameters implemented using DCT with Inpainting. From table 1, we observe that the compression ratio achieved using DCT is slightly greater than the compression using the combination of DCT and Inpainting. N :N L L [I(x,y) -1'(x,y)]Z y=l x=l (6) It is very useful measure as it gives an average value of energy lost in the lossy compression of the original image I.A very small MSE indicates that the image is very close to original. 000161 I of Quality assessment table for an image using OCT and Inpianting Table lReeonClruelecllmage Originaili mage �--- -------� �------� 2:1 Original Image Recol'IstruCI.Bd Ima!;le Re-conS\!ruC:led Imag1e 8: 1 Recolls1ructed Image V CONCLUSION A technique for the filling-in of mlssmg blocks in wireless transmission of compressed images is discussed. Here we are dealing with filling-in of missing blocks which are intentionally removed before the transmission over the channel. As long as the features in the image are not completely lost, they can be satisfactorily reconstructed using computationally efficient image inpainting algorithms. In this way, the channel bandwidth is utilized effectively by sending only limited information which is sufficient to reconstruct the image at receiver. This eliminates the need for retransmission of lost blocks. When the image resolution is increased, the quality of reconst,f� tion improves and a retransmission request is rarely required, resulting in a better effective data transmission rate. The lost data can be effectively recovered by using texture synthesis algorithms if the information embraces some sharp features. The future work which could be done in this field is to improve the performance by utilizing the bandwidth efficiently. Still more work need to be done in future to improve the image 16: 1 Fig5.lOriginal image Fig5.2Received image Fig 5.3lnpainted Image without r",cn""p«i"" Fig 4 Reconstructed images of dimension 256x256 using OCT Fig 5.4 Inpainted Image Compression OCT and [npainting Fig 5.5 Inpainted Image for for Compression Factor of 2 Compression Factor of 4 ��------------� Ratio MSE PSNR Time Compression (dB) (in sec) Achieved 2:1 3.86 42.27 5.81 1.9961 4:1 20.23 35.10 4.55 4.0124 8:1 78.47 29.18 4.12 8.0223 16:1 192.49 25.29 4.672 16.0331 32:1 1.5015e+003 16.36 4.2969 33.4750 for Compression Factor of 8 Compression Factor of 16 000162 REFERENCES: [1] G. K.Wallace, "The JPEG still picture compression [2] standard," Commun.ACM, vol. 34, no. 4, pp. 30-44, 1991. M. Bertalmio, G. Sapiro, V. Caselles and C. Ballester, "Image Inpainting", Proceedings ofS1GGRAPH 2000 , New Orleans, USA, pp 417-424, July 2000. [3] Shantanu D. Rane, GuillermoSapiro, and Marcelo Bertalmio"Structure and Texture Filling-In of Missing Image Blocks in Wireless Transmission and mpression Applications"IEEE transactions on image processing March 2003 [4] Dong Liu, Xiaoyan Sun, Feng Wu," Edge-based inpainting and texture synthesis for image compression" IEEE 2007,TCME 2007 [5] Dong Liu, Xiaoyan Sun, Feng Wu," Edge-based inpainting and texture synthesis for image compression" IEEE 2008, lCME 2008 [6] Chen Wang, XiaoyanSun, Feng Wu and Hongkai Xiong "Image Compression with Structure-Aware Inpainting", IEEE 2006,ISCAS 2006 [7] Samuel Adolfson, "Algorithms Regarding Automatic Retouching of User Selected Regions in Digital Images"Master's Degree Project, Royal Institute of Technology , Stockholm,Sweden, 2004. 000163