Multi-frame Image Super-resolution Reconstruction Using Multi-grained Cascade Forest

Yaming Wang, Zhikang Luo, Wenqing Huang


Super-resolution image reconstruction utilizes two algorithms, where one is for single-frame image reconstruction, and the other is for multi-frame image reconstruction. Single-frame image reconstruction generally takes the first degradation and is followed by reconstruction, which essentially creates a problem of insufficient characterization. Multi-frame images provide additional information for image reconstruction relative to single frame images due to the slight differences between sequential frames. However, the existing super-resolution algorithm for multi-frame images do not take advantage of this key factor, either because of loose structure and complexity, or because the individual frames are restored poorly. This paper proposes a new SR reconstruction algorithm for images using Multi-grained Cascade Forest. Multi-frame image reconstruction is processed sequentially. Firstly, the image registration algorithm uses a convolutional neural network to register low-resolution image sequences, and then the images are reconstructed after registration by the Multi-grained Cascade Forest reconstruction algorithm. Finally, the reconstructed images are fused. The optimal algorithm is selected for each step  to get the most out of the details and tightly connect the internal logic of each sequential step.This novel approach proposed in this paper, in which the depth of the cascade forest is procedurally generated for recovered images, rather than being a constant. After training each layer, the recovered image is automatically evaluated, and new layers are constructed for training until an optimal restored image is obtained. Experiments show that this method improves the quality of image reconstruction while preserving the details of the image.

Full Text:



R. C. Gonzalez,and R. E. Woods , "Digital image processingr",3rd ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2008.% Journal name 12 (3), 45--67 (2015).

M. Jung.,A. Marquina, and L. A. Vese, "Variational multiframe restoration of images degraded by noisy (stochastic) blur kernels", J. Comput. Appl. Math vol. 240, no. 1,pp. 45--67, 2015.

Y. Fan , X. Wei, and S. Qin, "Fast and robust deblurring method with multi-frame images based on PSF estimation and total variation optimization", Optik-Int. J. Light Electron Opt vol. 124, no. 16,pp. 2285--2291, 2013.

F. W. Wheeler, X. Liu, P. H. Peter, and R. T. Hoctor, "Multiframe image restoration for face recognition", in Proc. IEEE Workshop Signal Process. Appl. Public Secur. Forensics,pp. 1–6, Apr. 2007.

R.Y. Tsai,and T.S. Huang, "Multipleframe image restoration and registration", Advances in computer vision and image processing. Greenwich, CT: JAI Press Inc., 1984, pp. 317-339.

N.K. Bose, H.C. Kim,and H.M. Valenzuela, "Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes", in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, Minneapolis, MN, 1993, pp. 269-272.

S.H. Rhee, and M.G. Kang, "Discrete cosine transform based regularized high-resolution image reconstruction algorithm", Opt. Eng vol. 38, no. 8,pp. 1348--1356, 1999.

H. Hanchuan, W. Peikang, "Super-resolution image reconstruction based on an improvement frequency domain registration approach", Electron Meas Technol vol. 33, no. 5,pp. 58--61, 2010.

P. Vanderwalle, Sbaiz L., J. Vanderwalle, "Super-resolution from unregistered and totally aliased signals using subspace methods", IEEE Trans. Signal Procss vol. 55, no. 70,pp. 3687--3703, 2007.

C.C. Chen, "A multi-frame super-resolution algorithm using POCS and wavelet",Montreal: Concordia University, 2010.

C. Liu, D. Q. Sun, "A bayesian approach to adaptive video super resolution", IEEE Conf. Comput. Vis Pattern Recognit, vol.42, no. 7, pp. 209-216, 2011.

F. Shi, J. Cheng, L. Wang, "Low-rank total variation for image super-resolution", Proc. Conf. Med. Image Comput. and Comput-Assist Interv, vol.16, no. 1, pp. 155-162, 2013.

A. Kappeler, S. Yoo, Q. Dai, "Video super-resolution with convolutional neural networks", IEEE Trans. Comput Image vol. 2, no. 2,pp. 1--1, 2016.

C. Dong, C.L. Chen, K. He, "Image super-resolution using deep convolutianal networks, IEEE Trans Pattern Anal Mach Intell vol. 38, no. 2,pp. 295--307, 2016.

W. Shi, J. Caballero, F. Huszar, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network", IEEE Conf Comput Vis Pattern Recognit pp.1874-1883, 2016.

J. Caballero, C. Ledig, A. Aitken, "Real-time video super-resolution with spatio-temporal networks and motion compensation", Comput Vis Pattern Recognit, 2017, arXiv preprint arXiv:1611. 05250v2.

S. Jeong, I. Yoon, J. Joonki, "Multi-frame example-based superresolution using locally directional self-similarity, IEEE Trans. Consum. Electron vol. 61, no. 3,pp. 353--358, 2015.

Y. M. Wang , Z. K. Luo, "Image SR based on MultiGrained Cascade Forests.” Multiresolution and Information Processing".

X. Liu , J. Yang , J. Sun, "Image registration approach based on SIFT, Infrared and Laser Eng vol. 37, no. 1,pp. 156--160, 2008.

Z. H. Zhou, J. Feng, "Deep forest: Towards an alternative to deep neural networks, arXiv preprint arXiv:1702.08835, 2017.


  • There are currently no refbacks.

International Journal of Electronics and Telecommunications
is a periodical of Electronics and Telecommunications Committee
of Polish Academy of Sciences

eISSN: 2300-1933