[1]牟 琦,夏 蕾,李占利,等.采用曲率扩散和边缘重建的深度图像空洞修复[J].西安科技大学学报,2020,(02):369-376.
 MU Qi,XIA Lei,LI Zhan-li,et al.Depth image hole inpainting method using curvature diffusion and edge reconstruction[J].Journal of Xi'an University of Science and Technology,2020,(02):369-376.
点击复制

采用曲率扩散和边缘重建的深度图像空洞修复(/HTML)
分享到:

西安科技大学学报[ISSN:1672-9315/CN:61-1434/N]

卷:
期数:
2020年02期
页码:
369-376
栏目:
出版日期:
2020-03-30

文章信息/Info

Title:
Depth image hole inpainting method using curvature diffusion and edge reconstruction
文章编号:
1672-9315(2020)02-0369-08
作者:
牟 琦12夏 蕾1李占利1李洪安1
(1.西安科技大学 计算机科学与技术学院,陕西 西安 710054; 2.西安科技大学 机械工程学院,陕西 西安710054)
Author(s):
MU Qi12XIA Lei1LI Zhan-li1LI Hong-an1
(1.College of Computer Science and Engineering,Xi'an University of Science and Technology,Xi'an 710054,China; 2.College of Mechanical and Engineering,Xi'an University of Science and Technology,Xi'an 710054,China)
关键词:
深度图像修复 空洞掩膜 曲率扩散模型 二值分割 马尔科夫随机场模型
Keywords:
depth image inpainting hole mask curvature driven diffusion model binary segmentation Markov random field model
分类号:
TP 391.41
文献标志码:
A
摘要:
传统的深度图像空洞修复算法,针对离散空洞和物体内部及背景中的空洞修复效果较好,但当物体边缘处存在较大面积空洞时,采用传统的修复方法,会出现物体边缘过填充或欠填充现象,造成边缘几何失真、边界模糊等问题。采用曲率扩散和边缘重建的深度图像空洞修复方法。首先获取深度图像空洞掩膜,确定空洞区域; 然后使用曲率扩散模型填充空洞,并使用二值分割滤波获取边缘信息,得到待重建的像素; 最后,通过马尔科夫随机场模型重建深度图像中的物体边缘纹理,去除模糊现象。曲率扩散模型将深度图像视为一个流体扩散方程,利用等照度线和曲率分布确定扩散强度,将局部结构从空洞的外部向内部扩散,能够准确的填充较大面积的空洞结构; 马尔科夫随机场模型利用邻域系和连通系的能量函数表示结构信息,能够有效重建修复后的深度图像边缘纹理,从而去除物体边缘模糊。实验结果表明,相比传统的深度图像修复方法,采用文中方法修复后深度图像的平均梯度指标提高了10%-25%,可以有效地实现对深度图像中物体边缘处较大面积空洞的修复,得到结构完整、边缘纹理清晰的深度图像。
Abstract:
The traditional depth image hole inpainting methods fill discrete hole points,and the inner part of the object and the hole in the image background achieve better results.However,when there is a large area of the hole at the edge of the object in the image,the traditional inpainting methods may cause over-filling or under-filling of the edge of the object,causing geometric distortion of the edge and blurring of the boundary.To solve the above problems,this paper proposes a depth image hole inpainting method based on curvature diffusion and edge reconstruction,combining holes filling and texture restoration.Firstly,the depth image hole mask is obtained to determine the holes region.Then the holes are filled with the Curvature Driven Diffusion(CDD)model,and the edge blur pixels are removed by binary segmentation filtering to obtain the pixels to be reconstructed.Finally,the edge texture details of the object in the depth image are reconstructed by the Markov Random Field(MRF).The CDD model considers the depth image as a fluid diffusion equation,and uses the isolux line and curvature distribution to determine the diffusion intensity,and spreads the local structure from the outside to the inside of the holes,which can better fill the large-area hole structure; MRF model uses the energy functions of the neighborhood system and the communication system to introduce structural information,which can effectively reconstruct the edge information of the restored depth image,thus effectively avoiding object edge blur.The experimental results show that compared with the traditional depth image inpainting methods,the average gradient value of the depth image is improved by 10%~25% processed by the proposed method,which can effectively fill the large-area holes both in the interior and edge of the object in the depth image,and a depth image with complete structure and clear edge texture is obtained.

参考文献/References:

[1] 杨宇翔,高明煜,尹 克,等.结合同场景立体图对的高质量深度图像重建[J]. 中国图象图形学报,2015,20(1):1-10. YANG Yu-xiang,GAO Ming-yi,YIN Ke,et al.High-quality depth map reconstruction combining stereo image pair[J]. Journal of Image and Graphics,2015,20(1):1-10. [2]李诗锐,李 琪.基于Kinect v2的实时精确三维重建系统[J].软件学报,2016,27(10):2519-2529. LI Shi-rui,LI Qi. Real-time accurate 3D reconstruction based on Kinect v2[J].Journal of Software,2016,27(10):2519-2529. [3]Schwarz M,Milan A,Periyasamy A S,et al.RGB-D object detection and semantic segmentation for autonomous manipulation in clutter[J].The International Journal of Robotics Research,2018,37(4-5):437-451. [4]刘天亮,冯希龙,顾雁秋.一种由粗至精的RGB-D室内场景语义分割方法[J].东南大学学报(自然科学版),2016,46(4):681-687. LIU Tian-liang,FENG Xi-long,GU Yan-qiu.Coarse-to-Fine semantic parsing method for RGB-D indoor scenes[J].Journal of Southeast University(Natural Science Edition),2016,46(4):681-687. [5]季一木,陈治宇,田鹏浩.无人驾驶中3D目标检测方法研究综述[J].南京邮电大学学报(自然科学版),2019(4):72-79. JI Yi-mu,CHEN Zhi-yu,TIAN Peng-hao.Research review of 3D target detection method in unmanned vehicle[J].Journal of Nanjing University of Posts and Telecommunications(Natural Science Edition),2019(4):72-79. [6]Atapour-Abarghouei A,Breckon T P.A comparative review of plausible hole filling strategies in the context of scene depth image completion[J].Computers & Graphics,2018,72(5):39-58. [7]Mallick T,Das P P,Majumdar A K.Characterizations of noise in kinect depth images:a review[J].IEEE Sensors Journal,2014,14(6):1731-1740. [8]刘继忠,吴文虎,程 承,等.基于像素滤波和中值滤波的深度图像修复方法[J].光电子·激光,2018,29(5):539-544. LIU Ji-zhong,WU Wen-hu,CHENG Cheng,et al.Depth image repair method based on pixel filter and median filter[J].Journal of Optoelectronics Laser,2018,29(5):539-544. [9]李少敏,张 倩,王 沛.基于高斯混合模型的Kinect深度图像增强算法[J].上海师范大学学报(自然科学版),2016,45(1):28-33. LI Shao-min,ZHANG Qian,WANG Pei.Kinect sensor's depth image enhancement based on gaussian mixture model[J].Journal of Shanghai Normal University(Natural Sciences),2016,45(1):28-33. [10]吕朝辉,沈萦华,李精华.基于Kinect的深度图像修复方法[J].吉林大学学报(工学版),2016,186(5):1697-1703. LU Zhao-hui,SHEN Ying-hua,LI Jing-hua.Depth map inpainting method based on Kinect sensor[J].Journal of Jilin University(Engineering and Technology Edition),2016,186(5):1697-1703. [11]胡天佑,彭宗举,焦任直,等.基于超像素分割的深度图像修复算法[J].光电子·激光,2016(27):1120-1128. HU Tian-you,PENG Zong-ju,JIAO Ren-zhi,et al.Depth map inpainting algorithm based on superpixel segmentation[J].Journal of Optoelectronics Laser,2016(27):1120-1128. [12]Camplani M,Salgado L.Efficient spatio-temporal hole filling strategy for Kinect depthmaps[J].Proceedings of SPIE,2012(1):316-320. [13] Le A,Jung S W,Won C.Directional joint bilateral filter for depth images[J].Sensors,2014,14(7):11362-11378. [14]Kulkarni M,Rajagopalan A N.Depth inpainting by tensor voting[J].Journal of the Optical Society of America Aoptics Image Science and Vision,2013,30(6):1155-1165. [15]Telea Alexandru.An image inpainting technique based on the fast marching method[J].Journal of Graphics Tools,2004,9(1):23-34. [16]Gong X,Liu J,Zhou W,et al.Guided depth enhancement via a fast marching method[J].Image and Vision Computing,2013,31(10):695-703. [17]万 红,钱 锐.模糊C-均值聚类引导的Kinect深度图像修复算法[J].计算机应用研究,2019,36(5):290-294. WAN Hong,QIAN Rui.Kinect depth map inpainting under fuzzy C-mean clustering guidance[J].Application Research of Computers,2019,36(5):290-294. [18]陈国军,程 琰,曹 岳.基于目标特征的植株深度图像修复[J].图学学报,2019,40(3):460-465. CHEN Guo-jun,CHENG Yan,CAO Yue.Plant depth maps recovery based on target features[J].Journal of Graphics,2019,40(3):460-465. [19]王殿伟,陈 鹏,李大湘,等.融合纹理信息的深度图像修复[J].系统工程与电子技术,2019,41(8):1720-1725. WANG Dian-wei,CHEN Peng,LI Da-xiang,et al.Depth maps inpainting with fused texture information[J].Systems Engineering and Electronics,2019,41(8):1720-1725. [20] Wang Y,Zhong F,Peng Q,et al.Depth map enhancement based on color and depth consistency[J].The Visual Computer,2014,30(10):1157-1168. [21]Zuo Y,Wu Q,Zhang J,et al.Explicit edge inconsistency evaluation model for color-guided depth map enhancement[J].Transactions on Circuits and Systems for Video Technology,2018,28(2):439-453. [22]Li J,Gao W,Wu Y.High-quality 3D reconstruction with depth super-resolution and completion[J].IEEE Access,2019(7):19370-19381. [23]Chan T F,Shen J.Nontexture inpainting by curvature-driven diffusions[J].Journal of Visual Communication and Image Representation,2001,12(4):436-449. [24]Zhao L,Bai H,Wang A,et al.Two-stage filtering of compressed depth images with markov random field[J].Signal Processing:Image Communication,2017(54):11-22. [25]Richardt C,Stoll C,Dodgson N A,et al.Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos[J].Computer Graphics Forum,2012,31(2):247-256. [26]李知菲,陈 源.基于联合双边滤波器的Kinect深度图像滤波算法[J].计算机应用,2014,34(8):2231-2234. LI Zhi-fei,CHEN Yuan.Kinect depth image filtering algorithm based on joint bilateral filter[J].Journal of Computer Applications,2014,34(8):2231-2234.

备注/Memo

备注/Memo:
收稿日期:2019-07-11 责任编辑:李克永
基金项目:中国博士后科学基金(No.2016M602941XB); 陕西省自然科学基础研究计划(2019JM-62,2019JM-348); 西安科技大学博士启动金项目(2019QDJ007)
通信作者:牟 琦(1982-),女,陕西西安人,副教授,E-mail:muqi@xust.edu.cn
更新日期/Last Update: 2020-03-30