基于改进SegNet的沥青路面病害提取与分类方法

张志华; 邓砚学; 张新秀

doi:10.3963/j.jssn.1674-4861.2022.03.013

基于改进SegNet的沥青路面病害提取与分类方法

doi: 10.3963/j.jssn.1674-4861.2022.03.013

张志华^{1, 2, 3, ,},
邓砚学^{1, 2, 3},
张新秀⁴

1.
兰州交通大学测绘与地理信息学院兰州 730070
2.
兰州交通大学地理国情监测技术应用国家地方联合工程研究中心兰州 730070
3.
甘肃省地理国情监测工程实验室兰州 730070
4.
甘肃省公路路网检测重点实验室兰州 730070

基金项目:

国家自然科学基金项目 40861059

国家自然科学基金项目 42161069

兰州交通大学优秀平台支持项目 201806

详细信息

通讯作者:
张志华（1980—），博士，教授. 研究方向：三维地学模拟、摄影测量与图像识别.E-mail：43447077@qq.com

中图分类号: U416.2; TP183
计量
- 文章访问数: 994
- HTML全文浏览量: 474
- PDF下载量: 123
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-22
- 网络出版日期: 2022-07-25

A Method for Detecting and Differentiating Asphalt Pavement Distress Based on an Improved SegNet Algorithm

ZHANG Zhihua^{1, 2, 3
, ,},
DENG Yanxue^{1, 2, 3},
ZHANG Xinxiu⁴

1.
Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China
2.
National-Local Joint Engineering Research Center of Technologies and Applications for National Geographic State Monitoring, Lanzhou Jiaotong University, Lanzhou 730070, China
3.
Gansu Provincial Engineering Laboratory for National Geographic State Monitoring, Lanzhou 730070, China
4.
Key Laboratory of Highway Network Monitoring in Gansu Province, Lanzhou 730070, China

摘要

摘要: 针对现有SegNet算法难以精确区分裂缝和灌封裂缝等具有相似特征的沥青路面病害的问题，提出了基于改进SegNet网络的沥青路面病害提取方法。针对道路标线和光照不均匀等导致路面病害图像质量差异化的因素，本研究在去除道路标线的基础上，运用带色彩恢复的多尺度视网膜增强算法，降低道路标线和光照对图像质量的影响以及增强路面病害图像的对比度、色调和亮度，提高病害的识别精度；为了充分利用图像的上下文信息，解决SegNet网络对细微病害分割效果不佳的问题，引入残差神经网络（ResNet）作为编码器，并对解码器中每个上采样产生的特征图拼接2个分别由卷积层（1×1的卷积核）和空洞卷积层从对应的编码器中获取的尺度相同的特征图；运用形态学闭运算连接分割结果中不连续的裂缝。为了验证改进算法的有效性，将其与典型的语义分割方法（SegNet和BiSeNet）在测试集上进行测试和性能对比。研究结果表明，3种方法的平均交并比（MIoU）和F1分数（F1-score）分别为（82.4%，98.9%），（69.4%，94.0%），（80.5%，98.1%）；利用这3种方法对甘肃省部分路段路面病害的提取效果进行对比测试，提出方法的裂缝漏检率和误检率分别为2.91%，1.94%，优于SegNet（10.68%，14.56%）和BiSeNet（6.80%，12.62%）。本研究所提方法能够更精确地提取和区分沥青路面裂缝和灌封裂缝。
- 智能交通 /
- 裂缝和灌封裂缝 /
- 编解码网络 /
- 空洞卷积 /
- 语义分割
Abstract: The existing SegNet methods are unable to accurately differentiate asphalt pavement distress with similar characteristics, such as cracks and sealed cracks. To solve this problem, a method of detecting asphalt pavement distress based on an improved SegNet network is proposed. In order to remove the negative impact of road markings and uneven illumination onto image quality of road surface and subsequent failure detection, a multi-scale Retinex algorithm with color restoration(MSRCR)is used to reduce the impact of road marking and uneven lighting on image quality. Through enhancing the contrast, hue and brightness of the images for road surface, the accuracy of distress recognition is improved. In order to fully use of the contextual information of the image, and overcome the issue with the SegNet network of being ineffective in segmenting and identifying subtle diseases, a residual neural network(ResNet)is introduced as the encoder, and two feature maps with a same scale obtained by a convolutional layer with a 1×1 kernel and a dilated convolutional layer with different dilation rates are fused for each feature map, generated by up-sampling in the decoder. And a closed, morphological operation is used to connect discontinuous cracks. To verify the effectiveness of the improved algorithm, it is compared with the classic semantic segmentation methods(such as SegNet and BiSeNet)over the test sets. The average intersection over Union(MIoU)and F1 score are(82.4%, 98.9%), (69.4%, 94.0%)and(80.5%, 98.1%), respectively. The three methods are compared in terms of their extraction efficiency in identifying pavement diseases using the pavement images collected at several freeway sections in the Gansu Province. The misdetection rate and false detection rate of cracks of the proposed method are 2.91%, 1.94%, respectively, which are much better than those of the SegNet(10.68%, 14.56%)and BiSeNet(6.80%, 12.62%). The above results show that the proposed method can be used to extract and identify asphalt pavement cracks and sealed cracks with a higher accuracy.
- intelligent transportation /
- crack and sealing crack /
- encoder-decoder network /
- dilation convolution /
- semantic segmentation

HTML全文

图 1 裂缝和灌封裂缝提取流程

Figure 1. The workflow of extraction crack and sealed crack

下载: 全尺寸图片幻灯片

图 2 图像增强处理效果

Figure 2. The results of image enhancement processing

下载: 全尺寸图片幻灯片

图 3 样本扩增

Figure 3. Sample amplification

下载: 全尺寸图片幻灯片

图 4 SegNet网络结构

Figure 4. SegNet network structure

下载: 全尺寸图片幻灯片

图 5 改进的编解码网络结构

Figure 5. Improved encoder-decoder network structure

下载: 全尺寸图片幻灯片

图 6 实验结果

Figure 6. Experiment results

下载: 全尺寸图片幻灯片

图 7 比较3种方法的MIoU和Loss

Figure 7. Comparing the MIoU and Loss of three methods

下载: 全尺寸图片幻灯片

图 8 对比3种方法的分割结果

Figure 8. Comparision segmentation results of three metholds

下载: 全尺寸图片幻灯片

表 1 数据集构成

Table 1. Dataset composition

数据	横、纵向裂缝	灌封裂缝	总数
训练	2 970	1 485	4 455
测试	990	495	1 485
总数	3 960	1 980	5 940

下载: 导出CSV

表 2 对比3种方法的平均测试结果

Table 2. Comparision the average testing results of three metholds

方法	Precision	Recall	F1 score	MIoU	裂缝精度	灌封裂缝精度
SegNet	0.951 016 2	0.949 304 7	0.940 000 5	0.694 327 7	0.614 384 1	0.867 085 1
BiSeNet	0.990 421	0.990 492 5	0.980 700 8	0.804 804	0.844 634 7	0.958 933
改进方法	0.990 865 9	0.991 129 2	0.989 067 2	0.823 605	0.876 625 9	0.969 096 6

下载: 导出CSV

表 3 3种方法的漏检率和误检率

Table 3. The missed detection rate and false detection rate of the three methods

病害类别	手动提取/条	SegNet		BiSeNet		本文方法
病害类别	手动提取/条	漏检率/%	误检率/%	漏检率/%	误检率/%	漏检率/%	误检率/%
灌封裂缝	164	9.15	3.05	7.93	3.05	0.60	0.00
裂缝	103	10.68	14.56	6.80	12.62	2.91	1.94

下载: 导出CSV

参考文献(33)

[1]	马建, 赵祥模, 贺拴海, 等. 路面检测技术综述[J]. 交通运输工程学报, 2017, 17(5): 121-137. doi: 10.3969/j.issn.1671-1637.2017.05.012 MA J, ZHAO X M, HE S H, et al. Review of pavement detection technology[J]. Journal of Traffic and Transportation Engineering, 2017, 17(5): 121-137. (in Chinese) doi: 10.3969/j.issn.1671-1637.2017.05.012
[2]	ZOU Q, CAO Y, LI Q, et al. CrackTree: Automatic crack detection from pavement images[J]. Pattern Recognition Letters, 2012, 33(3): 227-238. doi: 10.1016/j.patrec.2011.11.004
[3]	KIRSCHKE K R, VELINSKY S A. Histogram-based approach for automated pavement crack sensing[J]. Transportation Engineering, 1992, 118(5): 700-710. doi: 10.1061/(ASCE)0733-947X(1992)118:5(700)
[4]	李清泉, 邹勤, 毛庆洲. 基于最小代价路径搜索的路面裂缝检测[J]. 中国公路学报, 2010, 23(6): 28-33. doi: 10.3969/j.issn.1001-7372.2010.06.005 LI Q Q, ZOU Q, MAO Q Z. Pavement crack detection based on minimum cost path searching[J]. China Journal of Highway and Transport, 2010, 23(6): 28-33. (in Chinese) doi: 10.3969/j.issn.1001-7372.2010.06.005
[5]	ABDEL-QADER I, ABUDAYYEH O, KELLY M E. Analysis of edge-detection techniques for crack identification in bridges[J]. Journal of Computing in Civil Engineering, 2003, 17 (4): 255-263. doi: 10.1061/(ASCE)0887-3801(2003)17:4(255)
[6]	KAMALIARDAKANI M, SUN L, ARDAKANI M K. Sealed crack detection algorithm using heuristic thresholding approach[J]. Journal of Computing in Civil Engineering, 2016, 30(1): 04014110. doi: 10.1061/(ASCE)CP.1943-5487.0000447
[7]	HU Y, ZHAO C X. A local binary pattern-based method for pavement crack detection[J]. Pattern Recognit, 2010, 5(1): 140-147. doi: 10.13176/11.167
[8]	ZALAMA E, GOMEZ-GARCIA-BERMEJO J, MEDINA R. et al. Road crack detection using visual features extracted by Gabor filters[J]Computer-Aided Civil and Infrastructure Engineering, 2014, 29(5): 342-358. doi: 10.1111/mice.12042
[9]	SHI Y, CUI L M, QI Z Q, et al. Automatic road crack detection using random structured forest[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12), 3434-3445. doi: 10.1109/TITS.2016.2552248
[10]	NEJAD F M, ZAKERI H. A comparison of multiresolution methods for detection and isolation of pavement distress[J]. Expert Systems with Applications, 2011, 38(3): 2857-2872. doi: 10.1016/j.eswa.2010.08.079
[11]	CHA Y J, CHOI W, BÜYÜKÖZTÜRK O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361-37. doi: 10.1111/mice.12263
[12]	DORAFSHAN S, THOMAS R J, MAGUIRE M. Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete[J]. Construction and Building Materials, 2018(186): 1031-1045.
[13]	沙爱民, 童峥, 高杰. 基于卷积神经网络的路表病害识别与测量[J]. 中国公路学报, 2018, 31(1): 1-10. doi: 10.3969/j.issn.1001-7372.2018.01.001 SHA A M, TONG Z, GAO J. Recognition and measurement of pavement disasters based on convolutional neural networks[J]. China Journal of Highway and Transportation, 2018, 31(1): 1-10. (in Chinese) doi: 10.3969/j.issn.1001-7372.2018.01.001
[14]	ZHANG L, YANG F, ZHANG Y D. et al. Road crack detection using deep convolutional neural network[C]. IEEE International Conference on Image Processing, Phoenix: IEEE, 2016.
[15]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683
[16]	景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述[J]. 计算机工程, 2020, 46(10): 1-17. doi: 10.3778/j.issn.1002-8331.2001-0320 JING Z W, GUAN H Y, PENG D F, et al. Survey of research in image semantic segmentation based on deep neural network[J]. Computer Engineering, 2020, 46(10): 1-17. (in Chinese) doi: 10.3778/j.issn.1002-8331.2001-0320
[17]	HUANG H W, LI Q T, ZHANG D M. Deep learning-based image recognition for crack and leakage defects of metro shield tunnel[J]. Tunnelling and Underground Space Technology, 2018(77): 166-176.
[18]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. The European Conference on Computer Vision, Cham: Springer, 2018.
[19]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
[20]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention, Cham: Springer, 2015.
[21]	YU C Q, WANG J, PENG C. BiSeNet: Bilateral segmentation network for real-time semantic segmentation[C]. 15th European Conference in Computer Science, Cham: Springer, 2018.
[22]	BANG S, PARK S, KIM H, et al. Encoder-decoder network for pixel-level road crack detection in black-box images[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713-727. doi: 10.1111/mice.12440
[23]	XUE L L, LIANG K D, LI W, et al. FPGA accelerates deep residual learning for image recognition[C]. IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference, Chengdu: IEEE, 2017.
[24]	ZEILER M D, FERGUS R. Visualizing and understandingconvolutional networks[C]. European Conference on Computer Vision, Cham, Switzerland: Springer, 2014.
[25]	WANG Y K, HUANG W B. A CUDA-enabled parallel algorithm for accelerating retinex[J]. Journal of Real-Time Image Processing, 2014, 9(3): 407-425. doi: 10.1007/s11554-012-0301-6
[26]	侯越, 陈逸涵, 顾兴宇, 等. 基于卷积自编码的沥青路面目标与裂缝智能识别[J]. 中国公路学报, 2020, 206(10): 292-307. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202010023.htm HOU Y, CHEN Y H, GU X Y, et al. Automatic identification of pavement objects and cracks using the convolutional auto-encoder[J]. China Journal of Highway and Transport, 2020, 206(10): 292-307. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202010023.htm
[27]	RIZZI A, GATTA C, MARINI D. A new algorithm for unsupervised global and local color correction[J]. Pattern Recognition Letters, 2003, 24(11): 1663-1677. doi: 10.1016/S0167-8655(02)00323-9
[28]	王鹏, 神和龙, 尹勇, 等. 基于深度学习的船舶驾驶员疲劳检测算法[J]. 交通信息与安全, 2022, 40(1): 1-9. doi: 10.3963/j.jssn.1674-4861.2022.01.008 WANG P, SHEN H L, YIN Y, et al. A detection algorithm for the fatigue of ship ffficers based on deep learning technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 1-9. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.01.008
[29]	KASTHURIRANGAN G, SIDDHARTHA K K, ALOK C, et al. Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection[J]. Construction and Building Materials, 2017 (157): 322-330.
[30]	MANDAL V, UONG L, ADU G Y. Automated road crack detection using deep convolutional neural networks[C]. IEEE International Conference on Big Data, Seattle: IEEE, 2018.
[31]	HANG K, CHENG H D, ZHANG B. Unified approach to pavement crack and sealed crack detection using pre-classification based on transfer learning[J]. Journal of Computing in Civil Engineering, 2018, 32(2): 04018001/1-12. doi: 10.1061/(ASCE)CP.1943-5487.0000736
[32]	CHEN T Y, CAI Z H, ZHAO X. et al. Pavement crack detection and recognition using the architecture of segNet[J]. Journal of Industrial Information Integration, 2020, 18(2): 1-12.
[33]	徐辉, 祝玉华, 甄彤, 等. 深度神经网络图像语义分割方法综述[J]. 计算机科学与探索, 2021, 15(1): 47-59. https://www.cnki.com.cn/Article/CJFDTOTAL-KXTS202101003.htm XU H, ZHU Y H, ZHEN T, et al. Survey of image semantic segmentation methods based on deep neural network[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(1): 47-59. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-KXTS202101003.htm