Abstract
Camouflaged object detection (COD) bas3ed on deep learning is an emerging visual detection task, which aims to detect the camouflaged objects “perfectly” embedded in the surrounding environment. However, most exiting work primarily focuses on building different COD models with the little summary work for the existing COD models. COD models with little summary work for the existing methods.
Therefore, this paper summarises the existing COD methods based on deep learning and discusses the future development of COD. Firstly, 23 existing COD models based on deep learning are introduced and analyzed according to five detection mechanisms: coarse-to-fine strategy, multi-task learning strategy, confidence-aware learning str4ategy, multi-source information fusion strategy and transformed-based strategy. The adwantages and disadvantages of each strategy are analyzed in dept, and then, 4 widely used datasets and 4 evaluation metrics for COD are introduced. In addition, the performance of the existing COD models based on deep learning is compared on four datasets, including quantitive comparison, visual comparison, efficiency analysis, and the detection effects on the camouflaged objects of different types. Furthermore, the practical applications of COD in medicine, industry, agriculture, military, art, etc. are mentioned. Finally, the deficiencies and challenges of existing methods in complex scenes, multi-scale objects, real-time performance, practical application requirements, and COD in other multimodalities are pointed out along with potential directions of COD are discussed.
Keywords: detection and classification of camouflaged objects, deep neural networks, feature enhancement
1. Analysis of the camouflaged object detection and classification problem.
Camouflage is a widespread biological phenomenon in nature that help organisms in nature use their structural and physiological characteristics to blend into their surroundings and thus avoid predators. In addition to biological camouflage in nature, artificial camouflage also exists, such as camouflaged soldiers in the military and body paint in art. In order to recognize these camouflaged creatures and artificially camouflaged targets that are perfectly embedded in the surrounding environment, researchers have proposed a number of camouflage object detection (COD) methods. However compared with other tasks (common object detection [1, 2, 3], salient object detection[4, 5, 6]), camouflaged targets are highly similar to the background in terms of texture, color, shape, etc., and their surroundings, which leads to more challenging task of detecting camouflaged targets. Figure 1 introduces multiple type of camouflages, where (1)–(8) are natural camouflage, (9)-(16) are artificial camouflage. Specifically on (1), (4) and (8) show land camouflage, (2) and (3) show sea creature camouflage, (5) and (7) show camouflaged creatures in low light conditions. Images (9) and (10) show military camouflage in different conditions where (9) is mountain camouflage and (10) is winter camouflage. Images (11)-(16) show artificial camouflage showing people in different camouflage conditions.
Camouflaged target detection can be traced back as far as 1998(7), when Tankus et al. proposed a non-edge region of interest mechanism for detecting artificially camouflaged targets in natural environments and combat scenarios. Since then, researchers have utilized direct visual based camouflage target detection such as face.
Camouflage targets, variety of camouflage target detection methods based on traditional feature extraction have been proposed. However, traditional methods usually suffer from time-consuming manual feature extraction, poor mobility, and low detection performance when facing camouflage scenes with extremely low contrast between foreground and background.
Figure 1: Multiple type of camouflaged targets selected from the four COD datasets
In recent years, camouflage target detection based on deep learning has become a research hotspot in the field of target detection, and more and more camouflage target detection algorithms based on deep learning have been proposed, and the detection accuracy and timeliness have been continuously improved.
Most of the existing deep learning based camouflage target detection methods firstly use convolutional neural network (CNN), suchas VGG (visual geometry gop ResNet (residual neural network) "Res2Net, etc. to extract features, and then different strategies such as coarse-to-fine, multi-task learning, confidence-aware learning, multi-source information fusion, Transformer, etc. are used to further enhance the features, and thus improve the performance of camouflage target detection.
Therefore, this paper analyzes the existing deep learning-based camouflage target detection methods from five perspectives: coarse-to-fine strategy, multi-task learning strategy, confidence-aware learning strategy, multi- source information fusion strategy and Transformer strategy. Existing deep learning-based camouflage target detection methods listing 23 existing camouflage target detection methods based on five different strategies and their backbone network.
1.1 Disguised target detection based on course-to-fine strategy
The coarse-to-fine strategy is an architecture that combines global prediction and local refinement. This structure can decouple complex targets, first make rough predictions for the overall area, and then refine the predictions through a variety of means. According to the different refinement methods, the existing camouflage target detection methods based on the coarse-to-fine strategy can be divided into three categories: camouflage target detection methods that use feature fusion refinement, and camouflage target detection methods that use distraction mining to refine and camouflaged target detection methods utilizing edge cue refinement.
1.2 Disguised target detection based on multi-task learning strategy
The multi-task learning strategy introduces common classification, positioning, etc. tasks or other detection tasks to assist the main task of binary segmentation to improve the detection performance of camouflaged targets. Through the collaborative work of multiple tasks, richer camouflaged target information can be mined. According to different tasks, camouflage target detection methods based on multi-task learning strategies are mainly divided into: camouflage target detection methods based on classification + segmentation, camouflage target detection methods based on positioning, sorting and segmentation, and camouflage target detection based on bionic attack + segmentation. Methods, camouflaged target detection method based on texture detection + segmentation and camouflaged target detection based on edge detection segmentation task.
1.3 Disguised target detection based on confidence-aware learning strategy
Confidence-aware learning aims to estimate the uncertainty that represents the quality of the data (arbitrary uncertainty) or the perceived uncertainty about the true model (epistemic uncertainty). In fully supervised models, confidence-aware learning is used to measure the high-order inconsistency between predictions and true labels, and it has been proven to effectively improve the robustness of deep neural networks. In the task of camouflaged target detection, some work introduces confidence-aware learning strategies to explicitly model the confidence of network predictions to promote model learning.
2 Datasets and evaluation metrics
This section introduces commonly used data sets and evaluations for camouflage target detection evaluation indicators.
2.1 Dataset
Due to the difficulty of detection and the particularity of camouflaged targets, the camouflaged target detection task has only begun to receive widespread attention in recent years. Therefore, there are only four COD public data sets. The specific information is shown in Table 2.
CHAMELEON dataset: It is a public dataset that has not been peer-reviewed and contains only 76 images collected from the Internet with the keyword "camouflaged animals". It mainly focuses on biological camouflage in nature, that is, camouflaged animals in complex backgrounds. This data set is often used to verify the usability of camouflaged target detection models.
CAMO data set: This data set contains two subsets, a camouflage image data set CAMO and another non-camouflage image data set MS-COCO. CAMO and MS-COCO each include 1250 images, of which 1000 are used for training and the remaining 250 are used for testing. The CAMO data set commonly used for camouflage tasks includes natural camouflage (camouflaging animals) and artificial camouflage (body painting and military camouflage), which has greater recognition difficulty and can be used to verify the effectiveness of the camouflage model.
COD10K data set: This data set is currently the largest camouflage target data set, including 5 superclasses and 69 subclasses, with a total of 10,000 camouflage images (6,000 for training and 4,000 for testing). The camouflage target categories of this data set include land, ocean, flying, and amphibious creatures in natural camouflage. The target dimensions include large, medium, and small dimensions, which can be used for model training and verification. This dataset greatly facilitates the development of camouflaged target detection.
NC4K data set: It is currently the largest camouflage target test set. It contains 4121 camouflage images downloaded from the Internet. Most of the camouflage target categories are natural camouflage and also include a small amount of artificial camouflage.
2.2 Evaluation indicators
Table 1
Analysis and comparison of different types of camouflaged object detection methods
Table 2
Main information of four camouflaged object detection datasets
E-metric, Eφ is defined as:
Among them, φ is the enhanced consistency matrix, W and H represent the width and height of the input respectively, C and G represent the prediction map and the true value map respectively.
F measure (Fβ) [8]: used to calculate the relationship between precision P and recall R. It can calculate the average harmonic measurement value between P and R and display its value. F measure F is defined as:
Mean absolute error MAE(M)[61]: used to calculate the mean absolute error of each pixel, its definition formula is:
Among them, the smaller the M value, the better the model performance.
3. Performance comparison of camouflaged target detection methods based on deep learning
3.1 Quantitative comparison
This section makes a quantitative comparison of the above different camouflaged target detection methods based on deep learning, using the S measure (Sα), the average of the E measure (Eφ), the average of the F measure (Fβ) and the mean absolute error MAE (M) As an evaluation criterion, experiments were conducted on the data sets CHAMELEON, CAMO COD10K, and NC4K. The experimental results are shown in Table 3. “1”, “2”, “3”, “4” and “5” in the table respectively represent “coarse-to-fine strategy”, “multi-task learning strategy”, “confidence-aware learning strategy” and “multi-task learning strategy”.
Assuming the information from Table 3:
From the overall performance, suggested model achieved the best performance on all four indicators of the CAMO-Test, COD10K-Test and NC4K data sets, and achieved the second best performance on the CHAMELEON data set. The proposed model is based on Transformer. It takes advantage of self-attention to capture long-distance dependencies and is suitable for tasks such as camouflage target detection that require comprehensive contextual information. The outstanding performance on the four camouflage data sets demonstrates the application of Transformer. Great potential for camouflaged target detection tasks. Ranked second in performance is Zoom-Net, which is a method based on the confidence-aware learning strategy and can pay more attention to the detection of uncertain pixels. In addition, the performance superiority of this model also benefits from the acquisition of multi-scale information by the zoom-in and zoom-out strategy. The third and fourth performance rankings are DCNet and FDNet (frequency domain network). These two methods are based on multi-source information fusion strategies. Due to the additional depth information or frequency domain information as supplements, these two algorithms show better performance. High detection accuracy. Based on CNN and not in the algorithm that introduces other source information, DGNet achieves the same level as Zoom-Net competitive performance, especially on the CAMO dataset.
Table 3
Quantitative comparison of camouflaged target detection methods based on deep learning
In addition to the above five methods with the best overall performance, BASNet achieved the best performance on the CHAMELEON data set, thanks to the U-Net structure and hybrid loss it uses, but it is more comprehensive in camouflage types and has a larger amount of data. Outstanding detection performance was not achieved on the other three large datasets.
In the algorithm that introduces other source information, DGNet achieves the same level as Zoom-Net competitive performance, especially on the CAMO dataset. In addition to the above five methods with the best overall performance, BASNet achieved the best performance on the CHAMELEON data set, thanks to the U-Net structure and hybrid loss it uses, but it is more comprehensive in camouflage types and has a larger amount of data. Outstanding detection performance was not achieved on the other three large datasets.
• Judging from the performance of different strategies, the best performance is the camouflage target detection method based on the Transformer strategy and the multi-source information fusion strategy. The method based on the multi-task learning strategy and the confidence-aware learning strategy is based on CNN without introducing multiple sources. The source information method performed best, followed by SINetV2 based on the coarse-to-fine strategy also showed good performance. Different strategies have high research value, so the pros and cons of different strategies should be weighed to design a camouflaged target detection model with better performance.
3.2 Visual comparison
This section gives the visual detection results of 13 camouflaged target detection algorithms under 7 different types of camouflaged targets, as shown in Figure 6, in which from left to right they are: (1) large targets; (2) small targets; (3) Many and small (4) occlusion targets; (5) ghost targets; (6) targets with rich edge details; (7) body painting targets and military camouflage targets in artificial camouflage. There is no comparison here for other camouflaged target detection methods that do not provide open source code or result prediction maps. It can be seen on Figure 2.
From the overall detection effect, among these algorithms, Our suggested model based on Transformer, Zoom-Net based on confidence-aware learning strategy, and DGNet based on multi-task learning have shown good detection results in a variety of challenging scenarios. , Especially the detection of large targets (1), multiple targets (3) and occluded targets (4) is better than other algorithms, the detected target area is more complete and the outline is clearer. LSR based on multi-task learning shows good detection results in occluded targets and ghost targets, and it uses positioning and sorting tasks to promote the detection of camouflaged targets. JCSOD based on the confidence-aware learning strategy can detect large targets relatively completely and obtain rich edge details. This is because JCSOD combines SOD data and uses confidence-aware learning to help the model eliminate the interference of salient areas in non-camouflage. SINetV2 based on the coarse-to-fine strategy detects richer target information, especially in complex scenes ((4)(6)(7)). This is due to the feature fusion method of neighbor connection and Group inverse attention as a means of goal refinement. It is worth noting that although the targets detected by BASNet are not complete enough, the boundaries and contours of the targets it detects are very fine, which is mainly due to its design. Refinement of hybrid loss function and decoder with U-Net-like structure.
Judging from the detection effects of different targets, these camouflaged target detection methods have good detection performance for ordinary camouflaged targets in simple scenes, but have poor detection performance for camouflaged targets in challenging scenarios. As shown in Figure 6, these methods have poor detection results for small targets, occluded targets, and artificially camouflaged targets. For small targets, the target position cannot be accurately located; for occluded targets, the target information cannot be completely separated; for artificially camouflaged targets, too much irrelevant information is detected.
Through quantitative analysis and visual analysis, it can be seen that the suggested target detection algorithm can achieve better detection accuracy and show better segmentation results for such challenging targets as camouflage. However, due to the particularity of camouflaged targets, the segmentation results of existing algorithms still have defects such as edge blur and inaccurate positioning.
Figure 2:Visual comparison of deep learning-based camouflaged object detection methods
Conclusions
This article summarizes the existing deep learning-based camouflage target detection methods from the perspective of five strategies: coarse-to-fine, multi-task learning, confidence-aware learning, multi-source information fusion and Transformer, and analyzes and discusses the advantages and disadvantages of different models and also compares the results with suggested model. ,Quantitative analysis, visual comparison and efficiency analysis of ,different methods are given. Although camouflaged target detection has received more and more research and its performance continues to improve, existing camouflaged target detection algorithms based on deep learning still have many shortcomings and challenges because the camouflaged target itself is extremely challenging. mainly include:
• The detection effect is poor in complex scenes. Existing camouflage target detection algorithms can basically detect camouflage targets in simple scenes. However, in reality, camouflage targets are usually in complex scenarios such as cluttered backgrounds, large-area occlusions, and overly prominent backgrounds, resulting in confusion of the boundaries and shape of the camouflage targets. discontinuity. Therefore, camouflaged target detection for occluded targets, artificial camouflaged targets, etc. in complex scenes is still very challenging.
• Multi-scale target detection performance is poor. Faced with camouflaged targets of various scales in actual scenes, existing camouflaged target detection methods usually cannot completely detect large targets, cannot accurately locate small target locations (the target is incorrectly positioned as a background area with a prominent background), etc., resulting in The detection effect of large targets and many and small camouflaged targets is poor.
Suggested model brings massive improvements into detection and classification quality. However it still need further research in order to be ready for use in everyday life. In view of the shortcomings and challenges in the above camouflage target detection, the following analysis lists the future research directions of camouflage target detection based on deep learning.
• Research on multi-scale camouflaged target detection methods in complex backgrounds. Maximize the simulation of the concept of human visual recognition of camouflage targets, design targeted models to reason and judge camouflage targets in complex backgrounds; fully capture the global, local information and local salience information of camouflage targets, and improve multi-scale camouflage target detection performance.
• Research on lightweight camouflaged target detection methods. Make full use of existing lightweight model ideas, such as depth-separable convolution, small convolution instead of large convolution, compression coding, weight quantization, weight sharing, transfer learning/knowledge distillation, computing acceleration, etc., to design more sophisticated It can meet the real-time application requirements and camouflage the target detection model to be applied in scenarios with very high real-time requirements such as military combat environments and search and rescue activities.
• Research on multi-modal camouflage target detection methods. The performance of a single image modality for detecting camouflaged targets is still very limited. Methods such as multi-source information fusion, multi-view learning, and collaborative learning are used to combine multiple modalities such as text, images, audio, and video, and use multi-modality to detect camouflaged targets. Improve the detection performance of camouflaged targets.
References
[1] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// Proceedings of the Annual Conference on Neural Information Processing Systems 2015, Montreal, Dec 7-12, 2015. Red Hook: Curran Associates, 2015: 91-99.
[2] LIN TY, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2999-3007.
[3] HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2980-2988.
[4] WU Y H, LIU Y, ZHANG L, et al. EDN: salient object detection via extremely-down sampled network[J]. IEEE Transactions on Image Processing, 2022, 31: 3125-3136.
[5] PANG Y, ZHAO X, ZHANG L, et al. Multi-scale interactive network for salient object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: ΙΕΕΕ, 2020: 9410-9419
[6] FENG M, LU H, DING E. Attentive feedback network for boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1623-1632.
[7] TANKUS A, YESHURUN Y. Detection of regions of interest and camouflage breaking by direct convexity estimation [C]//Proceedings of the 1998 IEEE Workshop on Visual Surveillance, Bombay, Dec 5-8, 1998. Washington: IEEE Computer Society, 1998: 42-48.
[8] ZHANG X, ZHU C, WANG S, et al. A Bayesian approach to camouflaged moving object detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 27 (9): 2001-2013.
[9] BEIDERMAN Y, TEICHER M, GARCIA J, et al. Optical technique for classification, recognition and identification of obscured objects[J]. Optics Communications, 2010, 283 (21): 4274-4282.
[10] GALUN M, SHARON E, BASRI R, et al. Texture segmen- tation by multiscale aggregation of filter responses and shape elements[C]//Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, Oct 14-17, 2003. Washington: IEEE Computer Society, 2003: 716-723.
[11] GUO H, DOU Y, TIAN T, et al. A robust foreground segmentation method by temporal averaging multiple video frames[C]//Proceedings of the 2008 International Conference on Audio, Language and Image Processing, Shanghai, Jul 7- 9, 2008. Piscataway: IEEE, 2008: 878-882.
[12] KAVITHA C, RAO B P, GOVARDHAN A. An efficient content based image retrieval using color and texture of image sub-blocks[J]. International Journal of Engineering Science and Technology, 2011, 3(2): 1060-1068.
[13] HALL J R, CUTHILL I C, BADDELEY R, et al. Camouflage, detection and identification of moving targets[J]. Proceedings of the Royal Society B: Biological Sciences, 2013, 280 (1758): 20130064.
[14] BỊ H, ZHANG C, WANG K, et al. Rethinking camouflaged object detection: models and datasets [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32 (9): 5708-5724.
[15] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [C]//Proceedings of the 2015 International Conference on Learning Represen- tations, San Diego, May 7-9, 2015. Piscataway: IEEE, 2015: 1-14.
[16] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Con- ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[17] FAN DP, JIGP, SUN G, et al. Camouflaged object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 2774-2784.
[18] WU Z, SU L, HUANG Q. Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the 2019 IEEE/CVF conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 3907-3916.
[19] WANG K, BI H, ZHANG Y, et al. D2C-Net: a dual-branch, dual-guidance and cross-refine network for camouflaged object detection[J]. IEEE Transactions on Industrial Electr- onics, 2022, 69(5): 5364-5374.
[20] RONNEBERGER O, FISCHER P, BROX T. U-net: con- volutional networks for biomedical image segmentation [C]//LNCS 9351: Proceedings of the 2015 International Conference on Medical Image Computing and Computer- Assisted Intervention, Munich, Oct 5-9, 2015. Cham:Springer, 2015: 234-241.
[21] ZHUGE M, LU X, GUO Y, et al. CubeNet: X-shape connection for camouflaged object detection[J]. Pattern Recognition, 2022, 127: 108644.
[22] SUN Y, CHEN G, ZHOU T, et al. Context-aware cross- level fusion network for camouflaged object detection[J]. arXiv:2105.12555, 2021.
[23] MEI H, JIG P, WEI Z, et al. Camouflaged object segmentation with distraction mining [C]//Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 8772-8781.
[24] FAN D P, JIG P, CHENG M M, et al. Concealed object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10): 6024-6042.
[25] QIN X, FAN DP, HUANG C, et al. Boundary- aware segmentation network for mobile and web applications[J]. arXiv:2101.04704, 2021.
[26] DE BOER PT, KROESE D P, MANNOR S, et al. A tutorial on the cross-entropy method [J]. Annals of Operations Research, 2005, 134(1): 19-67.
[27] WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, Nov 9-12, 2003. Piscataway: ΙΕΕΕ, 2003: 1398-1402.
|
Голосування 
Ця робота ліцензується відповідно до Creative Commons Attribution 4.0 International License
Знайшли помилку? Виділіть помилковий текст мишкою і натисніть Ctrl + Enter