Skip to content

Commit cdc82e2

Browse files
yiztyizt
committed
更新readme
1 parent 7e4dbad commit cdc82e2

File tree

1 file changed

+98
-1
lines changed

1 file changed

+98
-1
lines changed

‎README.md

+98-1
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,11 @@
2020
6.2 [测试](#测试)<br>
2121
6.3 [Grad-CAM结果](#Grad-CAM结果)<br>
2222
6.4 [总结](#总结)
23+
7. [目标检测-fcos](#目标检测-fcos)<br>
24+
7.1 [AdelaiDet安装](#AdelaiDet安装)<br>
25+
7.2 [测试](#测试)<br>
26+
7.3 [Grad-CAM结果](#Grad-CAM结果)<br>
27+
7.4 [总结](#总结)
2328

2429
**Grad-CAM整体架构**
2530

@@ -449,4 +454,96 @@ a) retinanet的Grad-CAM图效果都不算好，相对来说中间层head.cls_sub
449454

450455
b) 个人认为retinanet效果不要的原因是，retinanet最后的分类是卷积层，卷积核实3\*3，也就是说反向传播到最后一个卷积层的feature map上，只有3\*3个单元有梯度。而分类网络或者faster r-cnn分类都是全连接层，感受全局信息，最后一个卷积层的feature map上所有单元都有梯度。
451456

452-
c) 反向传播到浅层的feature map上，有梯度的单元会逐渐增加，但是就像Grad-CAM论文中说的，越浅层的feature map语义信息越弱，所以可以看到head.cls_subnet.0的CAM图效果很差。
457+
c) 反向传播到浅层的feature map上，有梯度的单元会逐渐增加，但是就像Grad-CAM论文中说的，越浅层的feature map语义信息越弱，所以可以看到head.cls_subnet.0的CAM图效果很差。
458+
459+
460+
461+
## 目标检测-fcos
462+
463+
​ 在目标检测网络faster r-cnn和retinanet的Grad-CAM完成后，有两位网友[**linsy-ai**](<https://github.com/linsy-ai>) 问道怎样在fcos中实现Grad-CAM。fcos与retinanet基本类似，因为它们整体网络结构类似；这里使用[AdelaiDet](https://github.com/aim-uofa/AdelaiDet) 工程中的fcos网络，以下是详细的过程：
464+
465+
### AdelaiDet安装
466+
467+
a) 下载
468+
469+
```shell
470+
git clone https://github.com/aim-uofa/AdelaiDet.git
471+
```
472+
473+
b) 安装
474+
475+
```shell
476+
cd AdelaiDet
477+
python setup.py build develop
478+
```
479+
480+
注意：1. AdelaiDet安装依赖[detectron2](https://github.com/facebookresearch/detectron2.git),需要首先安装$\color{red}{detectron2}$
481+
482+
​ 2. fcos的不支持CPU,只支持GPU,请确保在$\color{red}{GPU环境}$下安装和测试
483+
484+
485+
486+
### 测试
487+
488+
a) 预训练模型下载
489+
490+
```shell
491+
wget https://cloudstor.aarnet.edu.au/plus/s/glqFc13cCoEyHYy/download -O fcos_R_50_1x.pth
492+
```
493+
494+
495+
496+
b) 测试Grad-CAM图像生成
497+
498+
​ 在本工程目录下执行如下命令:
499+
500+
```shell
501+
export CUDA_DEVICE_ORDER="PCI_BUS_ID"
502+
export CUDA_VISIBLE_DEVICES="0"
503+
python AdelaiDet/demo_fcos.py --config-file AdelaiDet/R_50_1x.yaml \
504+
--input ./examples/pic1.jpg \
505+
--layer-name proposal_generator.fcos_head.cls_tower.8 \
506+
--opts MODEL.WEIGHTS /path/to/fcos_R_50_1x.pth MODEL.DEVICE cuda
507+
```
508+
509+
510+
511+
### Grad-CAM结果
512+
513+
| | 图像1 | 图像2 | 图像3 | 图像4 |
514+
| ---------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
515+
| 原图 | ![](./examples/pic1.jpg) | ![](./examples/pic2.jpg) | ![](./examples/pic3.jpg) | ![](./examples/pic4.jpg) |
516+
| 预测边框 | ![](./results/pic1-fcos-predict_box.jpg) | ![](./results/pic2-fcos-predict_box.jpg) | ![](./results/pic3-fcos-predict_box.jpg) | ![](./results/pic4-fcos-predict_box.jpg) |
517+
| GradCAM-cls_tower.0 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap.jpg) |
518+
| GradCAM-cls_tower.1 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap.jpg) |
519+
| GradCAM-cls_tower.2 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap.jpg) |
520+
| GradCAM-cls_tower.3 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap.jpg) |
521+
| GradCAM-cls_tower.4 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap.jpg) |
522+
| GradCAM-cls_tower.5 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap.jpg) |
523+
| GradCAM-cls_tower.6 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap.jpg) |
524+
| GradCAM-cls_tower.7 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap.jpg) |
525+
| GradCAM-cls_tower.8 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap.jpg) |
526+
| GradCAM-cls_tower.9 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap.jpg) |
527+
| GradCAM-cls_tower.10 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap.jpg) |
528+
| GradCAM-cls_tower.11 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap.jpg) |
529+
| GradCAM++-cls_tower.0 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.0-heatmap++.jpg) |
530+
| GradCAM++-cls_tower.1 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.1-heatmap++.jpg) |
531+
| GradCAM++-cls_tower.2 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.2-heatmap++.jpg) |
532+
| GradCAM++-cls_tower.3 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.3-heatmap++.jpg) |
533+
| GradCAM++-cls_tower.4 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.4-heatmap++.jpg) |
534+
| GradCAM++-cls_tower.5 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.5-heatmap++.jpg) |
535+
| GradCAM++-cls_tower.6 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.6-heatmap++.jpg) |
536+
| GradCAM++-cls_tower.7 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.7-heatmap++.jpg) |
537+
| GradCAM++-cls_tower.8 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.8-heatmap++.jpg) |
538+
| GradCAM++-cls_tower.9 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.9-heatmap++.jpg) |
539+
| GradCAM++-cls_tower.10 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.10-heatmap++.jpg) |
540+
| GradCAM++-cls_tower.11 | ![](./results/pic1-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap++.jpg) | ![](./results/pic2-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap++.jpg) | ![](./results/pic3-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap++.jpg) | ![](./results/pic4-fcos-proposal_generator.fcos_head.cls_tower.11-heatmap++.jpg) |
541+
542+
543+
注：以上分别对proposal_generator.fcos_head.cls_tower..0~head.cls_subnet.11共12个层生成Grad-CAM图，这12层分别对应fcos分类子网络的4层卷积feature map、组标准化后的feature map及ReLu激活后的feature map
544+
545+
546+
547+
### 总结
548+
549+
​ 不总结了，看图效果吧！

0 commit comments

Comments
 (0)