forked from xuos/xiuos
1 changed files with 167 additions and 0 deletions
@ -0,0 +1,167 @@ |
|||
# Helmet detection demo |
|||
|
|||
### A helmet and head without helmet object detection task demo. Running MobileNet-yolo on K210-based edge devices. |
|||
|
|||
--- |
|||
|
|||
## Training |
|||
|
|||
### Enviroment preparation |
|||
|
|||
Model generated by [aXeleRate](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate) and converted to kmodel by [nncase](https://github.com/kendryte/nncase/tree/v0.1.0-rc5). |
|||
|
|||
```shell |
|||
# master branch for MobileNetv1-yolov2 and unstable branch to test MobileNetv1(v2)-yolov2(v3) |
|||
git clone https://git.trustie.net/yangtuo250/aXeleRate.git (-b unstable) |
|||
cd aXeleRate |
|||
pip install -r requirments.txt && pip install -e . |
|||
``` |
|||
|
|||
### training config setting |
|||
|
|||
Example [config](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate/tree/master/configs/detector.json), some hyper-parameters: |
|||
|
|||
- architecture: backbone, MobileNet7_5 for default, MobileNet1_0(α = 1.0) and above cannot run on K210 because of OOM on feature map in master branch. For unstable branch MobileNetV2_1_0 is OK. |
|||
|
|||
- input_size: fixed model input size, single integer for height equals to width, otherwise a list([height, width]). |
|||
- anchors: yolov2 anchor(for master) or anchor scaled to 1.0(for unstable), can be generate by [darknet](https://github.com/AlexeyAB/darknet). |
|||
- labels: labels of all classes. |
|||
- train(valid)_image(annot)_folder: path of images and annoations for training and validation. |
|||
- saved_folder: path for trainig result storage(models, checkpoints, logs ...). |
|||
|
|||
Mine config for unstable: |
|||
```json |
|||
{ |
|||
"model": { |
|||
"type": "Detector", |
|||
"architecture": "MobileNetV2_1_0", |
|||
"input_size": [ |
|||
224, |
|||
320 |
|||
], |
|||
"anchors": [ |
|||
[ |
|||
[ |
|||
0.1043, |
|||
0.1560 |
|||
], |
|||
[ |
|||
0.0839, |
|||
0.3036 |
|||
], |
|||
[ |
|||
0.1109, |
|||
0.3923 |
|||
], |
|||
[ |
|||
0.1378, |
|||
0.5244 |
|||
], |
|||
[ |
|||
0.2049, |
|||
0.6673 |
|||
] |
|||
] |
|||
], |
|||
"labels": [ |
|||
"human" |
|||
], |
|||
"obj_thresh": 0.5, |
|||
"iou_thresh": 0.45, |
|||
"coord_scale": 1.0, |
|||
"class_scale": 0.0, |
|||
"object_scale": 5.0, |
|||
"no_object_scale": 3.0 |
|||
}, |
|||
"weights": { |
|||
"full": "", |
|||
"backend": "" |
|||
}, |
|||
"train": { |
|||
"actual_epoch": 2000, |
|||
"train_image_folder": "mydata/human/Images/train", |
|||
"train_annot_folder": "mydata/human/Annotations/train", |
|||
"train_times": 2, |
|||
"valid_image_folder": "mydata/human/Images/val", |
|||
"valid_annot_folder": "mydata/human/Annotations/val", |
|||
"valid_times": 1, |
|||
"valid_metric": "precision", |
|||
"batch_size": 32, |
|||
"learning_rate": 2e-5, |
|||
"saved_folder": "mydata/human/results", |
|||
"first_trainable_layer": "", |
|||
"augmentation": true, |
|||
"is_only_detect": false, |
|||
"validation_freq": 5, |
|||
"quantize": false, |
|||
"class_weights": [1.0] |
|||
}, |
|||
"converter": { |
|||
"type": [ |
|||
"k210" |
|||
] |
|||
} |
|||
} |
|||
``` |
|||
|
|||
*(For more detailed config usage, please refer to original aXeleRate repo.)* |
|||
|
|||
### data preparation |
|||
|
|||
Please refer to [VOC format](https://towardsdatascience.com/coco-data-format-for-object-detection-a4c5eaf518c5), path as config above. |
|||
|
|||
### train it! |
|||
|
|||
```shell |
|||
python -m aXeleRate.train -c PATH_TO_YOUR_CONFIG |
|||
``` |
|||
|
|||
### model convert |
|||
|
|||
Please refer to [nncase repo](https://github.com/kendryte/nncase/tree/v0.1.0-rc5). |
|||
|
|||
--- |
|||
|
|||
## Deployment |
|||
|
|||
### compile and burn |
|||
|
|||
Use `(scons --)menuconfig` in bsp folder *(Ubiquitous/RT_Thread/bsp/k210)*, open: |
|||
|
|||
- More Drivers --> ov2640 driver |
|||
- Board Drivers Config --> Enable LCD on SPI0 |
|||
- Board Drivers Config --> Enable SDCARD (spi1(ss0)) |
|||
- Board Drivers Config --> Enable DVP(camera) |
|||
- RT-Thread Components --> POSIX layer and C standard library --> Enable pthreads APIs |
|||
- APP_Framework --> Framework --> support knowing framework --> kpu model postprocessing --> yolov2 region layer |
|||
- APP_Framework --> Applications --> knowing app --> enable apps/helmet detect |
|||
|
|||
`scons -j(n)` to compile and burn in by *kflash*. |
|||
|
|||
### json config and kmodel |
|||
|
|||
Copy json config for deployment o SD card */kmodel*. Example config file is *helmet.json* in this directory. Something to be modified: |
|||
|
|||
- net_input_size: same as *input_size* in training config file, but array only. |
|||
- net_output_shape: final feature map size, can be found in **nncase** output. |
|||
- sensor_output_size: image height and width from camera, same as *dvp_set_image_size(Ubiquitous/RT_Thread/bsp/k210/base-drivers/drv_dvp.c L70(71))*. |
|||
- kmodel_size: kmodel size shown in file system. |
|||
- anchors: same as *anchor* in training config file(multi-dimention anchors flatten to 1 dim). |
|||
- labels: same as *label* in training config file. |
|||
- obj_thresh: array, object threshold of each label. |
|||
- nms_thresh: NMS threshold of boxes. |
|||
|
|||
Copy final kmodel to SD card */kmodel* either. |
|||
|
|||
--- |
|||
|
|||
## Run |
|||
|
|||
In serial terminal, `helmet_detect` to start a detection thread, `helmet_detect_delete` to stop it. Detection results can be found in output. |
|||
|
|||
--- |
|||
|
|||
## TODO |
|||
|
|||
- [ ] Fix LCD real-time result display. |
|||
- [ ] Test more object detection backbone and algorithm(like yolox). |
Loading…
Reference in new issue