docs(knowing app): add README for helmet detect

2 years ago · fc825ea557
1 changed files with 167 additions and 0 deletions
--- a/APP_Framework/Applications/knowing_app/helmet_detect/README.md
+++ b/APP_Framework/Applications/knowing_app/helmet_detect/README.md
@ -0,0 +1,167 @@
+# Helmet detection demo
+
+### A helmet and head without helmet object detection task demo. Running MobileNet-yolo on K210-based edge devices.
+
+---
+
+## Training
+
+### Enviroment preparation
+
+Model generated by [aXeleRate](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate) and converted to kmodel by [nncase](https://github.com/kendryte/nncase/tree/v0.1.0-rc5).
+
+```shell
+# master branch for MobileNetv1-yolov2 and unstable branch to test MobileNetv1(v2)-yolov2(v3)
+git clone https://git.trustie.net/yangtuo250/aXeleRate.git (-b unstable)
+cd aXeleRate
+pip install -r requirments.txt && pip install -e .
+```
+
+### training config setting
+
+Example [config](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate/tree/master/configs/detector.json), some hyper-parameters:
+
+- architecture: backbone, MobileNet7_5 for default, MobileNet1_0(α = 1.0) and above cannot run on K210 because of OOM on feature map in master branch. For unstable branch MobileNetV2_1_0 is OK.
+
+- input_size: fixed model input size, single integer for height equals to width, otherwise a list([height, width]).
+- anchors: yolov2 anchor(for master) or anchor scaled to 1.0(for unstable), can be generate by [darknet](https://github.com/AlexeyAB/darknet).
+- labels: labels of all classes.
+- train(valid)_image(annot)_folder: path of images and annoations for training and validation.
+- saved_folder: path for trainig result storage(models, checkpoints, logs ...).
+
+Mine config for unstable:
+```json
+{
+    "model": {
+        "type": "Detector",
+        "architecture": "MobileNetV2_1_0",
+        "input_size": [
+            224,
+            320
+        ],
+        "anchors": [
+            [
+                [
+                    0.1043,
+                    0.1560
+                ],
+                [
+                    0.0839,
+                    0.3036
+                ],
+                [
+                    0.1109,
+                    0.3923
+                ],
+                [
+                    0.1378,
+                    0.5244
+                ],
+                [
+                    0.2049,
+                    0.6673
+                ]
+            ]
+        ],
+        "labels": [
+            "human"
+        ],
+        "obj_thresh": 0.5,
+        "iou_thresh": 0.45,
+        "coord_scale": 1.0,
+        "class_scale": 0.0,
+        "object_scale": 5.0,
+        "no_object_scale": 3.0
+    },
+    "weights": {
+        "full": "",
+        "backend": ""
+    },
+    "train": {
+        "actual_epoch": 2000,
+        "train_image_folder": "mydata/human/Images/train",
+        "train_annot_folder": "mydata/human/Annotations/train",
+        "train_times": 2,
+        "valid_image_folder": "mydata/human/Images/val",
+        "valid_annot_folder": "mydata/human/Annotations/val",
+        "valid_times": 1,
+        "valid_metric": "precision",
+        "batch_size": 32,
+        "learning_rate": 2e-5,
+        "saved_folder": "mydata/human/results",
+        "first_trainable_layer": "",
+        "augmentation": true,
+        "is_only_detect": false,
+        "validation_freq": 5,
+        "quantize": false,
+        "class_weights": [1.0]
+    },
+    "converter": {
+        "type": [
+            "k210"
+        ]
+    }
+}
+```
+
+*(For more detailed config usage, please refer to original aXeleRate repo.)*
+
+### data preparation
+
+Please refer to [VOC format](https://towardsdatascience.com/coco-data-format-for-object-detection-a4c5eaf518c5), path as config above.
+
+### train it!
+
+```shell
+python -m aXeleRate.train -c PATH_TO_YOUR_CONFIG
+```
+
+### model convert
+
+Please refer to [nncase repo](https://github.com/kendryte/nncase/tree/v0.1.0-rc5).
+
+---
+
+## Deployment
+
+### compile and burn
+
+Use `(scons --)menuconfig` in bsp folder *(Ubiquitous/RT_Thread/bsp/k210)*, open:
+
+- More Drivers --> ov2640 driver
+- Board Drivers Config --> Enable LCD on SPI0
+- Board Drivers Config --> Enable SDCARD (spi1(ss0))
+- Board Drivers Config --> Enable DVP(camera)
+- RT-Thread Components --> POSIX layer and C standard library --> Enable pthreads APIs
+- APP_Framework --> Framework --> support knowing framework --> kpu model postprocessing --> yolov2 region layer
+- APP_Framework --> Applications --> knowing app --> enable apps/helmet detect
+
+`scons -j(n)` to compile and burn in by *kflash*.
+
+### json config and kmodel
+
+Copy json config for deployment o SD card */kmodel*. Example config file is *helmet.json* in this directory. Something to be modified:
+
+- net_input_size: same as *input_size* in training config file, but array only.
+- net_output_shape: final feature map size, can be found in **nncase** output.
+- sensor_output_size: image height and width from camera, same as *dvp_set_image_size(Ubiquitous/RT_Thread/bsp/k210/base-drivers/drv_dvp.c L70(71))*.
+- kmodel_size: kmodel size shown in file system.
+- anchors: same as *anchor* in training config file(multi-dimention anchors flatten to 1 dim).
+- labels: same as *label* in training config file.
+- obj_thresh: array, object threshold of each label.
+- nms_thresh: NMS threshold of boxes.
+
+Copy final kmodel to SD card */kmodel* either.
+
+---
+
+## Run
+
+In serial terminal, `helmet_detect` to start a detection thread, `helmet_detect_delete` to stop it. Detection results can be found in output.
+
+---
+
+## TODO
+
+- [ ] Fix LCD real-time result display.
+- [ ] Test more object detection backbone and algorithm(like yolox).