.. _ENOT Tutorials: ######### Tutorials ######### .. _E1 ref: `1. Tutorial - automatic quantization for enot-lite`_ ===================================================== .. _1. Tutorial - automatic quantization for enot-lite: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/1.%20Tutorial%20-%20automatic%20quantization%20for%20enot-lite.ipynb This notebook shows how to apply enot-autodl framework for automatic quantization to create quantized model for enot-lite framework. .. _E2 ref: `2. Tutorial - pruning`_ ================================== .. _2. Tutorial - pruning: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/2.%20Tutorial%20-%20pruning.ipynb This experimental notebook shows how to apply enot-autodl framework for automatic network pruning and fine-tuning. .. _E3 ref: `3. Tutorial - pruning (manual)`_ =========================================== .. _3. Tutorial - pruning (manual): https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/3.%20Tutorial%20-%20pruning%20(manual).ipynb This notebook shows how to apply enot-autodl framework for automatic network pruning and fine-tuning. Gradient accumulation is performed manually. .. _E4 ref: `4. Tutorial - Ultralytics YOLO-v5 quantization`_ ================================================= .. _4. Tutorial - Ultralytics YOLO-v5 quantization: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/4.%20Tutorial%20-%20Ultralytics%20YOLO-v5%20quantization.ipynb This notebook shows how to apply enot-autodl framework for automatic network quantization of Ultralytics YOLO-v5. .. _E5 ref: `5. Tutorial - ENOT baseline optimizer`_ ================================================= .. _5. Tutorial - ENOT baseline optimizer: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/5.%20Tutorial%20-%20ENOT%20baseline%20optimizer.ipynb This notebook describes how to use ENOT optimizer. ################## Advanced tutorials ################## .. _EA1 ref: `1. Tutorial - getting started with NAS`_ ========================================= .. _1. Tutorial - getting started with NAS: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/1.%20Tutorial%20-%20getting%20started%20with%20NAS.ipynb Describes the basic steps you need to optimize an architecture: - Create your model and move it into SearchSpace; - Pretrain search space on target task (Imagenette image classification in this example); - Search the best architecture for your task in search space; - Tune this model. .. _EA2 ref: `2. Tutorial - search space autogeneration`_ ============================================ .. _2. Tutorial - search space autogeneration: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/2.%20Tutorial%20-%20search%20space%20autogeneration.ipynb This notebook describes how to automatically generate search space from your model. .. _EA3 ref: `3. Tutorial - custom model`_ ============================= .. _3. Tutorial - custom model: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/3.%20Tutorial%20-%20custom%20model.ipynb This notebook describes the ways to implement your own model. Typical use cases: - Creating models which can not be built by "block models builders" from ENOT framework; - You already have your model, and you don't want to rewrite your code. .. _EA4 ref: `4. Tutorial - latency calculation`_ ==================================== .. _4. Tutorial - latency calculation: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/4.%20Tutorial%20-%20latency%20calculation.ipynb This notebook describes how to calculate latency using ENOT framework. Main chapters of this notebook: - Initialize latency of search space (SearchSpaceModel); - Calculate latency of an arbitrary model/module. .. _EA5 ref: `5. Tutorial - search with the specified latency`_ =================================================== .. _5. Tutorial - search with the specified latency: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/5.%20Tutorial%20-%20search%20with%20the%20specified%20latency.ipynb This notebook describes how to search only architectures with latency strictly lower than the specified value. .. _EA6 ref: `6. Tutorial - resolution search for image classification`_ ============================================================ .. _6. Tutorial - resolution search for image classification: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/6.%20Tutorial%20-%20resolution%20search%20for%20image%20classification.ipynb This notebook shows how to search optimal image resolution for computer vision task. In this tutorial, we search model and resolution with the best accuracy and fixed maximum latency. .. _EA7 ref: `7. Tutorial - search space autogeneration (EfficientNet-V2 S)`_ ================================================================= .. _7. Tutorial - search space autogeneration (EfficientNet-V2 S): https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/7.%20Tutorial%20-%20search%20space%20autogeneration%20(EfficientNet-V2%20S).ipynb This notebook describes how to generate search space automatically from EfficientNet-V2 Small model. This example uses `pytorch-image-models`_ EfficientNet-V2 implementation. .. _pytorch-image-models: https://github.com/rwightman/pytorch-image-models .. _EA8 ref: `8. Tutorial - metric learning`_ ================================= .. _8. Tutorial - metric learning: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/8.%20Tutorial%20-%20metric%20learning.ipynb This notebook shows an advanced example usage of ENOT framework. We will show how to use angular margin loss for metric learning task. .. _EA9 ref: `9. Tutorial - evolution search`_ ================================== .. _9. Tutorial - evolution search: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/advanced/9.%20Tutorial%20-%20evolution%20search.ipynb This notebook describes network architecture optimization using evolution search. It is useful in case of hardware aware architecture optimization. .. _Distributed (multi-gpu / multi-node) pretrain example reference: `Distributed (multi-gpu / multi-node) pretrain example`_ ======================================================== .. _Distributed (multi-gpu / multi-node) pretrain example: https://github.com/ENOT-AutoDL/ENOT_Tutorials/tree/v%ENOT_VERSION%/multigpu_pretrain .. _.sh script: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/multigpu_pretrain/run_multigpu_pretrain.sh .. _.py script: https://github.com/ENOT-AutoDL/ENOT_Tutorials/blob/v%ENOT_VERSION%/multigpu_pretrain/multigpu_pretrain.py In this folder you can find `.sh script`_ for running multi-gpu pretrain. You can change it's configuration to run on single GPU, on multiple GPU within a single node, or on multiple compute nodes with multiple GPUs. The second file in this folder is a `.py script`_ which is launched by .sh script. This script uses functions from other tutorials, but is adapted to run in distributed manner. This script should be viewed as a reference point for user-defined distributed pretrain scripts. Distributed search is not recommended as it is under development. Moreover, the search procedure is usually relatively fast. At the tuning stage, you will have a regular model without any of the ENOT specifics, so it is your responsibility to write correct distributed code (probably by wrapping found model with `DistributedDataParallel module `_).