#########################
ENOT optimization package
#########################

enot.optimize package contains optimizer definitions for three main neural
architecture search stages.

To optimize a neural network architecture, you should do the following:

- Train the search space (pretrain)
- Search for the optimal architecture (search)
- Tune obtained model (tune)

.. autofunction:: enot.optimize.utils.build_optimizer


*************************
Search space pre-training
*************************

Neural architecture search starts with the search space creation and its
pretrain procedure for further selection of the best operations.

.. autoclass:: enot.optimize.pretrain.PretrainOptimizer
    :members: __init__, state_dict, load_state_dict, zero_grad, add_param_group, model, model_step, step, search_space


***********************
Optimal model selection
***********************

The pretrained search space is used to select the best combination of
operations.

.. autoclass:: enot.optimize.search.SearchOptimizer
    :members: __init__, state_dict, load_state_dict, zero_grad, add_param_group, model, model_step, step, search_space, prepare_validation_model, bn_tune_batches, bn_validation_tune_batches

.. autoclass:: enot.optimize.search.FixedLatencySearchOptimizer
    :members: __init__, state_dict, load_state_dict, zero_grad, add_param_group, model, model_step, step, search_space, prepare_validation_model, bn_tune_batches, bn_validation_tune_batches, max_latency_value


********************
Optimal model tuning
********************

Final model is tuned for some epochs to match it's standalone accuracy.
Sometimes, however, it is necessary to re-train the obtained architecture from
scratch.

.. autoclass:: enot.optimize.utils.TrainOptimizer
    :members: __init__, state_dict, load_state_dict, zero_grad, add_param_group, model, model_step, step