Enot utility package
enot.utils package contains utility functional.
batch_norm
This package contains batch normalization layer utilities. Some examples are batch norm tuning functionality, resetting all batch norm layers and checking if module is an instance of PyTorch’s BatchNorm class.
- is_bn(module)
Checks whether torch.nn.Module is a BatchNorm instance or not.
- Parameters:
module (torch.nn.Module) – Module to check.
- Returns:
Whether input module is a BatchNorm instance or not.
- Return type:
- reset_bn(module)
Resets module running stats if module is a batch norm layer.
- Parameters:
module (torch.nn.Module) – Module to reset running stats (if module is an instance of torch BatchNorm).
- Return type:
- model_reset_bn(model)
Resets running stats in all batch norm layers.
- Parameters:
model (torch.nn.Module) – Model in which the running stats of all batch norm layers will be reset.
- Return type:
- model_set_bn_momentum(model, momentum=None)
Sets momentum value in all batch norm layers.
- Parameters:
model (torch.nn.Module) – Model to update batch norms.
momentum (float or None, optional) – Momentum value to set in all batch norm layers. Default value is None.
- Return type:
- tune_bn_stats(model, dataloader, reset_bns=False, set_momentums_none=False, n_steps=None, epochs=1, sample_to_model_inputs=<function default_sample_to_model_inputs>, verbose=0)
Tunes batch norm running statistics of the model.
- Parameters:
model (Module) – Model to update batch norms.
dataloader (torch.utils.data.DataLoader) – Dataloader which generates data that will be used to update model’s running statistics.
reset_bns (bool, optional) – Whether to reset ALL of the running statistics before tuning. Default value is False.
set_momentums_none (bool, optional) – Whether to set all of the momentums in batch norms to None. Default value is False.
n_steps (int or None, optional) – Number of steps in one epoch of batch norm tuning. Defaults to None, which completes each epoch.
epochs (int, optional) – Number of epochs to tune batch norms. Default value is 1.
sample_to_model_inputs (Callable, optional) – Function to map dataloader samples to model input format. Default value is
default_sample_to_model_inputs()
. See more here.verbose (int) – Procedure verbosity level. 0 disables all messages, 1 enables
tqdm
progress bar logging.
- Return type:
Notes
Typically, it is better to tune batch norm statistics on the same data you trained your model. When tuning batch norms on the holdout set, you may experience performance degradation due to the distribution shift.
data
This package contains data-related functional. You can find dataloader creation functions, dataset classes and image transformation functions.
csv_annotation_dataset
- class CsvAnnotationDataset(csv_annotation_path, root_dir=None, transform=None)
Bases:
Dataset
Creates dataset from csv file.
Read CSV annotation with fields ‘filepath’: str, ‘label’: int
- __init__(csv_annotation_path, root_dir=None, transform=None)
Init vision dataset from CSV-file.
- Parameters:
csv_annotation_path (Union[str, Path]) – Path to csv-file with [‘filepath’, ‘label’] columns.
root_dir (Union[str, Path]) – Optional absolute path to folder with images which prepends to ‘filepath’ field in csv. Useful for handle with different dataset locations.
transform (torchvision.transforms.Compose) – Transformation should be applied to image.
dataloaders
- get_default_train_transform(input_size, mean, std)
Returns common train augmentation.
Augments images via RandomCrop and flip, also transforms it into normalized tensor.
- get_default_validation_transform(input_size, mean, std)
Returns common validation transformations.
No train augmentation presented, only Resize and normalization.
- create_data_loader(dataset, batch_size, *, check_sampler=True, sampler=None, collate_fn=<function fast_collate>, **kwargs)
Creates dataloader for certain dataset.
Creates dataloader each sample of one will be placed automatically to CUDA.
- Parameters:
dataset (torch.utils.data.Dataset) – Data.
batch_size (int) – How much samples should be processing at once.
check_sampler (bool) – Should we check sample object. Important for multi-GPU procedure.
sampler (torch.utils.data.Sampler) – Way to sample object from dataset.
collate_fn (TCollateFunction) – Custom collate function.
kwargs – Additional parameters propagated to dataloader.
- Returns:
Custom dataloader.
- Return type:
- create_data_loader_from_csv_annotation(csv_annotation_path, dataset_root_dir=None, dataset_transform=None, **kwargs)
Creates dataloader from csv-file.
- Parameters:
csv_annotation_path (Union[str, Path]) – Path to csv-file with [‘filepath’, ‘label’] columns.
dataset_root_dir (Union[str, Path]) – Optional absolute path to folder with images which prepends to ‘filepath’ field in csv. Useful for handle with different dataset locations.
dataset_transform (torchvision.transforms.Compose) – Transformation should be applied to image.
kwargs – Additional parameters propagated to dataloader.
- Returns:
Dataloader from csv file.
- Return type:
common
This package contains regular PyTorch utility functions which can be used outside architecture optimization pipeline.
- is_floating_tensor(tensor)
Checks that tensor has floating point data type.
- Parameters:
tensor (torch.Tensor) – Tensor to check.
- Returns:
Whether input tensor is floating point tensor (one of float16, float32 or float64).
- Return type:
- profile_speed(fn, *fn_args, sort_type='cumtime', out_functions=80)
Profile function and print profile stats.
- Parameters:
fn (callable) – Target function to profile.
fn_args (args) – Target function arguments.
sort_type (str, optional) – Type of statistics sorting. Full list of possible values is available in the python profile documentation. Default value is “cumtime”.
out_functions (int, optional) – Number of top-n functions that profiler will print. Default value is 80.
- Returns:
Result of function call.
- Return type:
Any
- save(model, model_path)
Saves module state_dict.
- Parameters:
model (torch.nn.Module) – Module with state_dict.
model_path (str or Path) – Path to model state_dict checkpoint.
- Return type:
- load(model, model_path, cpu=True, strict=True)
Inplace load state_dict from model_path to model.
- Parameters:
- Return type:
- init_convnet_params(model, fc_std=0.01)
Initializes convolutional neural network weights, following Kaiming He strategy.
- Parameters:
model (torch.nn.Module) – Convolutional neural network to initialize.
fc_std (float, optional) – Standard deviation of Linear layers. Default value is 0.01.
- Return type:
- iterate_by_submodules(root_module, submodule_class=<class 'torch.nn.modules.module.Module'>, include_self=False)
Creates iterator over submodules with a specific type.
- Parameters:
root_module (torch.nn.Module) – Top level module for selection.
submodule_class (type, optional) – Type of submodules to iterate over. Default value is torch.nn.Module.
include_self (bool, optional) – Whether to include
root_module
(when type ofroot_module
is equal tosubmodule_class
) in selected submodules or not. Default value is False.
- Returns:
Iterator over submodules with a specific type.
- Return type:
iterator
- cast_to_numpy(storage, copy=True)
Moves PyTorch tensor or numpy.ndarray to numpy.ndarray.
- Parameters:
storage (torch.Tensor, numpy.ndarray, float or int) – Data container that will be casted to numpy.ndarray.
copy (bool, optional) – Whether copy is strongly necessary. If copy is set to true, then this function guarantees that the resulting container will be the copy of the original one. Otherwise it is not guarantied whether copy will be made or not (as moving torch tensor from cuda to cpu will force to make a copy). Default value is True.
- Returns:
Container converted to numpy and optionally copied.
- Return type:
- replace_ops(model, accept_lambda, module_constructor, *args, **kwargs)
According to accept_lambda, replaces some modules in the model.
- Parameters:
model (torch.nn.Module) – Model where we recursively replace attributes.
accept_lambda (Callable) – Function that returns true if module should be replaced and returns False otherwise.
module_constructor (Callable) – Function which creates new operation instead of previous one.
- Return type:
train
This package contains train-related utility functions.
- split_into_groups(iterable, group_size, length=None, drop_last=False)
Create iterator with groups of samples from iterable.