dair_pll.file_utils

Utility functions for managing saved files for training models.

File system is organized around a “storage” directory associated with data and training runs. The functions herein can be used to return absolute paths of and summary information about the content of this directory.

dair_pll.file_utils.CHECKPOINT_FILENAME = 'checkpoint.pt'

extensions for saved files

Type:

str

dair_pll.file_utils.assure_created(directory)

Wrapper to put around directory paths which ensure their existence.

Parameters:

directory (str) – Path of directory that may not exist.

Return type:

str

Returns:

directory, Which is ensured to exist by recursive mkdir.

dair_pll.file_utils.get_asset(asset_file_basename)

Gets

Parameters:

asset_file_basename (str) – Basename of asset file located in ASSET_DIR

Return type:

str

Returns:

Asset’s absolute path.

dair_pll.file_utils.assure_storage_tree_created(storage_name)

Assure that all subdirectories of specified storage are created.

Parameters:

storage_name (str) – name of storage directory.

Return type:

None

dair_pll.file_utils.import_data_to_storage(storage_name, import_data_dir)

Import data in external folder into data directory.

Parameters:

• storage_name (str) – Name of storage for data import.

• import_data_dir (str) – Directory to import data from.

Return type:

None

dair_pll.file_utils.storage_dir(storage_name)

Absolute path of storage directory

Return type:

str

dair_pll.file_utils.data_dir(storage_name)

Absolute path of data folder.

Return type:

str

dair_pll.file_utils.learning_data_dir(storage_name)

Absolute path of folder for data preprocessed for training/validation.

Return type:

str

dair_pll.file_utils.ground_truth_data_dir(storage_name)

Absolute path of folder for raw unprocessed trajectories.

Return type:

str

dair_pll.file_utils.all_runs_dir(storage_name)

Absolute path of tensorboard storage folder

Return type:

str

dair_pll.file_utils.all_studies_dir(storage_name)

Absolute path of tensorboard storage folder

Return type:

str

dair_pll.file_utils.delete(file_name)

Removes file at path specified by file_name

Return type:

None

dair_pll.file_utils.get_numeric_file_count(directory, extension='.pt')

Count number of whole-number-named files.

If folder /fldr has contents (7.pt, 11.pt, 4.pt), then:

get_numeric_file_count("/fldr", ".pt") == 3

Parameters:

• directory (str) – Directory to tally file count in

• extension (str) – Extension of files to be counted

Return type:

int

Returns:

Number of files in specified directory with specified extension that have an integer basename.

dair_pll.file_utils.get_trajectory_count(trajectory_dir)

Count number of trajectories on disk in given directory.

dair_pll.file_utils.trajectory_file(trajectory_dir, num_trajectory)

Absolute path of specific trajectory in storage

Return type:

str

dair_pll.file_utils.run_dir(storage_name, run_name)

Absolute path of run-specific storage folder.

Return type:

str

dair_pll.file_utils.get_trajectory_video_filename(storage_name, run_name)

Return the filepath of the temporary rollout video gif.

Return type:

str

dair_pll.file_utils.get_learned_urdf_dir(storage_name, run_name)

Absolute path of learned model URDF storage directory.

Return type:

str

dair_pll.file_utils.wandb_dir(storage_name, run_name)

Absolute path of tensorboard storage folder

Return type:

str

dair_pll.file_utils.get_evaluation_filename(storage_name, run_name)

Absolute path of experiment run statistics file.

Return type:

str

dair_pll.file_utils.get_configuration_filename(storage_name, run_name)

Absolute path of experiment configuration.

Return type:

str

dair_pll.file_utils.get_model_filename(storage_name, run_name)

Absolute path of experiment configuration.

Return type:

str

dair_pll.file_utils.study_dir(storage_name, study_name)

Absolute path of study-specific storage folder.

Return type:

str

dair_pll.file_utils.hyperparameter_opt_run_name(study_name, trial_number)

Experiment run name for hyperparameter optimization trial.

Return type:

str

dair_pll.file_utils.sweep_run_name(study_name, sweep_run, n_train)

Experiment run name for dataset size sweep study.

Return type:

str

dair_pll.file_utils.get_hyperparameter_filename(storage_name, study_name)

Absolute path of optimized hyperparameters for a study

Return type:

str

dair_pll.file_utils.load_binary(filename, load_callback)

Load binary file

Return type:

Any

dair_pll.file_utils.load_string(filename, load_callback=None)

Load text file

Return type:

Any

dair_pll.file_utils.save_binary(filename, value, save_callback)

Save binary file.

Return type:

None

dair_pll.file_utils.save_string(filename, value, save_callback=None)

Save text file.

Return type:

None

dair_pll.file_utils.load_configuration(storage_name, run_name)

Load configuration file.

Return type:

SupervisedLearningExperimentConfig

dair_pll.file_utils.save_configuration(storage_name, run_name, config)

Save configuration file.

Return type:

None

dair_pll.file_utils.load_evaluation(storage_name, run_name)

Load evaluation file.

Return type:

Any

dair_pll.file_utils.save_evaluation(storage_name, run_name, evaluation)

Save evaluation file.

Return type:

None

dair_pll.file_utils.load_hyperparameters(storage_name, study_name)

Load hyperparameter file.

Return type:

Any

dair_pll.file_utils.save_hyperparameters(storage_name, study_name, hyperparameters)

Save hyperparameter file.

Return type:

None