Documentation for utils.py

create_balanced_distributions

Combines outputs from feature_distribution and balanced_sample_partition to create class-balanced feature and sample partitions for the dataset.

balanced_sample_partition

Creates class-balanced sample partitions for the dataset.

Parameters:

Name	Type	Description	Default
labels	np.ndarray	data labels.	required
n_hbins	int	number of sample partiions of the data.	required

Returns:

Name	Type	Description
sample_partitions	np.ndarray	Class balanced version of sample_distribution.

sample_distribution

Creates sample bins for the dataset.

Parameters:

Name	Type	Description	Default
n_samples	int	number of samples in the data.	required
n_hbins	int	number of sample partiions of the data.	required
sample	np.ndarray, optional	if sample is not None, then the sample indexes are taken from sample, otherwise they are generated from np.arange(n_samples). sample argument required for creating balanced sample partitions.	None

Returns:

Name	Type	Description
sample_partitions	np.ndarray	Contains in each column the sample indexes of the samples that belong to that sample bin.

feature_distribution

Function that creates feature bins for the features in the dataset for use by DRFSC.

Parameters:

Name	Type	Description	Default
n_feats	int	number of features in the data	required
n_vbins	int	number of feature partitions of the data.	required

Returns:

Name	Type	Description
feature_partitions	np.ndarray	Contains in each column the features that belong to that feature bin

scale_data

Uses sklearn.preprocessing to rescale feature values to [0,1].

Parameters:

Name	Type	Description	Default
data	np.ndarray or pd.DataFrame	data to be transformed	required

Returns:

Name	Type	Description
data_out	np.ndarray or pd.DataFrame	[0,1] transform of data

extend_features

Uses sklearn.preprocessing to create polynomial features of data and add bias term.

Parameters:

Name	Type	Description	Default
data	np.ndarray or pd.DataFrame	data to be transformed	required
degree	int, optional	Degree of non-linearity to generate. Defaults to 1. By default just adds a bias term.	1

Returns:

Name	Type	Description
data_out	np.ndarray or pd.DataFrame	Polynomial transformed data

evaluate_model

Evaluates the performance of a model on the test set.

Parameters:

Name	Type	Description	Default
model_features	list	Subset of features to be included in the model	required
X_train	np.ndarray	Training data	required
X_val	np.ndarray	Validation data	required
Y_train	np.ndarray	Training labels	required
Y_val	np.ndarray	Validation labels	required
metric	str	metric used to evaluate model performance	required

Returns:

Name	Type	Description
model_final	object	Fitted logistic regression model. See statsmodels.Logit
model_performance	float	Performance of the model on the validation set

evaluate_interim_model

Evaluates the performance of a model on the validation set.

Parameters:

Name	Type	Description	Default
model_features	list	Subset of features to be included in the model	required
X_train	np.ndarray	Training data	required
X_val	np.ndarray	Validation data	required
Y_train	np.ndarray	Training labels	required
Y_val	np.ndarray	Validation labels	required
metric	str	metric used to evaluate model performance	required

Returns:

Name	Type	Description
model_final	object	Fitted logistic regression model. See statsmodels.Logit
model_performance	float	Performance of the model on the validation set

model_score

Evalutates model performance based on specified metric using sklearn.metrics.

Parameters:

Name	Type	Description	Default
method	str	metric used to evaluate model performance	required
y_true	np.ndarray	{0,1} ground truth labels	required
y_pred_label	np.ndarray	{0,1} predicted labels	required
y_pred_prob	np.ndarray	[0,1] predicted probabilities	required

Returns:

Name	Type	Description
out	float	output based on metric