ML_PT_Pyworkflow documentation

Module Documentation

class ML_PT_Pyworkflow.Dataset

Bases: object

ArcPl = 'External test, Filtered ArcPl compilation, original data from:Wieser et al.(2023)'

Jorgenson = 'dataset used for training and testing.'

Test_P = 'Internal test (20 % of original dataset) used to evaluate pressure estimates using cpx and  cpx_liquid models.'

Test_T = 'Internal test (20 % of original dataset) used to evaluate temperature estimates using cpx and cpx_liquid models.'

Train_P = 'training dataset for cpx and cpx_liquid barometers'

Train_T = 'training dataset for cpx and cpx_liquid thermometer models.'

Unknowns = 'The user unknowns'

ML_PT_Pyworkflow.ET_train_validation_balanced(X_train, y_train, bins, k_fold=10, n_estimators=200, max_depth=15, pwlt=False)

Cross-validation process for Extra-Trees. It can manage unbalanced data as output

Args:

X_train (numpy.array 2D): Input dataset. y_train (np.array 1D): Output dataset. bins (np.array 1D): index of dependent samples

Returns:

model: model trained total_score_avg (np.float): score from cross-validation

ML_PT_Pyworkflow.ET_train_validation_balanced_perturbation(X_train, y_train, std_dev_perc, bins, k_fold=10, n_estimators=200, max_depth=15, n_perturbations=15, pwlt=False)

Cross validation process for Extra-Trees. It can manage unbalanced data as output and dependencies between the input samples generated via data augmentation

Args:

X_train (numpy.array 2D): Input dataset. y_train (np.array 1D): Output dataset. bins (np.array 1D): index of dependent samples std_dev_perc (np.array 1D, or np.array 2D): standard deviation percentage. If you upload a 1D-array, all new samples are based on the same standard deviation, if you upload a 2D-array, new samples comes from different standard deviations.

Returns:

model: model trained total_score_avg (np.float): score from cross-validation

ML_PT_Pyworkflow.P_T_predictors(output, model)

Upload model for the entire pipeline: scaler, main model and bias function.

Args:

output (string): ‘Pressure’ or ‘Temperature’. model (string): ‘cpx_only’ or ‘cpx_liquid’.

Returns:

scaler (.joblib): scaler predictor (.onnx): main model bias_json (.json): parameters for the bias model.

class ML_PT_Pyworkflow.Parameters

Bases: object

# Relative errors (percentage. 3% for measurm. >1wt% and 8% for measurm. <1wt.%) ‘SiO2’ : 0.03, ‘TiO2’ : 0.08, ‘Al2O3’ : 0.03, ‘FeO’ : 0.03, ‘MgO’ : 0.03, ‘MnO’ : 0.08, ‘CaO’ : 0.03, ‘Na2O’ : 0.08, ‘Cr2O3’ : 0.08, ‘k2O_Liq’: 0.08

K_rel_err = array([0.08])

oxide_rel_err = array([0.03, 0.08, 0.03, 0.03, 0.03, 0.08, 0.03, 0.08, 0.08])

pressure_bins = [0, 5, 10, 15, 20, 25, 30]

temperature_bins = [600, 1050, 1140, 1250, 1800]

ML_PT_Pyworkflow.balanced_train_test(X, y, bins, test_size, sample_names)

Train-test split to manage unbalanced output and dependent samples.

Args:

X (numpy.array 2D): Input dataset. y (np.array 1D): Output dataset. bins (np.array 1D): index of dependent samples test_size (int) sample_names (list 1D)

Returns:

X_train (numpy.array 2D): Input dataset for training. X_test (np.array 2D): Input dataset for test. y_train (numpy.array 1D): Output dataset for training. y_test (np.array 1D): Output dataset for test. train_index (np.array 1D): Index for training. test_index (np.array 1D): Index for test.

ML_PT_Pyworkflow.bias_f_line(x, a)

half-line function

ML_PT_Pyworkflow.bias_f_temp(x, ang_left, popt_left, ang_right, popt_right)

Piece-wise function

ML_PT_Pyworkflow.data_imputation(dataFrame)

Fill Nan values with 0.

Args:

dataFrame (pandas.DataFrame): dataset to modify.

Returns:

pandas.DataFrame

ML_PT_Pyworkflow.input_perturbation(X, std_dev_perc, n_perturbations=15)

Data augmentation picking new elements from a Gaussian distribution with mean ‘value of each original sample’ and standard deviation ‘std_dev_perc’. Useful for the prediction phase because we are not managing the output.

Args:

X (numpy.array 2D): Input dataset. std_dev_perc (np.array 1D, or np.array 2D): standard deviation percentage. If you upload a 1D-array, all new samples are based on the same standard deviation, samples comes from different standard deviations. n_perturbations (int): numbers of elements to create from each row of X.

Returns:

X_perturb (np.array 2D): Input dataset augmented groups (np.array 1D): index of samples with the same origin

ML_PT_Pyworkflow.max_perc(x)

84-th percentile.

Args:

x (np.array): Array to define 84-th percentile.

Returns:

np.array (1D)

ML_PT_Pyworkflow.min_perc(x)

16-th percentile.

Args:

x (np.array): Array to define 16-th percentile.

Returns:

np.array (1D)

ML_PT_Pyworkflow.perturbation(X, y, std_dev_perc, n_perturbations=15)

Data augmentation picking new elements from a Gaussian distribution with mean ‘value of each original sample’ and standard deviation ‘std_dev_perc’. Useful for the training and testing phases because we are managing the output.

Args:

X (numpy.array 2D): Input dataset. y (np.array 1D): Output dataset. std_dev_perc (np.array 1D, or np.array 2D): standard deviation percentage. If you upload a 1D-array, all new samples are based on the same standard deviation, if you upload a 2D-array, new samples come from different standard deviations. n_perturbations (int): numbers of elements to create from each row of X.

Returns:

X_perturb (np.array 2D): Input dataset augmented y_rep (np.array 1D): Output dataset augmented groups (np.array 1D): index of samples with the same origin

ML_PT_Pyworkflow.piecewise_line(bias_split, X_train, y_train, model)

Piece-wise function fit on training dataset. It can manage model bias.

ML_PT_Pyworkflow.pwlt_transformation(X)

Pairwise Log-ratio transformation.

Args:

X (np.array 2D): Dataset to transform.

Returns:

np.array (2D)

ML_PT_Pyworkflow.replace_zeros(dataFrame, column_name)

Replace 0 values in a series with a random value from 0 to min of a series.

Args:

dataFrame (pandas.DataFrame): dataset to modify. column_name (string): column name of the dataset to modify.

Returns:

pandas.DataFrame