Utility Functions

IMPROVE general utility functions

The following functions are part of improvelib and should be used to standardize aspects of the workflow.

store_predictions_df

Save predictions with accompanying dataframe.

This allows to trace original data evaluated (e.g. drug and cell pairs) if corresponding dataframe is available(output from save_stage_ydf in preprocess), in which case the whole structure as well as the model predictions are stored. If the dataframe is not available, only ground truth and model predictions are stored.

Used in train and infer.

Args:

y_pred (np.array): Model predictions

y_col_name (str): Name of the column in the y_data predicted on (e.g. ‘auc’, ‘ic50’)

stage (str): Specify if evaluation is with respect to val or test set (‘val’, or ‘test’)

output_dir (str): Directory to write results

y_true (Optional: np.array): Ground truth

input_dir (Optional: str): Directory where df with ground truth with metadata is stored

round_decimals (Optional: int): Number of decimals in output (default is 4)

IMPROVE Drug Response Prediction Utility Functions

Drug Utility Functions (drug_utils)

class DrugsLoader

Class to load and manage drug data.

Args:

params (Dict): IMPROVE parameters.

sep (str): Character separator in the loaded files (e.g., “t” for TSV files).

verbose (bool): Whether to print detailed loading information.

Attributes:

smiles_fname (str): Filename for SMILES data.

mordred_fname (str): Filename for Mordred descriptors.

ecfp4_512bit_fname (str): Filename for ECFP4 512-bit data.

known_file_names (List[str]): List of known drug data filenames.

params (Dict): Configuration parameters for loading data.

sep (str): Separator used in data files.

inp (List): List of input drug data files.

drug_col_name (str): Column name for drug identifiers.

x_data_path (str): Path to the directory containing drug data files.

dfs (Dict[str, pd.DataFrame]): Dictionary to store loaded dataframes, keyed by filename.

verbose (bool): Flag to control verbosity of output.

inp_fnames (List[str]): List of input filenames extracted from inp.

Example:

from improve import drug_resp_pred as drp
params = {
    "x_data_drug_files": "[['drug_SMILES.tsv'], ['drug_mordred.tsv'], ['drug_ecfp4_nbits512.tsv']]",
    "drug_col_name": "DrugID",
    "x_data_path": "/path/to/drug/data"
}
drugs_loader = drp.DrugsLoader(params)
print(drugs_loader)
print(dir(drugs_loader))
smi = drugs_loader["drug_SMILES.tsv"]

Omics Utility Functions (omics_utils)

class OmicsLoader

Class aggregates methods to load omics data.

Args:

params (Dict): IMPROVE parameters.

sep (str): Character separator in the loaded files (e.g., “t” for TSV files).

verbose (bool): Flag for verbosity.

Attributes:

copy_number_fname (str): Filename for copy number data.

discretized_copy_number_fname (str): Filename for discretized copy number data.

dna_methylation_fname (str): Filename for DNA methylation data.

gene_expression_fname (str): Filename for gene expression data.

miRNA_expression_fname (str): Filename for miRNA expression data.

mutation_count_fname (str): Filename for mutation count data.

mutation_long_format_fname (str): Filename for mutation data in long format.

mutation_fname (str): Filename for mutation data in parquet format.

rppa_fname (str): Filename for RPPA data.

known_file_names (List[str]): List of known omics data filenames.

params (Dict): Configuration parameters for loading data.

sep (str): Separator used in data files.

inp (List): List of input omics data files.

x_data_path (str): Path to the directory containing omics data files.

canc_col_name (str): Column name for indexing in the data files.

dfs (Dict[str, pd.DataFrame]): Dictionary to store loaded dataframes, keyed by filename.

verbose (bool): Flag to control verbosity of output.

inp_fnames (List[str]): List of input filenames extracted from inp.

Example:

from improve import drug_resp_pred as drp
params = {
    "x_data_canc_files": "[['cancer_gene_expression.tsv', 'Gene_Symbol'], "
                        "['cancer_copy_number.tsv', 'Entrez']]",
    "canc_col_name": "SampleID",
    "x_data_path": "/path/to/omics/data"
}
omics_loader = drp.OmicsLoader(params)
print(omics_loader)
print(dir(omics_loader))
gene_expression_data = omics_loader["cancer_gene_expression.tsv"]

Response Utility Functions (drp_utils)

class DrugResponseLoader

Class for loading monotherapy drug response data.

Args:

params (Dict): IMPROVE parameters.

split_file (str): File name that contains the split ids (rows).

sep (str): Character separator in the loaded files (e.g., “t” for TSV files).

verbose (bool): Flag for verbosity. Default is True.

Attributes:

response_fname (str): Default response file name.

known_file_names (List[str]): List of known file names.

params (Dict): Parameters for loading data.

sep (str): Separator used in data files.

inp (List): Parsed input data files.

y_col_name (str): Column name for the target variable.

canc_col_name (str): Column name for cancer sample identifiers.

drug_col_name (str): Column name for drug identifiers.

y_data_path (str): Path to the directory containing y data files.

split_fpath (Path): Path to the file containing split identifiers.

dfs (Dict[str, pd.DataFrame]): Dictionary to store loaded dataframes.

verbose (bool): Verbosity flag.

Example:

from improve import drug_resp_pred as drp
drp_loader = drp.DrugResponseLoader(params)
print(drp_loader)
print(dir(drp_loader))
rsp = drp_loader["response.tsv"]