woodwork.table_accessor.WoodworkTableAccessor.pearson_correlation#

WoodworkTableAccessor.pearson_correlation(nrows=None, include_index=False, callback=None, extra_stats=False, min_shared=25, random_seed=0)[source]#

Calculates Pearson correlation coefficient between all pairs of columns in the DataFrame that support correlation. Works with numeric and datetime data. Call woodwork.utils.get_valid_pearson_types to see which Logical Types are supported.

Parameters:
  • nrows (int) – The number of rows to sample for when determining correlation. If specified, samples the desired number of rows from the data. Defaults to using all rows.

  • include_index (bool) – If True, the column specified as the index will be included as long as its LogicalType is valid for correlation calculations. If False, the index column will not have the Pearson correlation calculated for it. Defaults to False.

  • callback (callable, optional) – Function to be called with incremental updates. Has the following parameters: - update (int): change in progress since last call - progress (int): the progress so far in the calculations - total (int): the total number of calculations to do - unit (str): unit of measurement for progress/total - time_elapsed (float): total time in seconds elapsed since start of call

  • extra_stats (bool) – If True, additional column “shared_rows” recording the number of shared non-null rows for a column pair will be included with the dataframe. Defaults to False.

  • min_shared (int) – The number of shared non-null rows needed to calculate. Less rows than this will be considered too sparse to measure accurately and will return a NaN value. Must be non-negative. Defaults to 25.

  • random_seed (int) – Seed for the random number generator. Defaults to 0.

Returns:

A DataFrame containing Pearson correlation coefficients with columns column_1, column_2, and pearson that is sorted in decending order by correlation value. Pearson values are between -1 and 1, with 0 meaning no correlation.

Return type:

pd.DataFrame