Kernel Operations

This module implements the common kernel operations such as

• normalization of a kernel matrix (KM),

• centering (one- and two-sample cases),

• evaluating similarity, computing alignment,

• frobenius norms,

• linear combinations and

• checking whether a KM is PSD.

API

kernelmethods.operations.alignment_centered(km_one, km_two, value_if_zero_division='raise', centered_already=False)

Computes the centered alignment between two kernel matrices

(Alignment is computed on centered kernel matrices)

Implements Definition 4 (Kernel matrix alignment) from Section 2.3 in Cortes, Corinna, Mehryar Mohri, and Afshin Rostamizadeh, 2012, “Algorithms for Learning Kernels Based on Centered Alignment”, Journal of Machine Learning Research 13(Mar): 795–828.

Parameters

• km_two (km_one,) –

• value_if_zero_division (str or float) –

  determines the value of alignment, in case the norm of one of the two kernel matrices is close to zero and we are unable to compute it.

  Default is ‘raise’, requesting to raise an exception.

  One could also choose 0.0, which assigns lowest alignment, effectively discarding it for ranking purposes.

• centered_already (bool) – Flag to indicate whether the input kernel matrices are centered already or not. If False, input KMs will be centered.

Returns

centered_alignment – Value of centered_alignment between the two kernel matrices

Return type

float

kernelmethods.operations.center_km(KM)

Centers a given kernel matrix.

Implements the definition according to Lemma 1 in Section 2.2 in Cortes, Corinna, Mehryar Mohri, and Afshin Rostamizadeh, 2012, “Algorithms for Learning Kernels Based on Centered Alignment”, Journal of Machine Learning Research 13(Mar): 795–828.

Parameters

KM (ndarray) – Symmetric matrix to be centered.

Returns

centered_km – Centered kernel matrix

Return type

ndarray

kernelmethods.operations.eval_similarity(km_one, km_two)

Evaluate similarity between two kernel matrices

kernelmethods.operations.frobenius_norm(A)

Computes the Frobenius norm of a matrix A, which is the square root of the Frobenius product with itself.

Parameters

A (ndarray) – Matrix to compute the norm of

Returns

norm – Frobenious norm

Return type

float

kernelmethods.operations.frobenius_product(A, B)

Computes the Frobenious product between two matrices of equal dimensions.

<A, B>_F is equal to the sum of element-wise products between A and B.

\[<\mathbf{A}, \mathbf{B}>_F = \sum_{i, j} \mathbf{A}_{ij} \mathbf{B}_{ij}\]

Parameters

B (A,) – Two matrices of equal dimensions to compute the product.

Returns

product – Frobenious product

Return type

float

kernelmethods.operations.is_PSD(sym_matrix, tolerance=1e-06, verbose=False)

Tests whether a given matrix is positive-semidefinite (PSD).

A symmetric matrix is PSD if ALL its eigen values >= 0 (non-negative). If any of its eigen values are negative, it is not PSD.

This functions accounts for numerical instabilities with a tolerance parameter.

This function can also be called with a shorthand is_PSD()

Parameters

• sym_matrix (ndarray) – Matrix to be evaluted for PSDness

• tolerance (float) – Tolerance parameter to account for numerical instabilities in the eigen value computations (which can result in negative eigen values very slightly below 0)

• verbose (bool) – Flag to indicate whether to print traceback in case of errors during the computation of the eigen values

Returns

psd – Flag indicating whether the matrix is PSD.

Return type

bool

kernelmethods.operations.is_positive_semidefinite(sym_matrix, tolerance=1e-06, verbose=False)

Tests whether a given matrix is positive-semidefinite (PSD).

A symmetric matrix is PSD if ALL its eigen values >= 0 (non-negative). If any of its eigen values are negative, it is not PSD.

This functions accounts for numerical instabilities with a tolerance parameter.

This function can also be called with a shorthand is_PSD()

Parameters

• sym_matrix (ndarray) – Matrix to be evaluted for PSDness

• tolerance (float) – Tolerance parameter to account for numerical instabilities in the eigen value computations (which can result in negative eigen values very slightly below 0)

• verbose (bool) – Flag to indicate whether to print traceback in case of errors during the computation of the eigen values

Returns

psd – Flag indicating whether the matrix is PSD.

Return type

bool

kernelmethods.operations.linear_combination(km_set, weights)

Weighted linear combinations of a set of given kernel matrices

Parameters

• km_set (KernelSet) – Collection of compatible kernel matrices

• weights (Iterable) – Set of weights for the kernel matrices in km_set

Returns

lin_comb_KM – Final result of weighted linear combination of the kernel matrix set

Return type

ndarray

kernelmethods.operations.normalize_km(KM, method='cosine')

Normalize a kernel matrix to have unit diagonal.

Cosine normalization normalizes the kernel matrix to have unit diagonal. Implements definition according to Section 5.1 in book (Page 113) Shawe-Taylor and Cristianini, “Kernels Methods for Pattern Analysis”, 2004

Matrix must be square (and coming from a single sample: K(X,X), not K(X,Y)

Parameters

• KM (ndarray) – Symmetric matrix to be normalized

• method (str) – Method of normalization. Options: cosine only.

Returns

normed_km – Normalized kernel matrix

Return type

ndarray

kernelmethods.operations.normalize_km_2sample(cross_K_XY, diag_K_XX, diag_K_YY, method='cosine')

Normalize a kernel matrix K(X,Y) to have unit diagonal.

Cosine normalization normalizes the kernel matrix to have unit diagonal. Implements definition _similar_ to Section 5.1 in book (Page 113) Shawe-Taylor and Cristianini, “Kernels Methods for Pattern Analysis”, 2004

Parameters

• cross_K_XY (ndarray, 2D) – Matrix of inner-products for samples from X onto Y i.e. K(X,Y)

• diag_K_XX (array) – Diagonal from matrix of inner-products for samples from X onto itself i.e. K(X,X) K(X,X) must NOT be normalized (otherwise they will all be 1s)

• diag_K_YY (array) – Diagonal from matrix of inner-products for samples from Y onto itself i.e. K(Y,Y)

Returns

normed_km – Normalized version of K(X,Y)

NOTE: K_XY may NOT have unit diagonal, as k(x,y) != sqrt(k(x,x))*sqrt(k(y,y))

Return type

ndarray