Kernel Local Fisher Discriminant Analysis (KLFDA). This function implements the Kernel Local Fisher Discriminant Analysis with an unified Kernel function. Different from KLFDA function, which adopts the Multinomial Kernel as an example, this function empolys the kernel function that allows you to choose various types of kernels. See the kernel function from "kernelMatriax" (kernlab).

klfda_1(x, y, kernel = polydot(degree = 1, scale = 1, offset = 1), r = 20, tol, prior, CV = FALSE, usekernel = TRUE, fL = 0.5, metric = c("weighted", "orthonormalized", "plain"), knn = 6, reg = 0.001, ...)

Arguments

x

The input training data

y

The training labels

kernel

The kernel function used to calculate kernel matrix. Choose the corresponding kernel you want, see details.

r

The number of reduced features you want to keep.

tol

The tolerance used to reject the uni-variance. This is important when the variance between classes is small, and setting the large tolerance will avoid the data distortion.

prior

The weight of each class, or the proportion of each class.

CV

Whether to do cross validation.

usekernel

whether to use kernel classifier, if TRUE, pass to Naive Bayes classifier.

fL

If usekernel is TRUE, pass to the kernel function.

metric

type of metric in the embedding space (default: 'weighted') 'weighted' - weighted eigenvectors 'orthonormalized' - orthonormalized 'plain' - raw eigenvectors

knn

The number of nearest neighbours

reg

The regularization parameter

Details

This function empolys three different classifiers, the basic linear classifier, the Mabayes (Bayes rule and the Mahalanobis distance), and Niave Bayes classifier. The argeument "kernel" in the klfda function is the kernel function used to calculate the kernel matrix. If usekernel is TRUE, the corresponding kernel parameters will pass the the Naive Bayes kernel classifier. The kernel parameter can be set to any function, of class kernel, which computes the inner product in feature space between two vector arguments. kernlab provides the most popular kernel functions which can be initialized by using the following functions:

rbfdot Radial Basis kernel function

polydot Polynomial kernel function

vanilladot Linear kernel function

tanhdot Hyperbolic tangent kernel function

laplacedot Laplacian kernel function

besseldot Bessel kernel function

anovadot ANOVA RBF kernel function

splinedot the Spline kernel

(see example.)

kernelFast is mainly used in situations where columns of the kernel matrix are computed per invocation. In these cases, evaluating the norm of each row-entry over and over again would cause significant computational overhead.

Value

The results give the classified classes and the posterior possibility of each class using different classifier.

class

The class labels from linear classifier

posterior

The posterior possibility of each class from linear classifier

bayes_judgement

Discrimintion results using the Mabayes classifier

bayes_assigment

Discrimintion results using the Naive bayes classifier

Z

The reduced features

%% ...

References

Sugiyama, M (2007).Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, vol.8, 1027-1061.

Sugiyama, M (2006). Local Fisher discriminant analysis for supervised dimensionality reduction. In W. W. Cohen and A. Moore (Eds.), Proceedings of 23rd International Conference on Machine Learning (ICML2006), 905-912.

Original Matlab Implementation: http://www.ms.k.u-tokyo.ac.jp/software.html#LFDA

Tang, Y., & Li, W. (2019). lfda: Local Fisher Discriminant Analysis inR. Journal of Open Source Software, 4(39), 1572.

Moore, A. W. (2004). Naive Bayes Classifiers. In School of Computer Science. Carnegie Mellon University.

Pierre Enel (2020). Kernel Fisher Discriminant Analysis (https://www.github.com/p-enel/MatlabKFDA), GitHub. Retrieved March 30, 2020.

Karatzoglou, A., Smola, A., Hornik, K., & Zeileis, A. (2004). kernlab-an S4 package for kernel methods in R. Journal of statistical software, 11(9), 1-20.

Author

qinxinghu@gmail.com

See also

predict.klfda_1, KLFDA

Examples

require(kernlab) btest=klfda_1(as.matrix(iris[,1:4]),as.matrix(as.data.frame(iris[,5])),kernel=kernlab::rbfdot(sigma = 0.1),r=3,prior=NULL,tol=1e-90,reg=0.01,metric = 'plain')
#> Loading required package: WMDB
pred=predict.klfda_1(btest,as.matrix(iris[1:10,1:4]))