ponent Analysis (PCA).ppt

ponentAnalysis(PCA)J.-SRogerJang(張智星)jang@/jangMIRLab,CSIEDeptNationalTaiwanUniversityIntroductiontoPCAPCA(ponentAnalysis)Aneffectivemethodforreducingadataset’sdimensionalitywhilekeepingspatialcharacteristicsasmuchaspossibleCharacteristics:ForunlabeleddataAlineartransformwithsolidmathematicalfoundationApplicationsLine/planefittingFacerecognitionMachinelearning...Comparison:PCA&K-mongoal:ReductionofunlabeleddataPCA:dimensionalityreductionObjectivefunction:Variance↑K-meansclustering:datacountreductionObjectivefunction:Distortion↓Quiz!ExamplesofPCAProjectionsPCAprojections2D1D3D2DProblemDefinitionInputAdatasetXofnd-dimpointswhicharezerojustified:OutputAunityvectorusuchthatthesquaresumofthedataset’sprojectionontouismaximized.*Quiz!ProjectionAnglebetweenvectorsProjectionofxontou*Quiz!Extension:Whatistheprojectionofxontothesubspacespannedbyu1,u2,…,um?Eigenvalue&EigenvectorDefinitionofeigenvectorxandeigenvaluelofasquarematrixA:xisnon-zeroissingularQuiz!DemoofEigenvectorsandEigenvalueTry“eigshow”inMATLABtoplottrajectoriesofalineartransformin2DCleve’scommentsMathematicalFormulationDatasetrepresentation:Xisdbyn,withn>dProjectionofeachcolumnofXontou:Squaresum:Objectivefunctionwithaconstraintonu:*LagrangemultiplierLagrangeMultipliers|GeometricMeaning&:uistheeigenvectorwhilelistheeigenvalueWhenuistheeigenvector:Ifwearrangeeigenvaluessuchthat:MaxofJ(u)isl1,ursatu=u1MinofJ(u)isld,ursatu=ud*XXT:Covariancematrixtimesn