mis6356Dimension Reduction(1)

Dimension Reduction James Zhang MIS 6356 BA with R

Exploring the data Statistical summary of data: common metrics  Average  Median  Minimum  Maximum  Standard deviation  Counts & percentages

Combining Categories Many zoning categories are the same or similar with respect to CATMEDV

Principal Components Analysis Goal: Reduce a set of numerical variables. The idea: Remove the overlap of information between these variable. [“Information” is measured by the sum of the variances of the variables.] Final product: A smaller number of numerical variables that contain most of the information

Example – Breakfast Cereals (excerpt) name mfr type calories protein … rating 100%_Bran N C 70 4 … 68 100%_Natural_Bran Q C 120 3 … 34 All-Bran K C 70 4 … 59 All-Bran_with_Extra_Fiber K C 50 4 … 94 Almond_Delight R C 110 2 … 34 Apple_Cinnamon_Cheerios G C 110 2 … 30 Apple_Jacks K C 110 2 … 33 Basic_4 G C 130 3 … 37 Bran_Chex R C 90 2 … 49 Bran_Flakes P C 90 3 … 53 Cap'n'Crunch Q C 120 1 … 18 Cheerios G C 110 6 … 51 Cinnamon_Toast_Crunch G C 120 1 … 20

Description of Variables Name: name of cereal mfr: manufacturer type: cold or hot calories: calories per serving protein: grams fat: grams sodium: mg. fiber: grams carbo: grams complex carbohydrates sugars: grams potass: mg. vitamins: % FDA rec shelf: display shelf weight: oz. 1 serving cups: in one serving rating: consumer reports

Using linear combinations to redistribute the variability in a more polarized way Z 1 and Z 2 are two new variables  Linear combinations of Rating and Calories  Z 1 has the highest variation (spread of values)  Z 2 has the lowest variation

PCA output for these 2 variables Weights to project original data onto Z 1 & Z 2, e.g. (-0.847, 0.532) are weights for Z 1 pcs <- prcomp(data.frame(cereals.df$calories, cereals.df$rating)) summary(pcs) PC1 PC2 cereals.df.calories 0.8470535 0.5315077 cereals.df.rating -0.5315077 0.8470535 Importance of components: PC1 PC2 Standard deviation 22.3165 8.8844 Proportion of Variance 0.8632 0.1368 Cumulative Proportion 0.8632 1.0000 86% of the total variance is accounted for by component 1

> pcs <- prcomp(na.omit(cereals.df[,-c(1:3)])) > summary(pcs) PCA for the 13 Numerical Variables in the Cereals Data The first two components account for 93% of the total variance, so using 2-3 components in further modeling would probably be sufficient

The Weightings for the First Five Components

Normalizing data  In these results, sodium dominates first PC  Just because of the way it is measured (mg), its scale is greater than almost all other variables  Hence its variance will be a dominant component of the total variance  Normalize each variable to remove scale effect Divide by std. deviation (may subtract mean first)  Normalization (= standardization) is usually performed in PCA; otherwise measurement units affect results > pcs.cor <- prcomp(na.omit(cereals.df[,-c(1:3)]), scale. = T) Normalize the variables

PCA Output Using all 13 Normalized Numerical Variables

PCA in Classification/Prediction  Apply PCA to training data  Decide how many PC’s to use  Use variable weights in those PC’s with validation/new data  This creates a new reduced set of predictors in validation/new data

Summary  Data reduction is useful for compressing the information in the data into a smaller subset  Categorical variables can be reduced by combining similar categories  Principal components analysis transforms an original set of numerical data into a smaller set of weighted averages of the original data that contain most of the original information in less variables.