diff --git a/Solutions to the questions concerning R language.md b/Solutions to the questions concerning R language.md index 493a947..e09b9f4 100644 --- a/Solutions to the questions concerning R language.md +++ b/Solutions to the questions concerning R language.md @@ -217,4 +217,208 @@ table(OJ.test$Purchase, test.pred) ## [1] 0.1556 ``` The radial basis kernel with default gamma creates 367 support vectors, out of which, 184 belong to level `𝙲𝙷` and remaining 183 belong to level `𝙼𝙼`. The classifier has a training error of 14.7% and a test error of 15.6% which is a slight improvement over linear kernel. We now use cross validation to find optimal gamma. - +``` +set.seed(755) +tune.out = tune(svm, Purchase ~ ., data = OJ.train, kernel = "radial", ranges = list(cost = 10^seq(-2, + 1, by = 0.25))) +summary(tune.out) +``` +``` +## +## Parameter tuning of 'svm': +## +## - sampling method: 10-fold cross validation +## +## - best parameters: +## cost +## 0.5623 +## +## - best performance: 0.165 +## +## - Detailed performance results: +## cost error dispersion +## 1 0.01000 0.3850 0.06258 +## 2 0.01778 0.3850 0.06258 +## 3 0.03162 0.3762 0.06908 +## 4 0.05623 0.2100 0.03855 +## 5 0.10000 0.1862 0.03143 +## 6 0.17783 0.1837 0.03230 +## 7 0.31623 0.1712 0.03438 +## 8 0.56234 0.1650 0.03764 +## 9 1.00000 0.1750 0.03584 +## 10 1.77828 0.1738 0.04059 +## 11 3.16228 0.1763 0.03748 +## 12 5.62341 0.1763 0.03839 +## 13 10.00000 0.1738 0.03459 +``` +``` +svm.radial = svm(Purchase ~ ., data = OJ.train, kernel = "radial", cost = tune.out$best.parameters$cost) +train.pred = predict(svm.radial, OJ.train) +table(OJ.train$Purchase, train.pred) +``` +``` +## train.pred +## CH MM +## CH 452 40 +## MM 77 231 +``` +``` +(77 + 40)/(452 + 40 + 77 + 231) +``` +``` +## [1] 0.1462 +``` +``` +test.pred = predict(svm.radial, OJ.test) +table(OJ.test$Purchase, test.pred) +``` +``` +## test.pred +## CH MM +## CH 146 15 +## MM 28 81 +``` +``` +(28 + 15)/(146 + 15 + 28 + 81) +``` +``` +## [1] 0.1593 +``` +Tuning slightly decreases training error to 14.6% and slightly increases test error to 16% which is still better than linear kernel. + +**vii.** +``` +set.seed(8112) +svm.poly = svm(Purchase ~ ., data = OJ.train, kernel = "poly", degree = 2) +summary(svm.poly) +``` +``` +## +## Call: +## svm(formula = Purchase ~ ., data = OJ.train, kernel = "poly", +## degree = 2) +## +## +## Parameters: +## SVM-Type: C-classification +## SVM-Kernel: polynomial +## cost: 1 +## degree: 2 +## gamma: 0.05556 +## coef.0: 0 +## +## Number of Support Vectors: 452 +## +## ( 232 220 ) +## +## +## Number of Classes: 2 +## +## Levels: +## CH MM +``` +``` +train.pred = predict(svm.poly, OJ.train) +table(OJ.train$Purchase, train.pred) +``` +``` +## train.pred +## CH MM +## CH 460 32 +## MM 105 203 +``` +``` +(32 + 105)/(460 + 32 + 105 + 203) +``` +``` +## [1] 0.1713 +``` +``` +test.pred = predict(svm.poly, OJ.test) +table(OJ.test$Purchase, test.pred) +``` +``` +## test.pred +## CH MM +## CH 149 12 +## MM 37 72 +``` +``` +(12 + 37)/(149 + 12 + 37 + 72) +``` +``` +## [1] 0.1815 +``` +Summary shows that polynomial kernel produces 452 support vectors, out of which, 232 belong to level 𝙲𝙷 + and remaining 220 belong to level 𝙼𝙼 +. This kernel produces a train error of 17.1 +% and a test error of 18.1 +% which are slightly higher than the errors produces by radial kernel but lower than the errors produced by linear kernel. + +``` +set.seed(322) +tune.out = tune(svm, Purchase ~ ., data = OJ.train, kernel = "poly", degree = 2, ranges = list(cost = 10^seq(-2, 1, by = 0.25))) +summary(tune.out) +``` +``` +## +## Parameter tuning of 'svm': +## +## - sampling method: 10-fold cross validation +## +## - best parameters: +## cost +## 5.623 +## +## - best performance: 0.1837 +## +## - Detailed performance results: +## cost error dispersion +## 1 0.01000 0.3850 0.05426 +## 2 0.01778 0.3675 0.05076 +## 3 0.03162 0.3575 0.05177 +## 4 0.05623 0.3425 0.04937 +## 5 0.10000 0.3150 0.05231 +## 6 0.17783 0.2487 0.03929 +## 7 0.31623 0.2088 0.05684 +## 8 0.56234 0.2088 0.05653 +## 9 1.00000 0.2000 0.06095 +## 10 1.77828 0.1938 0.04497 +## 11 3.16228 0.1862 0.04185 +## 12 5.62341 0.1837 0.03336 +## 13 10.00000 0.1837 0.04042 +``` +``` +svm.poly = svm(Purchase ~ ., data = OJ.train, kernel = "poly", degree = 2, cost = tune.out$best.parameters$cost) +train.pred = predict(svm.poly, OJ.train) +table(OJ.train$Purchase, train.pred) +``` +``` +## train.pred +## CH MM +## CH 455 37 +## MM 84 224 +``` +``` +(37 + 84)/(455 + 37 + 84 + 224) +``` +``` +## [1] 0.1512 +``` +``` +test.pred = predict(svm.poly, OJ.test) +table(OJ.test$Purchase, test.pred) +``` +``` +## test.pred +## CH MM +## CH 148 13 +## MM 34 75 +``` +``` +(13 + 34)/(148 + 13 + 34 + 75) +``` +``` +## [1] 0.1741 +``` +