Feature Selection Bias in Assessing the Predictivity of SNPs for Alzheimer's Disease
| dc.contributor.advisor | Li, Longhai | |
| dc.contributor.advisor | Balbuena, Lloyd | |
| dc.contributor.committeeMember | Feng, Cindy | |
| dc.contributor.committeeMember | Wu, Fangxiang | |
| dc.contributor.committeeMember | Wang, JC | |
| dc.creator | Dong, Mei | |
| dc.date.accessioned | 2019-06-07T20:07:17Z | |
| dc.date.available | 2019-06-07T20:07:17Z | |
| dc.date.created | 2019-05 | |
| dc.date.issued | 2019-06-07 | |
| dc.date.submitted | May 2019 | |
| dc.date.updated | 2019-06-07T20:07:18Z | |
| dc.description.abstract | In the context of identifying related SNPs for a phenotype of interest (e.g., a disease status), we consider the problem of assessing the predictivity of SNPs that are selected by performing genome-wide association studies. Internal cross-validation (ICV) is a wrong but often used method for this assessment. With ICV, a subset of SNPs are pre-selected based on all samples then cross-validation (CV) is applied to assess the predictivity of the pre-selected SNPs. The predictivity estimate of the selected SNPs given by ICV is upwardly biased. This is often called the feature selection bias problem. The cause of this bias is that the feature selection procedure, which is a part of training procedure, is not external to the test samples in ICV. A correct method, called external cross-validation (ECV), is to re-select features based on only the training samples in each fold of CV such that the feature selection is external to test samples. The feature selection bias of ICV has been discussed by a few articles in the context of cancer diagnosis with microarray data. However, this problem has not received sufficient attention in the literature, especially in the context of predicting with SNP data. Many articles in the literature use ICV or do not state explicitly that their feature selection is external to test samples. In this thesis, we use an example of predicting late-onset Alzheimer's disease (LOAD) from selected SNPs to demonstrate that ICV could lead to severe false discovery. We use a real SNP dataset related to LOAD and two synthetic datasets (simulated response with real SNPs) for this demonstration. For the prediction, we compare the performances of three regularized logistic regression methods: LASSO, elastic-net, and a fully Bayesian hyper-LASSO method. For the LOAD dataset, we see that, except for APOE, no other SNPs can improve the prediction of LOAD using ECV method; however, the predictivity estimate of selected SNPs given by ICV can reach an $R^{2}$ as high as 80\%. For the synthetic datasets, we obtain the similar results as in the real dataset; additionally we see that the predictivity estimate of selected SNPs obtained with ICV can be even higher than the oracle predictivity of the truly related SNPs used to generate the response. In this study, we also find that the hyper-LASSO method can achieve better predictive performance than the LASSO and elastic-net. We recommend that ICV should not be used to measure the predictivity of selected SNPs and this statement should be made clear in research articles. | |
| dc.format.mimetype | application/pdf | |
| dc.identifier.uri | http://hdl.handle.net/10388/12118 | |
| dc.subject | Feature Selection bias | |
| dc.subject | Cross-validation | |
| dc.subject | Predictive Analysis | |
| dc.subject | GWAS | |
| dc.subject | Alzheimer's Disease | |
| dc.title | Feature Selection Bias in Assessing the Predictivity of SNPs for Alzheimer's Disease | |
| dc.type | Thesis | |
| dc.type.material | text | |
| thesis.degree.department | Mathematics and Statistics | |
| thesis.degree.discipline | Mathematics | |
| thesis.degree.grantor | University of Saskatchewan | |
| thesis.degree.level | Masters | |
| thesis.degree.name | Master of Science (M.Sc.) |