Repository logo
 

Data De-Duplication in NoSQL Databases

dc.contributor.advisorDeters, Ralphen_US
dc.contributor.committeeMemberCooke, Johnen_US
dc.contributor.committeeMemberVassileva, Julitaen_US
dc.contributor.committeeMemberDinh, Anhen_US
dc.creatorBrad, Nicoletaen_US
dc.date.accessioned2013-01-03T22:30:42Z
dc.date.available2013-01-03T22:30:42Z
dc.date.created2012-03en_US
dc.date.issued2012-05-22en_US
dc.date.submittedMarch 2012en_US
dc.description.abstractWith the popularity and expansion of Cloud Computing, NoSQL databases (DBs) are becoming the preferred choice of storing data in the Cloud. Because they are highly de-normalized, these DBs tend to store significant amounts of redundant data. Data de-duplication (DD) has an important role in reducing storage consumption to make it affordable to manage in today’s explosive data growth. Numerous DD methodologies like chunking and, delta encoding are available today to optimize the use of storage. These technologies approach DD at file and/or sub-file level but this approach has never been optimal for NoSQL DBs. This research proposes data De-Duplication in NoSQL Databases (DDNSDB) which makes use of a DD approach at a higher level of abstraction, namely at the DB level. It makes use of the structural information about the data (metadata) exploiting its granularity to identify and remove duplicates. The main goals of this research are: to maximally reduce the amount of duplicates in one type of NoSQL DBs, namely the key-value store, to maximally increase the process performance such that the backup window is marginally affected, and to design with horizontal scaling in mind such that it would run on a Cloud Platform competitively. Additionally, this research presents an analysis of the various types of NoSQL DBs (such as key-value, tabular/columnar, and document DBs) to understand their data model required for the design and implementation of DDNSDB. Primary experiments have demonstrated that DDNSDB can further reduce the NoSQL DB storage space compared with current archiving methods (from 17% to near 69% as more structural information is available). Also, by following an optimized adapted MapReduce architecture, DDNSDB proves to have competitive performance advantage in a horizontal scaling cloud environment compared with a vertical scaling environment (from 28.8 milliseconds to 34.9 milliseconds as the number of parallel Virtual Machines grows).en_US
dc.identifier.urihttp://hdl.handle.net/10388/ETD-2012-03-387en_US
dc.language.isoengen_US
dc.subjectduplicatesen_US
dc.subjecthash tableen_US
dc.subjectNoSQLen_US
dc.subjectCloud Computingen_US
dc.titleData De-Duplication in NoSQL Databasesen_US
dc.type.genreThesisen_US
dc.type.materialtexten_US
thesis.degree.departmentComputer Scienceen_US
thesis.degree.disciplineComputer Scienceen_US
thesis.degree.grantorUniversity of Saskatchewanen_US
thesis.degree.levelMastersen_US
thesis.degree.nameMaster of Science (M.Sc.)en_US

Files

Original bundle
Now showing 1 - 2 of 2
Loading...
Thumbnail Image
Name:
BRAD-THESIS.pdf
Size:
1.5 MB
Format:
Adobe Portable Document Format
Loading...
Thumbnail Image
Name:
nid690-DataDeDuplicationInNoSQLDatabasesRevised.pdf
Size:
1.5 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1006 B
Format:
Plain Text
Description: