University of SaskatchewanHARVEST
  • Login
  • Submit Your Research
  • About
    • About HARVEST
    • Guidelines
    • Browse
      • All of HARVEST
      • Communities & Collections
      • By Issue Date
      • Authors
      • Titles
      • Subjects
      • This Collection
      • By Issue Date
      • Authors
      • Titles
      • Subjects
    • My Account
      • Login
      JavaScript is disabled for your browser. Some features of this site may not work without it.
      View Item 
      • HARVEST
      • College of Graduate and Postdoctoral Studies
      • Electronic Theses and Dissertations
      • View Item
      • HARVEST
      • College of Graduate and Postdoctoral Studies
      • Electronic Theses and Dissertations
      • View Item

      Data De-Duplication in NoSQL Databases

      Thumbnail
      View/Open
      BRAD-THESIS.pdf (1.501Mb)
      nid690-DataDeDuplicationInNoSQLDatabasesRevised.pdf (1.501Mb)
      Date
      2012-05-22
      Author
      Brad, Nicoleta
      Type
      Thesis
      Degree Level
      Masters
      Metadata
      Show full item record
      Abstract
      With the popularity and expansion of Cloud Computing, NoSQL databases (DBs) are becoming the preferred choice of storing data in the Cloud. Because they are highly de-normalized, these DBs tend to store significant amounts of redundant data. Data de-duplication (DD) has an important role in reducing storage consumption to make it affordable to manage in today’s explosive data growth. Numerous DD methodologies like chunking and, delta encoding are available today to optimize the use of storage. These technologies approach DD at file and/or sub-file level but this approach has never been optimal for NoSQL DBs. This research proposes data De-Duplication in NoSQL Databases (DDNSDB) which makes use of a DD approach at a higher level of abstraction, namely at the DB level. It makes use of the structural information about the data (metadata) exploiting its granularity to identify and remove duplicates. The main goals of this research are: to maximally reduce the amount of duplicates in one type of NoSQL DBs, namely the key-value store, to maximally increase the process performance such that the backup window is marginally affected, and to design with horizontal scaling in mind such that it would run on a Cloud Platform competitively. Additionally, this research presents an analysis of the various types of NoSQL DBs (such as key-value, tabular/columnar, and document DBs) to understand their data model required for the design and implementation of DDNSDB. Primary experiments have demonstrated that DDNSDB can further reduce the NoSQL DB storage space compared with current archiving methods (from 17% to near 69% as more structural information is available). Also, by following an optimized adapted MapReduce architecture, DDNSDB proves to have competitive performance advantage in a horizontal scaling cloud environment compared with a vertical scaling environment (from 28.8 milliseconds to 34.9 milliseconds as the number of parallel Virtual Machines grows).
      Degree
      Master of Science (M.Sc.)
      Department
      Computer Science
      Program
      Computer Science
      Supervisor
      Deters, Ralph
      Committee
      Cooke, John; Vassileva, Julita; Dinh, Anh
      Copyright Date
      March 2012
      URI
      http://hdl.handle.net/10388/ETD-2012-03-387
      Subject
      duplicates
      hash table
      NoSQL
      Cloud Computing
      Collections
      • Electronic Theses and Dissertations
      University of Saskatchewan

      University Library

      © University of Saskatchewan
      Contact Us | Disclaimer | Privacy