We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
DMBVA - A COMPRESSION-BASED DISTRIBUTED DATA WAREHOUSE MANAGEMENT IN PARALLEL ENVIRONMENT.
- Authors
Siddiqui, Fazlul Hasan; Hoque, Abu Sayed Md. Latiful
- Abstract
Parallel and distributed data warehouse architectures have been evolved to support online queries on massive data in a short time. Unfortunately, the emergence of e-application has been creating extremely high volume of data that reaches to terabyte threshold. The conventional data warehouse management system is costlier in terms of storage space and processing speed and sometimes it is unable to handle such huge amount of data. As a result, there is a crucial need for the new algorithms and techniques to store and manipulate these data. In this paper, we have presented a compression-based distributed data warehouse architecture -- 'DMBVA' for storage of warehouse data, and support online queries efficiently. We have achieved a factor of 25-30 compression compared to SQL server data warehouse. The main computational component of data warehouse is the generation and querying on the data cube. Our algorithm -- 'PCVDC' generates data cube directly from the compressed form of data in parallel. The reduction in the size of data cube is a factor of 30-45 compared to existing methods. The response time has also been significantly improved. These improvements are achieved by eliminating the suffix and prefix redundancy, virtual nature of the data cube, direct addressability of compressed form of data and parallel computation. Experimental evaluation shows the improved performance over the existing systems.
- Subjects
DATA warehousing; DATA compression; DISTRIBUTED computing; PARALLEL processing; SQL; COMPUTER science
- Publication
Malaysian Journal of Computer Science, 2007, Vol 20, Issue 1, p63
- ISSN
0127-9084
- Publication type
Article
- DOI
10.22452/mjcs.vol20no1.6