`a`
Big Data and Information Analytics (BDIA)
 

Towards big data processing in clouds: An online cost-minimization approach

Pages: 15 - 29, Volume 1, Issue 1, January 2016      doi:10.3934/bdia.2016.1.15

 
       Abstract        References        Full Text (547.6K)       Related Articles       

Weidong Bao - College of Information System and Management, National University of Defense Technology, Changsha 410073, Hunan, China (email)
Wenhua Xiao - College of Information System and Management, National University of Defense Technology, Changsha, Hunan, 410073, China (email)
Haoran Ji - College of Information System and Management, National University of Defense Technology, Changsha, Hunan, 410073, China (email)
Chao Chen - College of Information System and Management, National University of Defense Technology, Changsha 410073, Hunan, China (email)
Xiaomin Zhu - College of Information System and Management, National University of Defense Technology, Changsha, Hunan, 410073, China (email)
Jianhong Wu - Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada M3J 1P3, Canada (email)

Abstract: Due to its elastic and on-demand nature of resource provisioning, cloud computing provides a cost effective and powerful technology for the processing of big data. Under this paradigm, Data Service Provider (DSP) may rent geographically distributed datacenters to process their large amount of data. As the data are dynamically generated and the resource pricing varies over time, moving the data from differently geographic locations to different datacenters while provisioning adequate computation resource to process them is an essential task to achieve cost effectiveness for DSP. In this paper, a joint online approach is proposed to address this task. We formulate the problem into a joint stochastic optimization problem, which is then decoupled into two independent subproblems via the Lyapunov framework. Our method is able to minimize the long-term time average cost including computing cost, storage cost, bandwidth cost and latency cost. Theoretical analysis shows that our online algorithm can produce a solution within an upper bound to the optimal solution achieved through offline computing and guarantee that the data processing can be completed with preset delays.

Keywords:  Big data, cloud computing, resource scheduling, data allocation, Lyapunov optimization.
Mathematics Subject Classification:  Primary: 90B15, 90B22; Secondary: 68W15, 68W27.

Received: July 2015;      Revised: August 2015;      Available Online: September 2015.

 References