Journal of South China University of Technology(Natural Science Edition) ›› 2012, Vol. 40 ›› Issue (9): 42-47.

• Computer Science & Technology • Previous Articles     Next Articles

Hadoop Data Load Balancing Method Based on Dynamic Bandwidth Allocation

Lin Wei-wei1  Liu Bo2   

  1. 1.  School of Computer Science and Engineering,South China University of Technology,Guangzhou 510006,Guangdong,China; 2. School of Computer Science,South China Normal University,Guangzhou 510631,Guangdong,China
  • Received:2012-01-05 Revised:2012-07-17 Online:2012-09-25 Published:2012-08-01
  • Contact: 林伟伟(1980-) ，男，博士，副教授，主要从事分布式计算、云计算、移动互联网研究． E-mail:linww@scut.edu.cn
  • About author:林伟伟(1980-) ，男，博士，副教授，主要从事分布式计算、云计算、移动互联网研究．
  • Supported by:

    广东省自然科学基金资助项目( 10451064101005155，S2011010001754) ; 广东省科技计划项目( 2012B010100030) ;广东省战略性新兴产业核心技术攻关项目( 2011A010801002) ; 广州市海珠区科技计划项目( x2jsB2120750)

PDF

1924

Abstract:

Data load balancing greatly affects the performance of the Hadoop distributed file system ( HDFS). In order to overcome the inefficiency and inflexibility of the default data load balancing method in HDFS,this paper devises a novel dynamic load balancing method,which dynamically allocates network bandwidth to achieve the data load balancing by controlling variables. Then,the corresponding mathematical model is constructed based on the controlled variables. Experimental results show that the devised method can not only guarantee the performance of the HDFS data access system but also improve the data load balancing efficiency in the presence of a new cluster node.

Key words: Hadoop, load balancing, bandwidth