Open Access
Subscription Access
Open Access
Subscription Access
Phase Level Scheduler for MapReduce using Grained Resource
Subscribe/Renew Journal
MapReduce is one of the important concepts of Hadoop that is used for data handling used by big companies today such as Google and Facebook. Here we divide each job into the map and reduce phases and try to complete the execution of the assigned task in a parallel form. In this paper, we suggest that it would be more efficient if we make the scheduler to work at the phase-level instead of the task-level. The reason is that the task demands a lot of requirements during its lifetime. For this very purpose, we introduce the concept called PRISM, which is a phase and information-aware scheduler for MapReduce and in this concept, we divide the tasks into unequal parts called as phases and apply phase-level scheduling to these phases and achieve efficient resource usage.
Keywords
Job Progress Monitor, MapReduce, Phase-Based Scheduler.
Subscription
Login to verify subscription
User
Font Size
Information
- Hadoop Map Reduce Distribution, 2015. [Online]. Available: http://hadoop.apache.org
- Hadoop Fair Scheduler, 2015. [Online]. Available: http://hadoopapache.org/docs/r0.20.2/fair_scheduler.html
- Hadoop Distributed File System, 2015. [Online]. Available: hadoop.apache.org/docs/hdfs/current/
- The Next Generation of Apache Hadoop MapReduce, 2015. [Online]. Available: http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
- R. Boutaba, L. Cheng, and Q. Zhang, “On cloud computational models and the heterogeneity challenge,” J. Internet Serv. Appl., vol. 3, no. 1, pp. 1-10, 2012.
- T. Condie, N. Conway, P. Alvaro, J. Hellerstein, K. Elmeleegy, and R. Sears, “MapReduce online,” In Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2010, p. 21.
- J. Dean, and S. Ghemawat, “MapReduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008.
- A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica, “Dominant resource fairness: Fair allocation of multiple resource types,” In Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2011, pp. 323-336.
- H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. Cetin, and S. Babu, “Starfish: A self-tuning system for big data analytics,” In Proc. Conf. Innovative Data Syst. Res., 2011, pp. 261-272.
- M. Isard, V. Prabhakaran, J. Currey, U. Wieder, and K. Talwar, “Quincy: Fair scheduling for distributed computing clusters,” In Proc. ACMSIGOPS Symp. Oper. Syst. Principles, 2009, pp. 261-276.
- C. Joe-Wong, S. Sen, T. Lan, and M. Chiang, “Multi-resource allocation: Flexible tradeoffs in a unifying framework,” In Proc. IEEE Int. Conf. Comput. Commun., 2012, pp. 1206-1214.
- J. Polo, C. Castillo, D. Carrera, Y. Becerra, I. Whalley, M. Steinder, J. Torres, and E. Ayguade, “Resource-aware adaptive scheduling for MapReduce clusters,” In Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 187-207.
- A. Rasmussen, M. Conley, R. Kapoor, V. T. Lam, G. Porter, and A. Vahdat, “ThemisMR: An I/O-Efficient MapReduce,” In Proc. ACM Symp. Cloud Compute, 2012, p. 13.
- A. Verma, L. Cherkasova, and R. Campbell, “Resource provisioning framework for MapReduce jobs with performance goals,” In Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 165-186.
- D. Xie, N. Ding, Y. Hu, and R. Kompella, “The only constant is change: Incorporating time-varying network reservations in data centers,” In Proc. ACM SIGCOMM, 2012, pp. 199-210.
Abstract Views: 375
PDF Views: 0