This article is second from our series, optimizing the spark command, we usually use two types of spark commands, spark-submit and spark-shell, both of them take the same parameters and options, however the second is a REPL which is used to mainly do debugging. In this, we will see what parameters are important and how to set/calculate the values for better performance.
Optimizing Spark Job (spark-submit/shell)
Optimizing Spark Job (spark-submit/shell)
Optimizing Spark Job (spark-submit/shell)
This article is second from our series, optimizing the spark command, we usually use two types of spark commands, spark-submit and spark-shell, both of them take the same parameters and options, however the second is a REPL which is used to mainly do debugging. In this, we will see what parameters are important and how to set/calculate the values for better performance.