This is the last article from our Spark Optimization Series. Optimizing a spark query is challenging as well as interesting, as a Data Engineer, I love to optimize and look into small things that can be improved to increase the performance of the process whether its transformation or aggregation.