Right now on each machine, we create M * R temporary files for shuffle, where M = number of map tasks, R = number of reduce tasks.
This can be pretty high when there are lots of mappers and reducers (e.g. 1k map * 1k reduce = 1 million files for a single shuffle). The high number can cripple the file system and significantly slow the system down.
We should cut this number down to O(R) instead of O(M*R).