Persistent web ui

Description

The Spark application web ui (at port 4040) is extremely helpful for debugging application correctness & performance. However, once the application completes (and thus SparkContext is stopped), the web ui is no longer accessible. It would be great to refactor the UI so stage informations (perhaps in JSON format or directly in HTML format) are stored persistently in the file system and can be viewed after the fact.

Environment

None

Status

Assignee

Unassigned

Reporter

Reynold Xin

Labels

None

Test

None

Test

None

Fix versions

Priority

Major