Scheduling Apache Spark Jobs on Clouds and On-Premise

Apache Oozie

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/running-spark-applications/spark-running-applications.pdf

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/running-spark-applications/content/automating_spark_jobs_with_oozie_spark_action.html

https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html

Workflow Manager + Oozie in Ambari

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.0.0/bk_workflow-management/content/ch_wfm_basics.html

Spark Submit Command Line (triggered via Cron or DevOps Tools)

ie. spark-submit

http://spark.apache.org/docs/latest/submitting-applications.html

Apache Livy - triggered from Apache NiFi or REST API

https://www.slideshare.net/bunkertor/running-apache-nifi-with-apache-spark-integration-options

Dockerized with Devops

https://hortonworks.com/blog/containerized-apache-spark-yarn-apache-hadoop-3-1/

Legacy Resources

https://www.slideshare.net/bunkertor/apache-spark-16-with-zeppelin-transformations-and-actions-on-rdds

https://www.slideshare.net/airisdata/apache-spark-overview-59903397