Skip to main content

Config

This class represents the configuration for Spark-specific settings. It includes default Spark configuration parameters, the URL for the Spark History Server, a list of supported optional features, and logging configuration for Spark applications.

Attributes

AttributeTypeDescription
DefaultSparkConfigmap[string]stringKey value pairs of default spark configuration that should be applied to every SparkJob.
SparkHistoryServerURLstringURL for SparkHistory Server that each job will publish the execution history to.
Features[][Feature](feature.md?sid=flyteplugins_go_tasks_plugins_k8s_spark_feature)List of optional features supported.
LogConfig[LogConfig](logconfig.md?sid=flyteplugins_go_tasks_plugins_k8s_spark_logconfig)Config for log links for spark applications.