Package org.apache.spark.sql.pipelines.graph

Class PipelineExecution

Object

org.apache.spark.sql.pipelines.graph.PipelineExecution

public class PipelineExecution
extends Object

Executes a DataflowGraph by resolving the graph, materializing datasets, and running the flows.

param: context The context for this pipeline update.

Constructor Summary

Constructors

Constructor	Description
PipelineExecution(PipelineUpdateContext context)

Method Summary

Modifier and Type	Method	Description
void	dryRunPipeline()	Validates that the pipeline graph can be successfully resolved and validates it.
boolean	executionStarted()
void	runPipeline()	Starts pipeline execution and waits for it to complete before returning.
void	startPipeline()	Starts the pipeline execution by initializing the graph and starting the graph execution thread.
void	stopPipeline()	Stops the pipeline execution by stopping the graph execution thread.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PipelineExecution

public PipelineExecution(PipelineUpdateContext context)

Method Details

dryRunPipeline

public void dryRunPipeline()

Validates that the pipeline graph can be successfully resolved and validates it.

executionStarted

public boolean executionStarted()

runPipeline

public void runPipeline()

Starts pipeline execution and waits for it to complete before returning.

startPipeline

public void startPipeline()

Starts the pipeline execution by initializing the graph and starting the graph execution thread. This function does not block on the completion of the graph execution thread.

stopPipeline

public void stopPipeline()

Stops the pipeline execution by stopping the graph execution thread.