Lifecycle Adapters¶
Currently a few of the API extensions are still experimental. Note this doesn’t mean they’re not well-tested or thought out – rather that we’re actively looking for feedback. More docs upcoming, but for now fish around the experimental package, and give the extensions a try!
The other extensions live within plugins. These are fully supported and will be backwards compatible across major versions.
Customization¶
The subsequent documents contain public-facing APIs for customizing Hamilton’s execution. Note that the public-facing APIs are still a work in progress – we will be improving the documentation. We plan for the APIs, however, to be stable looking forward.
- lifecycle.ResultBuilder
- lifecycle.LegacyResultMixin
- lifecycle.api.GraphAdapter
GraphAdapter
GraphAdapter.build_result()
GraphAdapter.check_input_type()
GraphAdapter.check_node_type_equivalence()
GraphAdapter.do_build_result()
GraphAdapter.do_check_edge_types_match()
GraphAdapter.do_node_execute()
GraphAdapter.do_validate_input()
GraphAdapter.execute_node()
GraphAdapter.input_types()
GraphAdapter.output_type()
- lifecycle.NodeExecutionHook
- lifecycle.api.GraphExecutionHook
- lifecycle.api.EdgeConnectionHook
- lifecycle.api.NodeExecutionMethod
- lifecycle.api.StaticValidator
- lifecycle.api.GraphConstructionHook
Available Adapters¶
In addition to the base classes for lifecycle adapters, we have a few adapters implemented and available for use. Note that some of these are plugins, meaning they require installing additional (external) libraries.
Recall to add lifecycle adapters, you just need to call the with_adapters
method of the driver:
dr = (
driver
.Builder()
.with_modules(...)
.with_adapters(
Adapter1(...),
Adapter2(...),
*more_adapters)
...build()
)
- lifecycle.PDBDebugger
- lifecycle.PrintLn
- plugins.h_tqdm.ProgressBar
ProgressBar
ProgressBar.__init__()
ProgressBar.post_graph_execute()
ProgressBar.post_node_execute()
ProgressBar.pre_graph_execute()
ProgressBar.pre_node_execute()
ProgressBar.run_after_graph_execution()
ProgressBar.run_after_node_execution()
ProgressBar.run_before_graph_execution()
ProgressBar.run_before_node_execution()
- plugins.h_ddog.DDOGTracer
DDOGTracer
DDOGTracer.__init__()
DDOGTracer.post_graph_execute()
DDOGTracer.post_node_execute()
DDOGTracer.post_task_execute()
DDOGTracer.pre_graph_execute()
DDOGTracer.pre_node_execute()
DDOGTracer.pre_task_execute()
DDOGTracer.run_after_graph_execution()
DDOGTracer.run_after_node_execution()
DDOGTracer.run_after_task_execution()
DDOGTracer.run_before_graph_execution()
DDOGTracer.run_before_node_execution()
DDOGTracer.run_before_task_execution()
AsyncDDOGTracer
- lifecycle.FunctionInputOutputTypeChecker
- plugins.h_slack.SlackNotifier
- lifecycle.GracefulErrorAdapter
- plugins.h_spark.SparkInputValidator
- plugins.h_narhwals.NarwhalsAdapter
- plugins.h_narhwals.NarwhalsDataFrameResultBuilder
- plugins.h_mlflow.MLFlowTracker
MLFlowTracker
MLFlowTracker.__init__()
MLFlowTracker.post_graph_construct()
MLFlowTracker.post_graph_execute()
MLFlowTracker.post_node_execute()
MLFlowTracker.pre_graph_execute()
MLFlowTracker.pre_node_execute()
MLFlowTracker.run_after_graph_construction()
MLFlowTracker.run_after_graph_execution()
MLFlowTracker.run_after_node_execution()
MLFlowTracker.run_before_graph_execution()
MLFlowTracker.run_before_node_execution()
- lifecycle.NoEdgeAndInputTypeChecking
- plugins.h_openlineage.OpenLineageAdapter