lifecycle.api.TaskGroupingHook

class hamilton.lifecycle.api.TaskGroupingHook

Implement this to run something after task grouping or task expansion. This will allow you to capture information about the tasks during Parallelize/Collect blocks in dynamic DAG execution.

final post_task_expand(*, run_id: str, task_id: str, parameters: Dict[str, Any])

Hook that is called immediately after a task is expanded into separate task. Note that this is only useful in dynamic execution.

Parameters:

• run_id – ID of the run, unique in scope of the driver.

• task_id – ID of the task.

• parameters – Parameters that are being passed to each of the expanded tasks.

final post_task_group(*, run_id: str, task_ids: List[str])

Hook that is called immediately after a task group is created. Note that this is only useful in dynamic execution, although we reserve the right to add this back into the standard hamilton execution pattern.

Parameters:

• run_id – ID of the run, unique in scope of the driver.

• task_ids – IDs of tasks that are in the group.

abstractmethod run_after_task_expansion(*, run_id: str, task_id: str, parameters: Dict[str, Any], **future_kwargs)

Hook that is called after task expansion. :param run_id: ID of the run, unique in scope of the driver. :param task_id: ID of the task that was expanded. :param parameters: Parameters that were passed to the task. :param future_kwargs: Additional keyword arguments – this is kept for backwards compatibility.

abstractmethod run_after_task_grouping(*, run_id: str, task_ids: List[str], **future_kwargs)

Hook that is called after task grouping. :param run_id: ID of the run, unique in scope of the driver. :param task_ids: List of tasks that were grouped together. :param future_kwargs: Additional keyword arguments – this is kept for backwards compatibility.