Overview of Concepts

Now that you’re familiar with the basics and have run your own dataflow, let’s dive into the concepts that makes Hamilton unique and powerful.

• Lexicon
• Hamilton Functions
  • All transforms are simple python functions
  • Dataflow structure == implementation
  • Dynamic DAGs
  • Modules and Helper Functions
• Running Your Code
  • Parameterizing the DAG
    • Calling Execute()
    • Materializing DAG Nodes
    • Visualizing Execution
      • Visualize just execution required to create outputs
      • Visualize the entire DAG constructed
  • Should I define my own Driver?
• Customizing Execution
• The Driver
  • Result Builders
  • Graph Adapters
    • Local Execution
• Dynamic DAGs/Parallel Execution
• Overview of Decorators
  • Determine if a function should be included
  • Specify the implementation of a function
  • Split the function into multiple nodes
• Best Practices
  • Migrating to Hamilton
    • Continuous Integration for Comparisons
    • Integrate into your code base via a “custom wrapper object”
  • Function Naming
    • It enables you to define your Hamilton dataflow
    • It drives collaboration and reuse
    • It serves as documentation itself
  • Code Organization
    • Team thinking
    • Helps isolate what you’re working on
    • Enables you to replace parts of your DAG easily for different contexts
  • Function Modifiers
    • Why would I use them?
  • Common Indices
    • Best practice:
  • Output Immutability
    • Best practice:
  • Using within your ETL System
    • Compatibility Matrix
    • ETL Recipe
  • Loading Data
    • Plugging in new Data Sources
      • Modules as Interfaces
      • Using the Config to Decide Sources
• Extensions