Airflow codemodding examples

This repo was created for the Toronto Airflow Meeting talk of May 27, 2025, demonstrating some examples of the use of codemods for manipulating Airflow DAGs.

Getting Started

To get started, ensure you have the dependencies installed (preferably in a virtualenv), eg:

python -m venv ~/.virtualenvs/codemod
. ~/.virtualenvs/codemod/bin/activate
pip install -r requirements.txt

To list the codemods available, run:

$ python -m libcst.tool list
basic.ConvertLegacyAirflowTaskFollows - Refactor v1-style task follows to use >> operators
datasetsql.AirflowInjectDatasetsForSql - Auto-inject Dataset inlet and outlets for SQL-based operators

To get help for a specific codemod:

python -m libcst.tool codemod datasetsql.AirflowInjectDatasetsForSql  --help

To run the follows auto-refactor:

python -m libcst.tool codemod basic.ConvertLegacyAirflowTaskFollows --no-format examples/follows/example_bash_operator.py 
Calculating full-repo metadata...
Executing codemod...
Finished codemodding 1 files!
 - Transformed 1 files successfully.
 - Skipped 0 files.
 - Failed to codemod 0 files.
 - 0 warnings were generated.

To run the dataset auto-inject example:

$ python -m libcst.tool codemod datasetsql.AirflowInjectDatasetsForSql --no-format examples/data-aware/example_postgres.py
Calculating full-repo metadata...
Executing codemod...
Finished codemodding 1 files!
 - Transformed 1 files successfully.
 - Skipped 0 files.
 - Failed to codemod 0 files.
 - 0 warnings were generated.

Notebooks

The notebook folder has some examples of how you can interactively debug your code with LibCST and SQLGlot. The LibCST notebook has a convenience method you can use to automatically generate graphs of CST code as shown in the presentation slides.