airflow-docker-compose

Description

A reasonably light wrapper around docker-compose to make it simple to start a local airflow instance in docker.

Usage

airflow-docker-compose --help
airflow-docker-compose up

Configuration

Note, this library assumes the docker-compose utility is available in your path.

In order to use this tool, you should have a local dags folder containing your dags. You should also have a pyproject.toml file which minimally looks like

[tool.airflow-docker-compose]
docker-network = 'network-name'

In order to set airflow configuration, you can use the airflow-environment-variables key. This allows you to set any airflow.cfg variables like so:

[tool.airflow-docker-compose]
airflow-environment-variables = {
    AIRWFLOW_WORKER_COUNT = 4
    AIRFLOW__AIRFLOWDOCKER__FORCE_PULL = 'false'
}