How to Schedule a Python Automation Script in n8n for Daily CSV Exports

Many teams write a small Python script that pulls data from an internal API, formats the result as CSV, and uploads the file to a shared location. Running that script manually or via a generic cron on a VM works, but it lacks visibility, retry logic, and easy scaling. n8n – an open‑source workflow automation tool – provides a visual canvas, built‑in scheduling, and native support for HTTP requests, making it a natural home for a python automation script that must run every day.

In this guide you will set up a complete end‑to‑end solution: install n8n with Docker, create a workflow that calls your Python code, schedule it with a Cron trigger, add error handling, and finally deploy the workflow as a production‑grade daily job. All steps are code‑first, so you can copy‑paste the commands and configuration files directly into your environment.

What You’ll Learn

• How to containerize a pure‑Python script for use inside n8n.
• How to configure n8n’s Cron trigger for daily execution.
• How to call external APIs from Python and write a CSV file.
• How to add retry, timeout, and notification logic in n8n.
• How to deploy the workflow on a self‑hosted Docker instance.

Prerequisites

• Docker Engine ≥ 20.10 (official docs: Docker install guide).
• Basic familiarity with Python 3.10+ and pip.
• Access to the API you will query (API key, endpoint URL, and required scopes).
• Optional: a cloud bucket (e.g., AWS S3, Google Cloud Storage) for CSV storage.

1. Install and run n8n with Docker

Why Docker?

Running n8n in a container isolates the runtime, guarantees the same Node.js version across environments, and simplifies upgrades. The official n8n Docker image is maintained on Docker Hub (n8nio/n8n).

Step‑by‑step

# Create a directory for n8n data
mkdir -p $HOME/n8n/data

# Pull the latest stable image
docker pull n8nio/n8n:latest

# Run n8n in detached mode, exposing port 5678
docker run -d \
  --name n8n \
  -p 5678:5678 \
  -v $HOME/n8n/data:/home/node/.n8n \
  -e N8N_BASIC_AUTH_ACTIVE=true \
  -e N8N_BASIC_AUTH_USER=admin \
  -e N8N_BASIC_AUTH_PASSWORD=changeme \
  n8nio/n8n:latest

After the container starts, open http://localhost:5678 and log in with the credentials you set. The UI will load the workflow canvas.

2. Write and containerize the Python automation script

Script requirements

• Make an HTTP GET request to an internal API.
• Transform JSON payload into a CSV file.
• Upload the CSV to a cloud bucket (optional).

Example script (save as export_data.py)

import os
import csv
import requests
from datetime import datetime
import boto3  # AWS S3 client – install via pip if needed

API_URL = os.getenv('API_URL')
API_TOKEN = os.getenv('API_TOKEN')
BUCKET_NAME = os.getenv('BUCKET_NAME')  # optional
EXPORT_PATH = f"/tmp/export_{datetime.utcnow().strftime('%Y%m%d')}.csv"

def fetch_data():
    headers = {'Authorization': f'Bearer {API_TOKEN}'}
    response = requests.get(API_URL, headers=headers, timeout=30)
    response.raise_for_status()
    return response.json()

def write_csv(data):
    # Assume data is a list of dicts with consistent keys
    if not data:
        raise ValueError("No data returned from API")
    keys = data[0].keys()
    with open(EXPORT_PATH, 'w', newline='') as f:
        writer = csv.DictWriter(f, fieldnames=keys)
        writer.writeheader()
        writer.writerows(data)

def upload_to_s3():
    s3 = boto3.client('s3')
    s3.upload_file(EXPORT_PATH, BUCKET_NAME, os.path.basename(EXPORT_PATH))

def main():
    data = fetch_data()
    write_csv(data)
    if BUCKET_NAME:
        upload_to_s3()
    print(f"Export completed: {EXPORT_PATH}")

if __name__ == "__main__":
    main()

Dockerfile for the script

Using a minimal Python base image keeps the final image small.

FROM python:3.11-slim

# Install runtime dependencies
RUN pip install --no-cache-dir requests boto3

# Copy script
COPY export_data.py /app/export_data.py
WORKDIR /app

# Default command – will be overridden by n8n
ENTRYPOINT ["python", "export_data.py"]

Build the image

docker build -t myorg/python-export:latest .

Push the image to a registry your n8n instance can reach (Docker Hub, GitHub Packages, or a private registry). Example for Docker Hub:

docker tag myorg/python-export:latest mydockerhubusername/python-export:latest
docker push mydockerhubusername/python-export:latest

3. Build the n8n workflow

Node layout

1. Cron Trigger – fires once per day.
2. Execute Command – runs the Docker container with the script.
3. If/Else – checks the exit code.
4. Slack / Email – notifies on failure (optional).

Configure the Execute Command node

• Command: docker run --rm -e API_URL="{{ $json.apiUrl }}" -e API_TOKEN="{{ $json.apiToken }}" -e BUCKET_NAME="{{ $json.bucket }}" mydockerhubusername/python-export:latest
• Set Working Directory to /tmp (default is fine).
• Enable Return Output so the next node can inspect stdout/stderr.

Pro Tip: Store secrets such as API_TOKEN in n8n’s built‑in Credentials store (see n8n docs Credentials) and reference them with {{$credentials.myApi.key}} to avoid hard‑coding them in the workflow.

Example Cron expression for 02:00 UTC daily

0 2 * * *

4. Add a daily Cron trigger

In the n8n UI, drag a Cron node onto the canvas, open its settings, and paste the expression above. The node’s “Timezone” dropdown lets you pick a region; choose UTC for consistency across deployments.

Connect the Cron node’s output to the Execute Command node. When you press “Execute Workflow” manually, you can verify that the Docker container runs and the CSV is generated.

5. Add error handling and notifications

Retry logic

Wrap the Execute Command node in an Execute Workflow node that has a “Retry on Failure” setting (max 3 attempts, 5‑minute interval). This avoids transient network hiccups.

Conditional branching

# In the If/Else node, test the exit code
{{ $node["Execute Command"].json["exitCode"] === 0 }}

• If true → end of workflow (success).
• If false → route to a Slack node (or Email) with the error details.

Logging

Enable n8n’s built‑in “Execution Log” (Settings → Execution) and set the retention period to 30 days. This gives you a searchable history of each daily run.

6. Deploy the workflow as a production job

Export the workflow JSON

Open the workflow, click the three‑dot menu, and choose “Export”. Save the .json file in your version‑control repository.

Automate deployment with Docker Compose

version: "3.9"

services:
  n8n:
    image: n8nio/n8n:latest
    restart: unless-stopped
    ports:
      - "5678:5678"
    volumes:
      - ./n8n/data:/home/node/.n8n
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=admin
      - N8N_BASIC_AUTH_PASSWORD=${N8N_PASSWORD}
      - EXECUTIONS_PROCESS=main
    command: >
      n8n start
      --workflow="export-data-daily"
      --loadWorkflowFromFile="/home/node/.n8n/workflows/export-data-daily.json"

Place the exported workflow file at ./n8n/data/workflows/export-data-daily.json. When the container starts, n8n will automatically import and activate the workflow.

Running in production

• Use a reverse proxy (e.g., Caddy or Nginx) with TLS termination.
• Store environment variables (API keys, passwords) in a secret manager such as Docker secrets, HashiCorp Vault, or the host OS’s .env file.
• Monitor the container with health checks (Docker health‑check command) and a service‑level alerting system (Prometheus + Alertmanager).

Comparison of scheduling options

Frequently Asked Questions

Can I run the Python script directly inside n8n without Docker?

Yes. n8n provides a community “Python” node that executes a script in a sandboxed environment. However, the sandbox does not include native binaries (e.g., awscli) and has limited package management, so containerizing the script is more reliable for production.

How do I secure API credentials?

Store them in n8n’s Credentials store (type “API Key” or “OAuth2”) and reference them with {{$credentials.myApi.key}}. The credentials are encrypted at rest in the .n8n folder.

What happens if the API returns a 429 (rate‑limited) response?

Modify the Python script to catch requests.exceptions.HTTPError, inspect response.status_code, and implement exponential back‑off. The script can then exit with a non‑zero code, letting n8n’s retry logic handle the delay.

Can I store the CSV in Google Cloud Storage instead of S3?

Yes. Replace the boto3 upload block with the google-cloud-storage client. Install the package in the Dockerfile (pip install google-cloud-storage) and set the appropriate environment variables (e.g., GOOGLE_APPLICATION_CREDENTIALS).

How do I view the CSV file generated by the script?

The script writes the file to /tmp inside the container, which is discarded after the run. To keep a copy, either upload it to a bucket (as shown) or mount a host volume into the container and write to that path. Example Docker run flag: -v /var/data/exports:/exports and adjust EXPORT_PATH accordingly.

Is there a way to version‑control the entire n8n workflow?

Yes. Export the workflow JSON and store it in a Git repository. You can also use the n8n CLI (n8n import:workflow) to load the JSON automatically during container startup, as demonstrated in the Docker Compose example.

What monitoring options are available for n8n?

n8n emits execution metrics that can be scraped by Prometheus (see the official monitoring guide). Combine this with Grafana dashboards or Alertmanager to get alerts on failed runs.

Conclusion & Next Steps

By containerizing your Python automation script and wiring it into an n8n workflow, you gain a maintainable, observable, and easily repeatable daily job. The next actions are:

1. Commit the Dockerfile, script, and workflow JSON to your code repository.
2. Set up a CI pipeline that builds and pushes the Docker image on every change.
3. Configure production secrets in n8n’s Credentials store and verify that the Cron trigger fires at the expected UTC time.
4. Add Prometheus scraping and a Grafana alert for any non‑zero exit codes.
5. Periodically review the CSV schema and update the script to handle API version changes.

With these steps completed, your daily CSV export will run reliably, and you’ll have full visibility into each execution without writing custom cron jobs.

Official documentation

• n8n Documentation
• n8n Docker image (Docker Hub)
• Docker run reference
• Python csv module
• Boto3 documentation
• n8n Credentials guide
• n8n monitoring guide