Error while upgrading Airflow 1.11 to 1.15

Question

Hi I'm planning to upgrade my Airflow version from 1.11 to 1.15 which is deployed in OpenShift. As there are very large numbers of DAG's so I planned to upgrade in the bride release rather than going to Airflow 2.2

The error which I'm getting is most probably due to the fernet key:

ERROR: The `secret_key` setting under the webserver config has an insecure value - Airflow has 
failed safe and refuses to start. Please change this value to a new, per-environment,
 randomly generated string, for example using this command `openssl rand -hex 30`

Earlier I was using static Fernet Key and the YAML file is as follows:

apiVersion:v1
kind:Secret
metadata:
    name : airflow-secret
    namespace : CUSTOM_NAMESPACE
    labels: 
        app:airflow
type: Opaque
stringData:
   fernet-key: my_fernet_key

My Python Version : 3.8 My Airflow Webserver Config :

apiVersion: v1
kind: DeploymentConfig
metadata:
  name: airflow-webserver
  namespace: CUSTOM_NAMESPACE
  labels:
    app: airflow
spec:
  strategy: 
    type: Rolling
  trigger: 
    - type : ConfigChange
    - type : ImageChange
      ImageChangeParams: 
        automatic: true
        containerNames:
        - airflow-webserver
        from: 
          kind: ImageStreamTag
          namespace: CUSTOM_NAMESPACE
  replicas: 1
  revisionHistoryLimit : 10
  paused: false
  selector :
    app : airflow
    deploymentconfig : airflow-webserver
  template:
    metadata:
      labels:
        name: airflow-webserver
        app: airflow
        deploymentconfig : airflow-webserver
    spec: 
      volumes: 
      - name: airflow-dags
      persistentVolumeClaims:
        claimName: airflow-dags
    containers:
    - name:  airflow-webserver
      image:  airflow:latest
      resources:  
        limits:
          memory: 4Gi
      env:
        - name : FERNET_KEY
          valueFrom:
            secretKeyRef: 
              name: airflow-secrets
              key : fernet-key
        - name : SERVICE_ACCOUNT_NAME
          valueFrom:
            secretKeyRef:
              name: airflow-service-account
              key : service-account-name
      ports:
        - containerPort: 8080
          protocol: TCP

      volumeMounts:
        - name: airflow-dags
          mountPath: /opt/airflow/dags
        - name: airflow-logs
          mountPath: /opt/airflow/logs

My understanding is we need to somehow provide dynamic value in fernet key but for my case its static, Any Possible way to resolve the error.

Thank!

Answer 1

The main issue there was default value stored in airflow.cfg i.e.

secret_key = temporary_value

We can generate the secret_key by seeing the error message:

openssl rand -hex 30 

suppose the value is --> 94b9d6124ff2e9a5783d94dc7aa3641ebb8929bdbbf2f3989402f9e400ac

We need to put the value into the secret_key in airflow.cfg

secret_key = 94b9d6124ff2e9a5783d94dc7aa3641ebb8929bdbbf2f3989402f9e400ac