Kubernetes Turn Server Connection Issue

Question

I have applications that have to use Turn Server. When I try to make all connections over the pods, I get a "Connection reset by peer" error on 6 out of 10 connections. The TURN address resolves over the host and provides access over ClusterIP. When I run this from a public IP address, there is no problem.

Turn YAML:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  namespace: default
  name: coturn
  labels:
    app.kubernetes.io/name: coturn
    app.kubernetes.io/instance: coturn
    app.kubernetes.io/version: 0.0.1
spec:
  # replicas: 1
  selector:
    matchLabels:
          app.kubernetes.io/name: coturn
          app.kubernetes.io/instance: coturn
          app.kubernetes.io/version: 0.0.1
  template:
    metadata:
      labels:
            app.kubernetes.io/name: coturn
            app.kubernetes.io/instance: coturn
            app.kubernetes.io/version: 0.0.1
    spec:
      hostNetwork: true
      containers:
        - name: coturn
          image: coturn/coturn
          imagePullPolicy: Always
          securityContext:
            privileged: true
          env:
          - name: MY_POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: STARTUP_SCRIPT
            value: |
              #! /bin/bash
              echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal
              echo done
          ports:
            - name: turn-port1
              containerPort: 3478
              hostPort: 3478
              protocol: UDP
            - name: turn-port2
              containerPort: 3478
              hostPort: 3478
              protocol: TCP
          args:
          #   - --stun-only
            - -v
            - --user "test:test"
            - --external-ip="$(detect-external-ip)/$MY_POD_IP"
            - --realm="$(detect-external-ip)"

--- 

apiVersion: v1
kind: Service
metadata:
  name: coturn
  namespace: default
  labels:
       app.kubernetes.io/name: coturn
       app.kubernetes.io/instance: coturn
       app.kubernetes.io/version: 0.0.1
spec:
  type: ClusterIP
  ports:
    - port: 3478
      targetPort: 3478
      protocol: UDP
      name: turn-port1
    - port: 3478
      targetPort: 3478
      protocol: TCP
      name: turn-port2
    
  selector:
       app.kubernetes.io/name: coturn
       app.kubernetes.io/instance: coturn
       app.kubernetes.io/version: 0.0.1

Log:

0: IPv4. Connected from: 10.2.5.12:52224
0: IPv4. Connected to: 10.3.57.50:3478
0: allocate sent
0: allocate response received: 
0: allocate sent
0: allocate response received: 
0: success
0: IPv4. Received relay addr: public_ip:55179
0: clnet_allocate: rtv=0
0: refresh sent
0: refresh response received: 
0: success
0: IPv4. Connected from: 10.2.5.12:52226
0: IPv4. Connected to: 10.3.57.50:3478
0: IPv4. Connected from: 10.2.5.12:52228
0: IPv4. Connected to: 10.3.57.50:3478
0: allocate sent
0: allocate response received: 
0: allocate sent
0: allocate response received: 
0: success
0: IPv4. Received relay addr: public_ip:52353
0: clnet_allocate: rtv=0
0: refresh sent
0: refresh response received: 
0: success
0: allocate sent
0: allocate response received: 
0: allocate sent
0: allocate response received: 
0: success
0: IPv4. Received relay addr: public_ip:54002
0: clnet_allocate: rtv=0
0: refresh sent
0: refresh response received: 
0: success
0: create perm sent: public_ip:54002
0: cp response received: 
0: success
0: create perm sent: public_ip:52353
0: cp response received: 
0: success
0: tcp connect sent
0: connection bind sent
recv: Connection reset by peer

Answer 1

My theory about the issue: connection reset by peer means that the packet has been marked as invalid due that the server is congested and serving large payloads; So the service ClusterIP will face some difficulties to attend the packets internally. To try to mitigate this issue, you should upgrade your Kubernetes version to V1.15+ or higher. Also, as a workaround, you can apply this rule in your cluster:

apiVersion: extensions/v1beta1
 
kind: DaemonSet
 
metadata:
 
 name: startup-script
 
 labels:
 
   app: startup-script
 
spec:
 
 template:
 
   metadata:
 
     labels:
 
       app: startup-script
 
   spec:
 
     hostPID: true
 
     containers:
 
     - name: startup-script
 
       image: gcr.io/google-containers/startup-script:v1
 
       imagePullPolicy: IfNotPresent
 
       securityContext:
 
         privileged: true
 
       env:
 
       - name: STARTUP_SCRIPT
 
         value: |
 
           #! /bin/bash
 
           echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal
 
           echo done            
 

You can find more details about this workaround using a Kubernetes service of type ClusterIP in this guide: