SFTP Gateway Support

3.0

Deploy SFTP Gateway on Kubernetes

TLDR - Quick Summary

What: Deploy SFTP Gateway as a containerized app on GKE Autopilot with Cloud SQL

Steps: Create Cloud SQL instance, configure Workload Identity, apply kubernetes-manifest.yaml

Overview

In this article we'll walk through deploying SFTP Gateway as a containerized application. The SFTP Gateway container images are available on Docker Hub, so you can pull them directly without needing to download and load image files.

A kubernetes manifest file will reference the Docker Hub images and deploy the SFTP Gateway containers.

Afterwards, you're able to access your deployment by navigating to your web browser and creating your initial Web Admin account.

Docker Hub Images

The SFTP Gateway container images are available on Docker Hub:

  • Backend: thorntech/sftpgateway-backend:latest
  • Admin UI: thorntech/sftpgateway-admin-ui:latest

You can pull these images directly:

docker pull thorntech/sftpgateway-backend:latest
docker pull thorntech/sftpgateway-admin-ui:latest

Prerequisites: Set up Cloud SQL and Workload Identity

Before deploying SFTP Gateway, you need to create a Cloud SQL PostgreSQL instance and configure Workload Identity for secure database access.

Step 1: Create a GKE Autopilot cluster

gcloud container clusters create-auto sftpgw-cluster \
  --region=us-central1 \
  --project=YOUR_PROJECT_ID

Step 2: Create a Cloud SQL PostgreSQL instance

# Create the Cloud SQL instance
gcloud sql instances create sftpgw-db \
  --database-version=POSTGRES_15 \
  --tier=db-f1-micro \
  --region=us-central1 \
  --project=YOUR_PROJECT_ID

# Set the postgres user password
gcloud sql users set-password postgres \
  --instance=sftpgw-db \
  --password=YOUR_SECURE_PASSWORD \
  --project=YOUR_PROJECT_ID

# Create the sftpgw database user
gcloud sql users create sftpgw \
  --instance=sftpgw-db \
  --password=YOUR_DATABASE_PASSWORD \
  --project=YOUR_PROJECT_ID

# Create the sftpgw database
gcloud sql databases create sftpgw \
  --instance=sftpgw-db \
  --project=YOUR_PROJECT_ID

Note: Save the database password - you'll need it for the Kubernetes manifest.

Step 3: Set up Workload Identity

Workload Identity allows GKE pods to authenticate to Cloud SQL without storing credentials.

# Create a GCP service account for Cloud SQL access
gcloud iam service-accounts create sftpgw-cloudsql \
  --display-name="SFTP Gateway Cloud SQL" \
  --project=YOUR_PROJECT_ID

# Grant Cloud SQL Client role
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:sftpgw-cloudsql@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/cloudsql.client"

# Allow the Kubernetes service account to impersonate the GCP service account
gcloud iam service-accounts add-iam-policy-binding \
  sftpgw-cloudsql@YOUR_PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:YOUR_PROJECT_ID.svc.id.goog[sftpgw/sftpgw-sa]" \
  --project=YOUR_PROJECT_ID

Step 4: Get the Cloud SQL connection name

gcloud sql instances describe sftpgw-db --project=YOUR_PROJECT_ID --format="value(connectionName)"

This will return something like YOUR_PROJECT_ID:us-central1:sftpgw-db. You'll need this for the manifest.

Step 5: Generate Security Credentials and Secrets

Before deploying, you must generate secure values for authentication and TLS. Run the following commands on your local workstation or in Cloud Shell.

Generate security credentials

# Generate security credentials
SECURITY_CLIENT_ID=$(openssl rand -hex 16)
SECURITY_CLIENT_SECRET=$(openssl rand -hex 32)
SECURITY_JWT_SECRET=$(uuidgen 2>/dev/null || cat /proc/sys/kernel/random/uuid 2>/dev/null || openssl rand -hex 16)

# Verify the values were generated
echo "CLIENT_ID: $SECURITY_CLIENT_ID"
echo "CLIENT_SECRET: $SECURITY_CLIENT_SECRET"
echo "JWT_SECRET: $SECURITY_JWT_SECRET"

Generate TLS certificate

Generate a self-signed certificate for the admin UI (or use your own certificate):

openssl req -x509 -newkey rsa:2048 -keyout tls.key -out tls.crt \
  -days 365 -nodes -subj "/CN=sftpgw-ui"

Create Kubernetes namespace and secrets

# Create namespace
kubectl create namespace sftpgw

# Create secrets from generated values
kubectl create secret generic sftpgw-secrets -n sftpgw \
  --from-literal=SECURITY_CLIENT_ID="$SECURITY_CLIENT_ID" \
  --from-literal=SECURITY_CLIENT_SECRET="$SECURITY_CLIENT_SECRET" \
  --from-literal=SECURITY_JWT_SECRET="$SECURITY_JWT_SECRET" \
  --from-literal=DB_PASSWORD="YOUR_DATABASE_PASSWORD"

# Create TLS secret from generated certificates
kubectl create secret generic sftpgw-ui-tls -n sftpgw \
  --from-file=tls.crt=tls.crt \
  --from-file=tls.key=tls.key

# Clean up local certificate files
rm -f tls.crt tls.key

Important: Replace YOUR_DATABASE_PASSWORD with the Cloud SQL database password you created in Step 2.

Verify secrets were created

kubectl get secrets -n sftpgw

# Expected output:
# NAME              TYPE     DATA   AGE
# sftpgw-secrets    Opaque   4      10s
# sftpgw-ui-tls     Opaque   2      10s

Step 6: Apply the Kubernetes manifest

On your local workstation (or in Cloud Shell), create the following file:

kubernetes-manifest.yaml

(See the contents of this file at the bottom of this document)

The manifest file is pre-configured to use the Docker Hub images:

  • Backend image: thorntech/sftpgateway-backend:latest
  • Admin UI image: thorntech/sftpgateway-admin-ui:latest

Required Configuration

Before applying the manifest, update the following values:

  • Cloud SQL connection name: Replace YOUR_PROJECT_ID:us-central1:sftpgw-db in the cloud-sql-proxy container args with your actual Cloud SQL connection name (from Step 4).
  • Workload Identity annotation: Replace YOUR_PROJECT_ID in the ServiceAccount annotation with your GCP project ID.

The manifest references the secrets you created in Step 5, so no need to edit security credentials or certificates.

Deploy the manifest

kubectl apply -f kubernetes-manifest.yaml

What did I just deploy

The kubernetes manifest file deploys the following resources:

Pods:

  • sftpgw-ui: The web admin portal running on nginx
  • sftpgw-backend: The SFTP server with Cloud SQL Auth Proxy sidecar
    • Main container: Java backend handling SFTP and API
    • Sidecar container: Cloud SQL Auth Proxy for secure database connectivity

Services:

  • sftpgw-ui (LoadBalancer): Public IP for the web admin portal
  • sftpgw-backend (ClusterIP): Internal service for backend
  • sftpgw-backend-lb (LoadBalancer): Public IP for SFTP connections

Database:

  • Cloud SQL PostgreSQL (managed service, not a pod)

Run the following command to get a list of pods:

kubectl get pods -n sftpgw

NAME                              READY   STATUS    RESTARTS   AGE
sftpgw-backend-79fdc45c97-grr8b   2/2     Running   0          5m
sftpgw-ui-76d6d557c7-bvkl7        1/1     Running   0          5m

Note: The backend pod shows 2/2 because it has two containers (backend + cloud-sql-proxy).

How to connect to the web admin portal

:::tip Restrict Admin UI Access The manifest includes loadBalancerSourceRanges on the UI service to restrict access to specific IP addresses. Before deploying, replace YOUR_IP_ADDRESS/32 with your actual IP (find it at https://ifconfig.me). This prevents unauthorized access to the admin portal from the public internet. :::

Run the following command to get a list of services:

kubectl get service -n sftpgw

NAME                TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                      AGE
sftpgw-backend      ClusterIP      34.118.231.200   <none>           8080/TCP,22/TCP              5m
sftpgw-backend-lb   LoadBalancer   34.118.226.57    34.139.141.133   22:31327/TCP                 5m
sftpgw-ui           LoadBalancer   34.118.234.34    34.74.250.188    80:31773/TCP,443:30730/TCP   5m

In the above output, make special note of the EXTERNAL-IP column:

  • sftpgw-ui: This is the public IP for connecting to the web admin portal. In this example, it’s 34.74.250.188
  • sftpgw-backend-lb: This is the public IP for connecting to the SFTP service. In this example, it’s 34.139.141.133

Paste the sftpgw-ui EXTERNAL-IP (e.g. 34.74.250.188) into your web browser.

You should see the SFTP Gateway web admin portal.

To configure your deployment, make the following changes:

  • Create a web admin user and password
  • Log in as this web admin user
  • Go to the Settings tab and create a new Cloud Connection to a GCS bucket
  • Go to the Folders tab and point the root folder to the Cloud Connection you just created
  • Go to the Users tab and create a new SFTP user

How to connect to the SFTP service

The SFTP service is hosted on the Java backend.

And in order to connect to the SFTP service, you will need to go through the load balancer.

In the above list of services, find the sftpgw-backend-lb and look at the EXTERNAL-IP. In my example, it’s 34.139.141.133

You can connect to this load balancer public IP on port 22:

sftp robtest@34.139.141.133

You should be prompted for a server fingerprint, and then connect via SFTP if your credentials are correct.

Scaling

UI Scaling

The Admin UI can safely scale to multiple replicas without any issues:

kubectl scale deployment/sftpgw-ui -n sftpgw --replicas=2

The UI is stateless - it serves static files and proxies API requests to the backend. As long as SECURITY_CLIENT_ID and SECURITY_CLIENT_SECRET are configured (and match the backend), multiple UI pods work seamlessly.

Backend Scaling

The backend deployment can be scaled, but requires additional configuration for session management:

kubectl scale deployment/sftpgw-backend -n sftpgw --replicas=2

Note: Without sticky sessions or a shared session store, users may experience login issues when requests are load-balanced across multiple backend pods. See the Troubleshooting section below.

Licensing

The built-in license included in the backend image supports up to 5 SFTP users. For production deployments requiring more users, add a license key via environment variable.

Add the LICENSE environment variable to the backend deployment:

- name: LICENSE
  value: "your-license-key-here"

Or add it to an existing deployment:

kubectl set env deployment/sftpgw-backend -n sftpgw LICENSE="your-license-key-here"

Contact Thorn Technologies to obtain a license key for additional users.

Client IP Preservation

The manifest configures externalTrafficPolicy: Local on the SFTP service, which preserves client IP addresses in audit logs.

How It Works

Client → LoadBalancer → Node with Pod → Pod (direct)
                                         ↑
                                 Client IP preserved

Without this setting (using Cluster mode), client IPs would be replaced with internal node IPs due to SNAT, and audit logs would show node IPs instead of actual client IPs.

Trade-off: With Local mode, traffic is distributed evenly across nodes rather than pods. If you have 2 pods on Node A and 1 pod on Node B, Node A receives 50% of traffic (not 66%). For most deployments, this is acceptable.

Performance Tuning

Resource Recommendations

Choose resources based on your expected throughput:

Light Workloads (<10 TB/day)

resources:
  requests:
    cpu: "2000m"
    memory: "2Gi"
  limits:
    cpu: "4000m"
    memory: "4Gi"
replicas: 1

Expected throughput: ~100-150 MB/s

Medium Workloads (10-50 TB/day)

resources:
  requests:
    cpu: "4000m"
    memory: "4Gi"
  limits:
    cpu: "8000m"
    memory: "8Gi"
env:
- name: JAVA_OPTS
  value: "-Xms2g -Xmx6g -XX:+UseG1GC"
replicas: 1

Expected throughput: ~400-500 MB/s

High Workloads (50-150 TB/day)

resources:
  requests:
    cpu: "4000m"
    memory: "4Gi"
  limits:
    cpu: "8000m"
    memory: "8Gi"
env:
- name: JAVA_OPTS
  value: "-Xms2g -Xmx6g -XX:+UseG1GC"
replicas: 3

Expected throughput: ~1,500-1,700 MB/s

Horizontal Pod Autoscaler

For automatic scaling based on load:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: sftpgw-backend-hpa
  namespace: sftpgw
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: sftpgw-backend
  minReplicas: 2
  maxReplicas: 8
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
    scaleUp:
      stabilizationWindowSeconds: 30

Troubleshooting

Login succeeds but immediately logs out

If you can log in to the web admin portal but are immediately logged out (or see 401 errors), this is typically caused by running multiple backend replicas without a shared JWT signing key.

Cause: If SECURITY_JWT_SECRET is not provided, each backend pod generates its own signing key at startup. When requests are load-balanced across multiple pods, a request may hit a different pod than the one that issued the token, causing validation to fail.

Solution: Ensure SECURITY_JWT_SECRET is set via Kubernetes Secret (as shown in Step 5). When all pods share the same JWT secret, they can validate tokens issued by any other pod.

Note: This issue does not affect SFTP connections - only the admin web portal.

Admin UI shows certificate error

If the browser shows a certificate warning, this is expected when using self-signed certificates. You can:

  • Proceed through the warning for testing
  • Replace the self-signed certificate with a valid certificate from a Certificate Authority

Cannot connect to SFTP service

  1. Verify the backend pod is running: kubectl get pods -n sftpgw
  2. Check the load balancer has an external IP: kubectl get svc sftpgw-backend-lb -n sftpgw
  3. Ensure your SFTP user has been created in the web admin portal
  4. Verify the user's SSH key or password is configured correctly

The kubernetes manifest file

# SFTP Home PersistentVolumeClaim
# Note: Namespace was created in Step 5 when creating secrets
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sftpgw-home-pvc
  namespace: sftpgw
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
# Service Account for Workload Identity
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sftpgw-sa
  namespace: sftpgw
  annotations:
    # Replace YOUR_PROJECT_ID with your GCP project ID
    iam.gke.io/gcp-service-account: sftpgw-cloudsql@YOUR_PROJECT_ID.iam.gserviceaccount.com
---
# SFTP Gateway Backend Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sftpgw-backend
  namespace: sftpgw
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sftpgw-backend
  template:
    metadata:
      labels:
        app: sftpgw-backend
    spec:
      serviceAccountName: sftpgw-sa
      containers:
      # Main SFTP Gateway container
      - name: sftpgw-backend
        image: thorntech/sftpgateway-backend:latest
        env:
        - name: SECURITY_CLIENT_ID
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: SECURITY_CLIENT_ID
        - name: SECURITY_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: SECURITY_CLIENT_SECRET
        - name: SECURITY_JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: SECURITY_JWT_SECRET
        # Cloud SQL Auth Proxy connects on localhost:5432
        - name: SPRING_DATASOURCE_URL
          value: "jdbc:postgresql://localhost:5432/sftpgw"
        - name: SPRING_DATASOURCE_USERNAME
          value: "sftpgw"
        - name: SPRING_DATASOURCE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: DB_PASSWORD
        - name: SPRING_PROFILES_ACTIVE
          value: "local"
        - name: FEATURES_INSTANCE_CLOUD_PROVIDER
          value: "gcp"
        ports:
        - containerPort: 8080
        - containerPort: 22
        volumeMounts:
        - name: sftpgw-home
          mountPath: /home
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 90
          periodSeconds: 15
          timeoutSeconds: 10
          failureThreshold: 4
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 45
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3
      # Cloud SQL Auth Proxy sidecar
      - name: cloud-sql-proxy
        image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0
        args:
          - "--port=5432"
          # Replace with your Cloud SQL connection name
          - "YOUR_PROJECT_ID:us-central1:sftpgw-db"
        securityContext:
          runAsNonRoot: true
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
      volumes:
      - name: sftpgw-home
        persistentVolumeClaim:
          claimName: sftpgw-home-pvc
---
# SFTP Gateway Backend Service
apiVersion: v1
kind: Service
metadata:
  name: sftpgw-backend
  namespace: sftpgw
spec:
  selector:
    app: sftpgw-backend
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: sftp
    port: 22
    targetPort: 22
  type: ClusterIP
---
# Note: TLS secret (sftpgw-ui-tls) was created in Step 5
# SFTP Gateway UI Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sftpgw-ui
  namespace: sftpgw
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sftpgw-ui
  template:
    metadata:
      labels:
        app: sftpgw-ui
    spec:
      containers:
      - name: sftpgw-ui
        image: thorntech/sftpgateway-admin-ui:latest
        env:
        - name: BACKEND_URL
          value: "http://sftpgw-backend:8080/"
        - name: SECURITY_CLIENT_ID
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: SECURITY_CLIENT_ID
        - name: SECURITY_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              name: sftpgw-secrets
              key: SECURITY_CLIENT_SECRET
        - name: FEATURES_INSTANCE_CLOUD_PROVIDER
          value: "gcp"
        - name: WEBSITE_BUNDLE_CRT
          valueFrom:
            secretKeyRef:
              name: sftpgw-ui-tls
              key: tls.crt
        - name: WEBSITE_KEY
          valueFrom:
            secretKeyRef:
              name: sftpgw-ui-tls
              key: tls.key
        ports:
        - containerPort: 80
        - containerPort: 443
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 30
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 10
          periodSeconds: 10
      restartPolicy: Always
---
# SFTP Gateway UI Service
apiVersion: v1
kind: Service
metadata:
  name: sftpgw-ui
  namespace: sftpgw
spec:
  selector:
    app: sftpgw-ui
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 443
  type: LoadBalancer
  # Restrict Admin UI access to specific IP addresses
  loadBalancerSourceRanges:
  - "YOUR_IP_ADDRESS/32"  # Replace with your IP (find it at https://ifconfig.me)
---
# SFTP LoadBalancer Service
apiVersion: v1
kind: Service
metadata:
  name: sftpgw-backend-lb
  namespace: sftpgw
spec:
  selector:
    app: sftpgw-backend
  ports:
  - name: sftp
    port: 22
    targetPort: 22
    protocol: TCP
  type: LoadBalancer
  # Preserve client IP addresses in audit logs
  externalTrafficPolicy: Local
← PreviousNext →