DeepSeek Harness with local GLM-4.7-Flash model on vLLM-Ascend

README.md

DeepSeek Harness with local Qwen3-30B-A3B-Thinking-2507 model on vLLM-Ascend

This gist demonstrates how to deploy DeepSeek Harness (DSH) on Kubernetes and connect it to a local Qwen3-30B-A3B-Thinking-2507 model running on the same cluster. The instructions have been tested on the GMKtec NucBox K11 Mini PC connected to the OrangePi AI Studio Pro extension dock.

Deploying Qwen3-30B-A3B-Thinking-2507 on vLLM-Ascend

kubectl apply -f vllm.yaml

Deploying DeepSeek Harness

First clone the repository runzhliu/deepseek-harness-docker. Use the release tag image-v0.1.2-rc.1-r1.

git clone https://github.com/runzhliu/deepseek-harness-docker.git \
    -b image-v0.1.2-rc.1-r1

Next, install the deepseek-harness Helm chart with the provided dsh-values.yaml.

helm install deepseek-harness \
    deepseek-harness-docker/charts/deepseek-harness \
    -f dsh-values.yaml

Once the DSH pod is running, copy the provided settings.yaml to $DSH_HOME/settings.yaml within the pod. By default DSH_HOME is /home/node/.dsh.

kubectl cp settings.yaml deepseek-harness-0:/home/node/.dsh/settings.yaml

Optionally issue the TLS certificate with cert-manager and configure Ingress.

kubectl apply -f dsh-extra.yaml

Now either access DSH web portal with kubectl port-forward and visit http://localhost:3080/?token=XXXX:

kubectl port-forward svc/deepseek-harness 3080:3080

Or access it via Ingress with the appropriate URL. The login token is available from the pod logs.

kubectl logs sts/deepseek-harness

Changelog

• 2026-09-06 22:00 HKT: add DeepSeek Harness related manifests and update instructions in README
• 2026-09-06 18:00 HKT: fix title and replace vLLM-Ascend example with working deployment
• 2026-09-05: add initial vLLM-Ascend example

dsh-extra.yaml

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: deepseek-harness-tls
spec:
  secretName: deepseek-harness-tls
  duration: 8760h
  renewBefore: 720h
  issuerRef:
    name: root-ca-clusterissuer
    kind: ClusterIssuer
  commonName: dsh.internal.donaldsebleung.com
  dnsNames:
    - dsh.internal.donaldsebleung.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cluster-name: donaldsebleung-nucbox-k11
  name: deepseek-harness
spec:
  tls:
    - hosts:
        - 'dsh.internal.donaldsebleung.com'
      secretName: deepseek-harness-tls
  rules:
    - host: 'dsh.internal.donaldsebleung.com'
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: deepseek-harness
                port:
                  number: 3080

dsh-values.yaml

image:
  repository: quay.io/donaldsebleung/deepseek-harness
  tag: 0.1.2-rc.1-r1
extraEnv:
  - name: OPENAI_API_KEY
    value: hunter2
args:
  - web
  - --patch
  - /opt/deepseek-harness/web.cordis.patch.yml
  - --no-open
  - --trusted-host
  - dsh.internal.donaldsebleung.com
networkPolicy:
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              app.kubernetes.io/name: traefik
      ports:
        - protocol: TCP
          port: 3080

settings.yaml

llm-pi-ai:
  providers:
    vllm:
      apiKeyEnv: OPENAI_API_KEY
      api: openai-completions
      baseURL: http://vllm:8000/v1
      models:
        - id: qwen3-30b-a3b-thinking-2507

vllm.yaml

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: hf-cache
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 128Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: vllm
  name: vllm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vllm
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: vllm
    spec:
      containers:
        - command:
            - sh
            - -c
            - |
              rm -rvf /usr/local/python3.12.13/lib/python3.12/site-packages/triton*
              vllm serve Qwen/Qwen3-30B-A3B-Thinking-2507 \
                  --tensor-parallel-size 2 \
                  --dtype float16 \
                  --enforce-eager \
                  --max-model-len 262144 \
                  --tool-call-parser hermes \
                  --reasoning-parser deepseek_r1 \
                  --enable-auto-tool-choice \
                  --served-model-name qwen3-30b-a3b-thinking-2507
          env:
            - name: ASCEND_VISIBLE_DEVICES
              value: '0-1'
            - name: VLLM_PORT
              value: '8000'
          image: quay.io/ascend/vllm-ascend:v0.23.0-310p
          name: vllm-ascend
          ports:
            - containerPort: 8000
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: hf-cache
              mountPath: /root/.cache/huggingface
      runtimeClassName: ascend
      volumes:
        - name: hf-cache
          persistentVolumeClaim:
            claimName: hf-cache
status: {}
---
apiVersion: v1
kind: Service
metadata:
  name: vllm
spec:
  type: ClusterIP
  selector:
    app: vllm
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000