OPEA ChatQnA on IBM Cloud

readme.md

OPEA ChatQnA on IBM Cloud

Setup Virtual Server

• Location:
• Image and profile:
• Storage:
• Summary:
• Attach floating IP:
• Login to VPC instance:

  ssh -i ibm-opea_rsa [email protected]
  

Configuration

• Install Docker:

  # Add Docker's official GPG key:
  sudo apt-get -y update
  sudo apt-get -y install ca-certificates curl
  sudo install -m 0755 -d /etc/apt/keyrings
  sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
  sudo chmod a+r /etc/apt/keyrings/docker.asc
  
  # Add the repository to Apt sources:
  echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  sudo apt-get -y update
  sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
  

• Replace HuggingFace API token and copy contents in a file named .env:

  HUGGINGFACEHUB_API_TOKEN="Your_Huggingface_API_Token"
  host_ip=localhost #private IP address of the host
  no_proxy=${host_ip}
  http_proxy=
  https_proxy=
  EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
  RERANK_MODEL_ID="BAAI/bge-reranker-base"
  LLM_MODEL_ID="Intel/neural-chat-7b-v3-3"
  TEI_EMBEDDING_ENDPOINT="http://${host_ip}:6006"
  REDIS_URL="redis://${host_ip}:6379"
  INDEX_NAME="rag-redis"
  REDIS_HOST=${host_ip}
  MEGA_SERVICE_HOST_IP=${host_ip}
  EMBEDDING_SERVER_HOST_IP=${host_ip}
  RETRIEVER_SERVICE_HOST_IP=${host_ip}
  RERANK_SERVER_HOST_IP=${host_ip}
  LLM_SERVER_HOST_IP=${host_ip}
  BACKEND_SERVICE_ENDPOINT="http://${host_ip}:8888/v1/chatqna"
  DATAPREP_SERVICE_ENDPOINT="http://${host_ip}:6007/v1/dataprep"
  DATAPREP_GET_FILE_ENDPOINT="http://${host_ip}:6007/v1/dataprep/get_file"
  DATAPREP_DELETE_FILE_ENDPOINT="http://${host_ip}:6007/v1/dataprep/delete_file"
  FRONTEND_SERVICE_IP=${host_ip}
  FRONTEND_SERVICE_PORT=5173
  BACKEND_SERVICE_NAME=chatqna
  BACKEND_SERVICE_IP=${host_ip}
  BACKEND_SERVICE_PORT=8888
  

• Download Docker Compose file:

  curl -O https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/ChatQnA/docker_compose/intel/cpu/xeon/compose.yaml
  

• Start the application:

  sudo docker compose -f compose.yaml up -d
  

• Verify the list of containers:

  ubuntu@opea-demo:~$ sudo docker container ls
  CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED              STATUS          PORTS                                                                                  NAMES
  89bd9c123a25   opea/nginx:latest                                                     "/docker-entrypoint.…"   31 seconds ago       Up 30 seconds   0.0.0.0:80->80/tcp, :::80->80/tcp                                                      chaqna-xeon-nginx-server
  85be07943a99   opea/chatqna-ui:latest                                                "docker-entrypoint.s…"   32 seconds ago       Up 30 seconds   0.0.0.0:5173->5173/tcp, :::5173->5173/tcp                                              chatqna-xeon-ui-server
  ba68b96945c9   opea/chatqna:latest                                                   "python chatqna.py"      32 seconds ago       Up 30 seconds   0.0.0.0:8888->8888/tcp, :::8888->8888/tcp                                              chatqna-xeon-backend-server
  515c467e0304   opea/dataprep-redis:latest                                            "python prepare_doc_…"   32 seconds ago       Up 31 seconds   0.0.0.0:6007->6007/tcp, :::6007->6007/tcp                                              dataprep-redis-server
  34eeba201c60   opea/retriever-redis:latest                                           "python retriever_re…"   32 seconds ago       Up 31 seconds   0.0.0.0:7000->7000/tcp, :::7000->7000/tcp                                              retriever-redis-server
  d151f2d48191   ghcr.io/huggingface/text-generation-inference:sha-e4201f4-intel-cpu   "text-generation-lau…"   About a minute ago   Up 31 seconds   0.0.0.0:9009->80/tcp, [::]:9009->80/tcp                                                tgi-service
  4f958a5789d4   redis/redis-stack:7.2.0-v9                                            "/entrypoint.sh"         About a minute ago   Up 31 seconds   0.0.0.0:6379->6379/tcp, :::6379->6379/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp   redis-vector-db
  5d9a887f54e2   ghcr.io/huggingface/text-embeddings-inference:cpu-1.5                 "text-embeddings-rou…"   About a minute ago   Up 31 seconds   0.0.0.0:8808->80/tcp, [::]:8808->80/tcp                                                tei-reranking-server
  979a252cd999   ghcr.io/huggingface/text-embeddings-inference:cpu-1.5                 "text-embeddings-rou…"   About a minute ago   Up 31 seconds   0.0.0.0:6006->80/tcp, [::]:6006->80/tcp                                                tei-embedding-server
  

Validate Services

• Export host_ip environment variable:

  export host_ip=localhost
  

• TEI Embedding service:

  curl ${host_ip}:6006/embed \
    -X POST \
    -d '{"inputs":"What is Deep Learning?"}' \
    -H 'Content-Type: application/json'
  

• Retriever microservice:

  export your_embedding=$(python3 -c "import random; embedding = [random.uniform(-1, 1) for _ in range(768)]; print(embedding)")
  curl http://${host_ip}:7000/v1/retrieval \
    -X POST \
    -d "{\"text\":\"test\",\"embedding\":${your_embedding}}" \
    -H 'Content-Type: application/json'
  

  This is giving the error: opea-project/GenAIExamples#949

Test ChatQnA