Knowledge graph and OPEA

readme.md

Knowledge graph and OPEA

Test

• Configure

  sudo apt update -y
  sudo apt install python3-pip -y
  pip3 install opea-comps --break-system-packages
  git clone https://github.com/opea-project/GenAIComps
  cd GenAIComps/comps/dataprep/neo4j/langchain
  pip3 install -r requirements.txt --break-system-packages
  sudo apt-get install libtesseract-dev -y
  sudo apt-get install poppler-utils -y
  

• Install Docker

  # Add Docker's official GPG key:
  sudo apt-get -y update
  sudo apt-get -y install ca-certificates curl
  sudo install -m 0755 -d /etc/apt/keyrings
  sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
  sudo chmod a+r /etc/apt/keyrings/docker.asc
  
  # Add the repository to Apt sources:
  echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  sudo apt-get -y update
  sudo apt-get -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
  

• Start Neo4J

  sudo docker run \
    -p 7474:7474 -p 7687:7687 \
    -v $PWD/data:/data -v $PWD/plugins:/plugins \
    --name neo4j-apoc \
    -d \
    -e NEO4J_AUTH=neo4j/password \
    -e NEO4J_PLUGINS=\[\"apoc\"\]  \
    neo4j:latest
  

• Set up environment vars:

  #export PYTHONPATH=/usr/bin/python3
  export NEO4J_URI=http://localhost:7687
  export NEO4J_USERNAME=neo4j
  export NEO4J_PASSWORD=password
  

• Data prep:

  python3 prepare_doc_neo4j.py
  

OLD

• Replace HuggingFace API token below and set the environment variables:

  export host_ip="localhost"
  export NEO4J_ENDPOINT="neo4j://${your_ip}:7687"
  export NEO4J_USERNAME="neo4j"
  export NEO4J_PASSWORD="neo4j"
  export HUGGINGFACEHUB_API_TOKEN=${your_huggingface_api_token}
  export LLM_ENDPOINT="http://${your_ip}:8080"
  export LLM_MODEL="meta-llama/Llama-2-7b-hf"
  export AGENT_LLM="HuggingFaceH4/zephyr-7b-beta"  
  

• Run Neo4J:

  docker run --rm \
    --publish=7474:7474 --publish=7687:7687 \
    --env NEO4J_AUTH=$NEO4J_USER/$NEO4J_PASSWORD \
    --volume=$PWD/neo4j_data:"/data" \
    --env='NEO4JLABS_PLUGINS=["apoc"]' \
    -d \
    neo4j
  

Per test in our environment, there might be several steps missing to run the prepare_doc_neo4j service

1. libGL is required to run the service which is included in the following packages
   sudo apt-get install ffmpeg libsm6 libxext6  -y

2. With respect to the error No module named 'comps.dataprep', the PYTHONPATH needs to be extended to include the path to GenAIComps.
   For instance, in my environment, the GenAIComps is checked out at /home/sdp/opea/GenAIComps. The following environment variable is required:
   export PYTHONPATH=/home/sdp/opea/GenAIComps:$PYTHONPATH

3. According to the example , the NEO4J_URI might be like this:
   export NEO4J_URI=bolt://localhost:7687
   To upload a document to the prepare_doc_neo4j service, either an OPENAPI_KEY or a local TGI service is required. We are still in the process of validating the service.