Running Harmony API tests - 2024-08-05

Running Harmony API tests - 2024-08-05.md

• Ubuntu 24.04 LTS
• Python 3.10.14

Running Tika

wget https://archive.apache.org/dist/tika/2.3.0/tika-server-standard-2.3.0.jar
java -jar tika-server-standard-2.3.0.jar 

Setting up the environment

git submodule update --init
pip install -r requirements.txt

export GOOGLE_APPLICATION_CREDENTIALS="service-account-file.json"
export AZURE_OPENAI_ENDPOINT="..."
export AZURE_OPENAI_API_KEY="..."
export OPENAI_API_KEY="..."

(I added the credentials copied/downloaded from Google, Azure, and Open AI.)

Running Uvicorn

uvicorn main:app_fastapi --host 0.0.0.0 --port 8000

Pydantic error - with Google application credentials

pydantic.env_settings.SettingsError: error parsing env var "google_application_credentials" 

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

For reference, here is the structure of my `service-account-file.json`

{
  "type": "service_account",
  "project_id": "...",
  "private_key_id": "...",
  "private_key": "...",
  "client_email": "...",
  "client_id": "...",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "...",
  "universe_domain": "googleapis.com"
}

Re-run Uvicorn

Without Google application credentials

unset GOOGLE_APPLICATION_CREDENTIALS

uvicorn main:app_fastapi --host 0.0.0.0 --port 8000

Run the tests

1. Remote tests — OK

python -m unittest discover tests/remote_tests

OK

2. Local tests — Failed

python -m unittest discover tests/local_tests

Ran 18 tests in 0.647s

FAILED (errors=7)

Example of a failed test:

• test_azure_openai_same_as_vanilla_openai
  • vanilla_openai_response.json()={'detail': 'Could not process request because the model is not available.'}

3. Staging tests — Failed

python -m unittest discover tests/staging_tests

Ran 18 tests in 5.174s

FAILED (failures=1, errors=2)

4. Selenium tests — Couldn't run; I will try a different setup

pip install -r tests/requirements.txt  
  
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo apt-get update && sudo apt-get upgrade  
sudo apt-get install -y ./google-chrome-stable_current_amd64.deb  
google-chrome --version  

Google Chrome 127.0.6533.88

python -m unittest discover tests/selenium_tests/  

Error:

OSError: [Errno 8] Exec format error: '/home/imt/.wdm/drivers/chromedriver/linux64/127.0.6533.88/chromedriver-linux64/THIRD_PARTY_NOTICES.chromedriver'