grpc_qa_draft.md

QA Test Plan

Summary

We have converted several internal services (sourcegraph-frontend-internal's zoekt configuration APIs, symbols, searcher, repo-updater, and gitserver) to use gRPC for communication instead of REST APIs.

This change improves performance and enables stronger typing and streaming APIs. It also paves the road for subsequent internal service changes to use gRPC and its large ecosystem.

Test Plan

Functional testing

• Test all endpoints of all converted services manually to ensure functionality is preserved
• Monitor logs for errors/warnings during regular use of Sourcegraph to catch any issues
• Monitor memory/CPU usage of services to ensure no major regressions

Integration testing

• Deploy a full Sourcegraph instance with only the gRPC-converted services and validate that core functionality (search, symbol resolve, etc) works as expected
• Flip gRPC on/off via feature flag and ensure results are consistent
• Upgrade/downgrade Sourcegraph with gRPC on/off and ensure no issues

Performance testing

• Compare API response times (ingest, search, etc) of gRPC vs REST and track improvements
• Test performance of large repositories and/or large result sets (many search results/symbols) to ensure no major regressions

Security testing

Note that we not built any mTLS support for gRPC in this release. For now, our gRPC communication is unencrypted in the same way that our existing REST communication is unencrypted.

Other than that, the following is worth considering:

• Ensure existing auth enforcement is preserved (search access control, actor logic, etc)
• Search for common gRPC vulnerabilities (DoS, etc) and ensure changes are not vulnerable

Usability testing

• No user-facing changes, so no usability testing required

Compatibility testing

• Deploy to all infra/cloud environments (dev clusters, etc) and ensure functionality is preserved
• Ensure services start/stay running and no errors are logged

QA Checklist

1. Have you made any infra related changes to environment variables, new services, or deployment methods that could affect customers?

   • No infra changes required (multiplexing gRPC on existing HTTP ports)

   For starship, we're in a transitional period where not all services / methods have been converted to use gRPC. During this time, the gRPC servers are multiplexed on the same ports as the existing HTTP servers so that no Kubernetes service configuration (or other firewall) changes are required.

   There is one exception to this. If users are using a firewall (ex: a service mesh) that's configured to block HTTP/2 traffic or any traffic that's not plain text, they might have to add an exception to their firewall

2. Which environments have the changes been tested on?

   • sgdev clusters (k8s, etc)
   • sourcegraph.com
   • (other environments if applicable)

3. Experimental features have been marked and behind a feature flag?

   • Yes, gRPC can be enabled/disabled via feature flag

     • For most services, gRPC communication can be enabled via:

       {
         "experimentalFeatures": {
           "enableGRPC": true
         }
       }

     • For indexed-search (zoekt), an environment variable needs to be set to enable gRPC (since it doesn't obtain its configuration from Sourcegraph's database)

       • For Kubernetes, that can be done by setting the GRPC_ENABLED=1 environment variable in the base/indexed-search/indexed-search.StatefulSet.yaml configuration YAML. Other deployment targets will need to update that environment variable in a similar manner.

         diff --git a/base/indexed-search/indexed-search.StatefulSet.yaml b/base/indexed-search/indexed-search.StatefulSet.yaml
         index 370b24db8a1..d3fa36354aa 100644
         --- a/base/indexed-search/indexed-search.StatefulSet.yaml
         +++ b/base/indexed-search/indexed-search.StatefulSet.yaml
         @@ -109,8 +109,10 @@ spec:
                     value: "10000"
                     - name: GOOGLE_CLOUD_PROFILER_ENABLED
                       value: "1"
         +           - name: GRPC_ENABLED
         +             value: "1"
                 # Upstream the image is "index.docker.io/sourcegraph/search-indexer"
                 image: sourcegraph/zoekt-indexserver:0.0.0-20230308131753-939eb52f3486
                 terminationMessagePolicy: FallbackToLogsOnError
                 ports:
                     - containerPort: 6072

4. Completed entry to release post.

   • Yes (will complete when testing is done)
   • No