Joining encrypted columsn using postgres pgcrypto
The following sample will create two tables which includes two sets of usernames and the AES encrypted form of their their fave animal.
Each table will use a different encryption key:
collaborator1 will use encryption key of 1234collaborator2 will use encryption key of 6789
Sample that generates a BQ ML model where the data is AEAD encrypted
A variaton of Create machine learning models in BigQuery ML
in which two different parties have encrypted a part of the source training dataset with their own keys.
The aead keys maybe released to an secure workstation where neither party has access to but trusts the code its running.
Te secure workstation that has both collaborator's keys and the encrypted dataset will construct a BQ ML model, evaluate it and use it to make a prediction
| package main | |
| /* | |
| see | |
| https://blog.salrashid.dev/articles/2022/bq_client_side_aead/#untrusted-multiparty-compute | |
| two tables in two different datasets encrypted with their own keys |
$ go run client/grpc_client.go --host dns:///grpc-ig-3kb7:50051
2022/10/25 02:42:52 INFO: [core] [Channel #1] Channel created
2022/10/25 02:42:52 INFO: [core] [Channel #1] original dial target is: "dns:///grpc-ig-3kb7:50051"
2022/10/25 02:42:52 INFO: [core] [Channel #1] parsed dial target is: {Scheme:dns Authority: Endpoint:grpc-ig-3kb7:50051 URL:{Scheme:dns Opaque: User: Host: Path:/grpc-ig-3kb7:50051 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}
2022/10/25 02:42:52 INFO: [core] [Channel #1] Channel authority set to "grpc.domain.com"
2022/10/25 02:42:52 INFO: [core] [Channel #1] Resolver state updated: {
"Addresses": [
{
root@xds-client:~/td_grpc# export GRPC_XDS_BOOTSTRAP=`pwd`/xds_bootstrap.json
root@xds-client:~/td_grpc# go run client/grpc_client.go --host xds:///grpc.domain.com:50051
2022/10/25 02:45:51 INFO: [core] [Channel #1] Channel created
2022/10/25 02:45:51 INFO: [core] [Channel #1] original dial target is: "xds:///grpc.domain.com:50051"
2022/10/25 02:45:51 INFO: [core] [Channel #1] parsed dial target is: {Scheme:xds Authority: Endpoint:grpc.domain.com:50051 URL:{Scheme:xds Opaque: User: Host: Path:/grpc.domain.com:50051 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}
2022/10/25 02:45:51 INFO: [core] [Channel #1] Channel authority set to "grpc.domain.com"
2022/10/25 02:45:51 INFO: [xds] [xds-resolver 0xc00036e7e0] Creating resolver for target: {Scheme:xds Authority: Endpoint:grpc.domain.com:50051 URL:{Scheme:xds Opaque: User: Host: Path:/grpc.domain.com:50051 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}
2022/10/25 02:45:51 INFO: [xds] [xds-bootstrap
| node: | |
| cluster: service_greeter | |
| id: test-id | |
| admin: | |
| access_log_path: /dev/null | |
| address: | |
| socket_address: | |
| address: 0.0.0.0 | |
| port_value: 9000 |
| package main | |
| import ( | |
| "context" | |
| "crypto/x509" | |
| "encoding/pem" | |
| "fmt" | |
| "io/ioutil" | |
| "log" | |
| "os" |
| export PROJECT_ID=`gcloud config get-value core/project` | |
| export BUCKET_NAME=$PROJECT_ID-enctest | |
| gsutil mb gs://$BUCKET_NAME | |
| # create a sample file | |
| openssl rand --base64 1000000 > secrets.txt | |
| sha256sum secrets.txt | |
| # generate kek and dek | |
| openssl genrsa -out kek.key 2048 |
| export PROJECT_ID=`gcloud config get-value core/project` | |
| export BUCKET_NAME=$PROJECT_ID-enctest | |
| gsutil mb gs://$BUCKET_NAME | |
| # create a sample file | |
| openssl rand --base64 1000000 > secrets.txt | |
| sha256sum secrets.txt | |
| # generate kek and dek | |
| openssl rand 32 > kek.key |
| #!/usr/bin/bash | |
| # get gcp service account OIDC token using curl, openssl, jq | |
| # GCP Service Account authentication with bash curl openssl jq | |
| # https://developers.google.com/identity/protocols/oauth2/service-account#authorizingrequests | |
| # code below credited to | |
| # https://gist.github.com/LindaLawton/cff75182aac5fa42930a09f58b63a309?permalink_comment_id=4232119#gistcomment-4232119 | |
| # usage: |