Skip to content

Instantly share code, notes, and snippets.

@guitarrapc
Last active December 4, 2024 06:30
Show Gist options
  • Save guitarrapc/18737aaa2d4de8c86cc4c103e530bf85 to your computer and use it in GitHub Desktop.
Save guitarrapc/18737aaa2d4de8c86cc4c103e530bf85 to your computer and use it in GitHub Desktop.
Amazon Aurora DSQL connect from C# by Npgsql

HOW TO

  1. Create Amazon Aurora DSQL Cluster on Virgnia with Ohio (Linked region) and Oregon (Witness region).
  2. Replace required section of C# program to yours, and run C# Program where ever you like, LINQPAD, Console App and etc...
  3. Launch Cloud Shell on virginia https://us-east-1.console.aws.amazon.com/cloudshell/home?region=us-east-1#
$ psql --host=DSQL_CLUSTER_ENDPOINT_VIRGINIA --port=5432 --dbname=postgres --username=admin
Password for user admin: COPY FROM Connect > Authentication token > Copy
postgres=> select count(*) from owner;
 count 
-------
   100
(1 row)

postgres=> select * from owner limit 20;
                  id                  |      name       | city  |      telephone       
--------------------------------------+-----------------+-------+----------------------
 01939031-16d7-7008-8764-fafb1d7fba44 | John Doe 298250 | 61253 | a27248af-e4c8-4242-9
 01939031-16d7-704c-b4b6-02d13a49ca58 | John Doe 221756 | 53198 | d930062a-9208-4f73-b
 01939031-16d7-70c0-9883-ca6215bc9f6f | John Doe 409502 | 38924 | a14f5003-b529-4b5c-9
 01939031-16d7-70e3-a672-5cfcdcd14101 | John Doe 223371 | 1503  | 59dac43e-8656-42b3-b
 01939031-16d7-7145-ba97-466f821bb673 | John Doe 677661 | 84179 | f6eeefed-b171-4b91-9
 01939031-16d7-716b-86ba-ed4abadfead5 | John Doe 356230 | 21694 | 433859fa-fe26-40c0-8
 01939031-16d7-71b1-b49a-afcfd25e39cc | John Doe 383846 | 98768 | 0c430216-e576-4131-9
 01939031-16d7-71e8-8a7e-091dfc5aff2f | John Doe 737747 | 51639 | 1fdf086c-0284-463a-a
 01939031-16d7-722a-b7f1-55e1d74ee645 | John Doe 513463 | 98008 | ede4d901-0e04-4427-b
 01939031-16d7-727c-b2a9-c389cf10da68 | John Doe 851841 | 25854 | 3db86316-4fa3-404a-a
 01939031-16d7-72d6-aae8-130f294963c2 | John Doe 781576 | 5740  | 0131772f-97b0-4d79-b
 01939031-16d7-7327-b606-d4126b233223 | John Doe 407915 | 52801 | 78220314-c920-4ddc-9
 01939031-16d7-7358-bdb8-0b43abd96d14 | John Doe 274231 | 67683 | 2c435b25-fa1a-4e86-b
 01939031-16d7-7370-9e3e-36c3c6ad5af5 | John Doe 439645 | 54563 | 165e26b1-4d6e-499c-a
 01939031-16d7-73b6-a177-84567ecd1035 | John Doe 151434 | 79293 | cf35f5e3-3883-4e5a-8
 01939031-16d7-740c-8025-87b8c81fdd41 | John Doe 283634 | 17129 | 22bc4d34-6b65-4f20-9
 01939031-16d7-743d-a63d-5ff19be9c053 | John Doe 766444 | 66751 | 84465da2-f92a-47d1-b
 01939031-16d7-7480-a512-e98224f4369a | John Doe 503196 | 28088 | f6c2e9b3-98f4-4012-a
 01939031-16d7-7488-a273-5c307f3d565f | John Doe 339391 | 64527 | 2f1701d6-e543-412b-b
 01939031-16d7-74b0-8e91-f5456622bebd | John Doe 575847 | 24751 | a5e9abc8-5e85-49c0-a
(20 rows)

postgres=> 
  1. Launch Cloud Shell on Ohio https://us-east-2.console.aws.amazon.com/cloudshell/home?region=us-east-2# and confirm data exists as same as virginia.
$ psql --host=DSQL_CLUSTER_ENDPOINT_OHIO --port=5432 --dbname=postgres --username=admin
Password for user admin: COPY FROM Connect > Authentication token > Copy
postgres=> select count(*) from owner;
postgres=> select count(*) from owner;
 count 
-------
   100
(1 row)
postgres=> select * from owner limit 20;
                  id                  |      name       | city  |      telephone       
--------------------------------------+-----------------+-------+----------------------
 01939031-16d7-7008-8764-fafb1d7fba44 | John Doe 298250 | 61253 | a27248af-e4c8-4242-9
 01939031-16d7-704c-b4b6-02d13a49ca58 | John Doe 221756 | 53198 | d930062a-9208-4f73-b
 01939031-16d7-70c0-9883-ca6215bc9f6f | John Doe 409502 | 38924 | a14f5003-b529-4b5c-9
 01939031-16d7-70e3-a672-5cfcdcd14101 | John Doe 223371 | 1503  | 59dac43e-8656-42b3-b
 01939031-16d7-7145-ba97-466f821bb673 | John Doe 677661 | 84179 | f6eeefed-b171-4b91-9
 01939031-16d7-716b-86ba-ed4abadfead5 | John Doe 356230 | 21694 | 433859fa-fe26-40c0-8
 01939031-16d7-71b1-b49a-afcfd25e39cc | John Doe 383846 | 98768 | 0c430216-e576-4131-9
 01939031-16d7-71e8-8a7e-091dfc5aff2f | John Doe 737747 | 51639 | 1fdf086c-0284-463a-a
 01939031-16d7-722a-b7f1-55e1d74ee645 | John Doe 513463 | 98008 | ede4d901-0e04-4427-b
 01939031-16d7-727c-b2a9-c389cf10da68 | John Doe 851841 | 25854 | 3db86316-4fa3-404a-a
 01939031-16d7-72d6-aae8-130f294963c2 | John Doe 781576 | 5740  | 0131772f-97b0-4d79-b
 01939031-16d7-7327-b606-d4126b233223 | John Doe 407915 | 52801 | 78220314-c920-4ddc-9
 01939031-16d7-7358-bdb8-0b43abd96d14 | John Doe 274231 | 67683 | 2c435b25-fa1a-4e86-b
 01939031-16d7-7370-9e3e-36c3c6ad5af5 | John Doe 439645 | 54563 | 165e26b1-4d6e-499c-a
 01939031-16d7-73b6-a177-84567ecd1035 | John Doe 151434 | 79293 | cf35f5e3-3883-4e5a-8
 01939031-16d7-740c-8025-87b8c81fdd41 | John Doe 283634 | 17129 | 22bc4d34-6b65-4f20-9
 01939031-16d7-743d-a63d-5ff19be9c053 | John Doe 766444 | 66751 | 84465da2-f92a-47d1-b
 01939031-16d7-7480-a512-e98224f4369a | John Doe 503196 | 28088 | f6c2e9b3-98f4-4012-a
 01939031-16d7-7488-a273-5c307f3d565f | John Doe 339391 | 64527 | 2f1701d6-e543-412b-b
 01939031-16d7-74b0-8e91-f5456622bebd | John Doe 575847 | 24751 | a5e9abc8-5e85-49c0-a
(20 rows)

postgres=> 

FAQ

Q. Which sample code did you use?

Based on AWS Amazon Aurora DDSQL Doc.

Q. Why are you specifying SslMode.Prefer instead of SslMode.VerifyFull

Because SslMode.VerifyFull shows following error related to some TLS issue.

Exception while performing SSL handshake
The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
  at Npgsql.Internal.NpgsqlConnector.NegotiateEncryption(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.Internal.NpgsqlConnector.RawOpen(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore|214_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.PoolingDataSource.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.PoolingDataSource.<Get>g__RentAsync|33_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlConnection.<Open>g__OpenAsync|42_0(Boolean async, CancellationToken cancellationToken)

Q. What is current limitation of Amazon Aurora DSQL

see Amazon Docs Known issues in Amazon Aurora DSQL

using System;
using System.Linq;
using System.Text;
using Amazon;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Amazon.Runtime.Internal;
using Amazon.Runtime.Internal.Auth;
using Amazon.Runtime.Internal.Util;
using Npgsql;
// required replacement
var profile = "YOUR_AWS_PROFILE"; // for local only YOUR_AWS_PROFILE
var clusterEndpoint = "DSQL_CLUSTER_ENDPOINT"; // DSQL CLUSTER ENDPOINT shown on AWS Console > Aurora DSQL
var region = RegionEndpoint.USEast1; // DSQL CLUSTER REGION
// fixed for most cases
var username = "admin"; // fixed if no user created
var password = TokenGenerator.GenerateAuthToken(clusterEndpoint, region, profile);
var database = "postgres"; // fixed if no schema created
//var sslMode = SslMode.VerifyFull;
var sslMode = SslMode.Prefer; // HACK: Current DSQL seems has problem with ssl verify
var connectionString = $"Host={clusterEndpoint};Username={username};Password={password};Database={database};Port=5432;SSLMode={sslMode};";
// gen sample data
var records = Enumerable
.Range(0, 100)
.Select(x => new Owner(Guid.CreateVersion7().ToString(), $"John Doe {Random.Shared.Next(100000, 999999)}", $"{Random.Shared.Next(1000, 99999)}", Guid.NewGuid().ToString().Substring(0, 20)))
.ToArray();
// connect
using var conn = new NpgsqlConnection(connectionString);
await conn.OpenAsync();
// table
using var create = new NpgsqlCommand("CREATE TABLE IF NOT EXISTS owner (id UUID PRIMARY KEY, name VARCHAR(30) NOT NULL, city VARCHAR(80) NOT NULL, telephone VARCHAR(20))", conn);
create.ExecuteNonQuery();
// insert
var sb = new StringBuilder("INSERT INTO owner (id, name, city, telephone) VALUES ");
for (var i = 0; i < records.Length; i++)
{
if (i != 0) sb.Append(',');
sb.Append("('").Append(records[i].Id).Append("'");
sb.Append(", '").Append(records[i].Name).Append("'");
sb.Append(", '").Append(records[i].City).Append("'");
sb.Append(", '").Append(records[i].Telephone).Append("'");
sb.Append(')');
}
var insertSQL = sb.ToString();
using var insert = new NpgsqlCommand(connection: conn, cmdText: insertSQL);
insert.ExecuteNonQuery();
public record Owner(string Id, string Name, string City, string Telephone);
public static class TokenGenerator
{
/// <summary>AWS Environment, use IAM Role Federated Credentials</summary>
public static string GenerateAuthToken(string? clusterEndpoint, Amazon.RegionEndpoint region)
{
return GenerateAuthToken(clusterEndpoint, region, FallbackCredentialsFactory.GetCredentials());
}
/// <summary>Local Environment, use Local AWS Profile</summary>
public static string GenerateAuthToken(string? clusterEndpoint, Amazon.RegionEndpoint region, string profile)
{
var chain = new CredentialProfileStoreChain();
AWSCredentials awsCredentials;
if (!chain.TryGetAWSCredentials(profile, out awsCredentials))
throw new Exception("AWS Credentials not found for specified profile");
return GenerateAuthToken(clusterEndpoint, region, awsCredentials);
}
public static string GenerateAuthToken(string? hostname, Amazon.RegionEndpoint region, AWSCredentials awsCredentials)
{
string accessKey = awsCredentials.GetCredentials().AccessKey;
string secretKey = awsCredentials.GetCredentials().SecretKey;
string token = awsCredentials.GetCredentials().Token;
const string DsqlServiceName = "dsql";
const string HTTPGet = "GET";
const string HTTPS = "https";
const string URISchemeDelimiter = "://";
const string ActionKey = "Action";
const string ActionValue = "DbConnectAdmin";
const string XAmzSecurityToken = "X-Amz-Security-Token";
ImmutableCredentials immutableCredentials = new ImmutableCredentials(accessKey, secretKey, token) ?? throw new ArgumentNullException("immutableCredentials");
ArgumentNullException.ThrowIfNull(region);
hostname = hostname?.Trim();
if (string.IsNullOrEmpty(hostname))
throw new ArgumentException("Hostname must not be null or empty.");
GenerateDsqlAuthTokenRequest authTokenRequest = new GenerateDsqlAuthTokenRequest();
IRequest request = new DefaultRequest(authTokenRequest, DsqlServiceName)
{
UseQueryString = true,
HttpMethod = HTTPGet
};
request.Parameters.Add(ActionKey, ActionValue);
request.Endpoint = new UriBuilder(HTTPS, hostname).Uri;
if (immutableCredentials.UseToken)
{
request.Parameters[XAmzSecurityToken] = immutableCredentials.Token;
}
var signingResult = AWS4PreSignedUrlSigner.SignRequest(request, null, new RequestMetrics(), immutableCredentials.AccessKey,
immutableCredentials.SecretKey, DsqlServiceName, region.SystemName);
var authorization = "&" + signingResult.ForQueryParameters;
var url = AmazonServiceClient.ComposeUrl(request);
// remove the https:// and append the authorization
return url.AbsoluteUri[(HTTPS.Length + URISchemeDelimiter.Length)..] + authorization;
}
private class GenerateDsqlAuthTokenRequest : AmazonWebServiceRequest
{
public GenerateDsqlAuthTokenRequest()
{
((IAmazonWebServiceRequest)this).SignatureVersion = SignatureVersion.SigV4;
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment