New configuration in Azure SDK for Java

azure-sdk-for-java-config-usage.md

Note: New APIs are still work in progress and need polishing

Please build azure-core and azure-data-appconfiguration from https://github.com/lmolkova/azure-sdk-for-java/tree/configuration-sdk/ branch

PR: Azure/azure-sdk-for-java#26420

Usage

// 1. Implement a ConfigurationSource
// 2. Build client-specific Configuration
Configuration appconfigSection = new ConfigurationBuilder(new FileSource("application.properties")) // your source goes here
                .root("az")  // prefix, for Spring it's spring.cloud.azure
                .section("appconfiguration") // relative path to client configuration
                .build();
                
// 3. Get may-be-client-specific credentials if needed
TokenCredential appconfigCredential = new DefaultAzureCredentialBuilder()
        // NOTE: no support for configuration in there yet
        .configuration(appconfigConfiguration) 
        .build();

// 4. Pass client configuration to builder
ConfigurationClientBuilder appConfigBuilder = new ConfigurationClientBuilder()
        .credential(appconfigCredential)
        .configuration(appconfigConfiguration);

1. Implement a ConfigurationSource interface.

   Source is only responsible for retrieval of properties from any location. Configuration will cache the values. Configuration updates after client is set up are not supported.

2. Build per-client Configuration using ConfigurationBuilder

   Builder needs a ConfigurationSource implemented in step 1. It also needs a path to Azure SDK properties root and path to client section. e.g. in following properties file, az is the prefix of everything.

   az.http.logging.application-id=logging-app-id
   az.http.logging.level=BODY_AND_HEADERS
   az.http.logging.pretty-print-body=true
   
   az.appconfiguration.endpoint=https://lmolkova-appconfig.azconfig.io

   The root and client sections paths are user/framework-defined. Client section can point to named client. SDK will look for client-spcific configurations there and fallback to global configuration for global properties.

3. Get credential.

   Credentails configuration is not yet implemented, but that's what we expect Spring integrations to do:

   • Decide which credentials to create (if any at all)
   • pass corresponding client section to credentials builder.

   If client section does not have credentials, Configuration will fallback to globals.

   Note: we expect users to either use global token credentials OR connection-string. If connection-string is defined along with credentials (global or local), it will be ignored.

   OK:

   az.appconfiguration.connection-string=...

   or

   az.appconfiguration.credential.client-id=...
   az.appconfiguration.credential.client-secret=...
   az.storage.connection-string=...

   or

   az.appconfiguration.credential.client-id=...
   az.appconfiguration.credential.client-secret=...
   az.credential.client-id=...

   NOT OK:

   az.credential.client-id=...
   az.appconfiguration.connection-string=... // ignored

4. Pass client configuration to builder.

Spring-specific use-cases

Conditional bean + named client.

• prefix az.appconfiguration.named-client is fully owned by Azure Spring team, it's opaque to Azure SDKs
• endpoint, connection-string are SDK configrations: they will be documented and published (TODO)

@Bean
ConfigurationBuilder configurationBuilder() {
    return new ConfigurationBuilder(new FileSource("application.yaml"))
    .root("spring.cloud.azure")
}

@Bean("appconfigNamedConfiguration")
Configuration appconfigConfigurationNamed(ConfigurationBuilder configBuilder) {
    return configBuilder.section("appconfiguration.named-client").build();
}

@Bean
@ConditionalOnAnyProperty(prefix = "spring.cloud.azure.appconfiguration.named-client", value = {"endpoint", "connection-string"})
ConfigurationClientBuilder configurationNamedClientBuilder(@Qualifier("appconfigNamedConfiguration") Configuration configuration) {
    TokenCredential tokenCredential = new DefaultAzureCredentialBuilder()
            .configuration(configuration)
            .build();

    return new ConfigurationClientBuilder()
            .credential(tokenCredential)
            .configuration(configuration);
}

Metadata json

• Phase 1: We will document all the properties manually, publish and share it with Spring.
• Phase 2: Come up with tooling allowing to automate generating documentation and potentially additonal spring metadata

Naive FileSource implementation

public static class FileSource implements ConfigurationSource {
    public final Map<String, String> properties;
    public FileSource(String fileName) {
        properties = CoreUtils.getProperties(fileName);
    }

    @Override
    public Iterable<String> getChildKeys(String prefix) {
        return properties.keySet().stream().filter(prop -> prop.startsWith(prefix + ".")).collect(Collectors.toUnmodifiableSet());
    }

    @Override
    public String getValue(String propertyName) {
        return properties.get(propertyName);
    }
}

Properties file example

az.client.application-id=client-app-id

az.http-client.application-id=http-app-id
az.http-client.proxy.http.host=localhost
az.http-client.proxy.http.port=8080

az.http.logging.application-id=logging-app-id
az.http.logging.level=BODY_AND_HEADERS
az.http.logging.pretty-print-body=true

az.appconfiguration.endpoint=https://lmolkova-appconfig.azconfig.io
az.appconfiguration.http-client.application-id=appconfig-http-app-id

az.appconfiguration.named-client.endpoint=https://lmolkova-appconfig.azconfig.io
az.appconfiguration.named-client.application-id=appconfig-http-app-id


az.appconfiguration.http-retry.mode=exponential
az.http-retry.exponential.max-retries=5
az.http-retry.exponential.base-delay=123
az.http-retry.exponential.max-delay=321
az.http-retry.retry-after-header=456
az.http-retry.retry-after-time-unit=MILLIS

Usage in Azure Client Libraries

• Core options support factory method with default value fromConfigration(config, defaultValue)
  • are global
  • have property names that describe path to this option like http-retry.exponential.max-retries. Path is relative and starts from either client configuration section or default section
  • naming conventions are documented and could be enforced with tooling
  • example
• Client options are local, builder support configuration(Configuration) method
  • which never updates any in-code properties
  • clients request properties with relative path starting from client section, e.g. connection-string
  • example
• Configuration is immutable

Open questions:

• error tolerance: we've been very tolerant to wrong configuration with env vars, I want to fail instead with new configuration options.
  • agreed: should keep old behavior for old properties, but for new ones we should fail on wrong format.
• Configuration immutability:
  • agreed to deprecate put, remove, clone, Configuration().
• Mixing global TokenCredentials and SAS-tokens/connection-strings is super-hard.

Implemented configurations

sourceType	name	type	defaultValue	Local?	env var/sys property name	Aliases	String format	Description
ProxyOptions	http-client.proxy.non-proxy-hosts	java.lang.String			"http.nonProxyHosts, NO_PROXY"
	http-client.proxy.create-unresolved	java.lang.Boolean	false				"""true"" is true	the rest is false"
	http-client.proxy.http.host	java.lang.String			http.proxyHost
	http-client.proxy.https.host	java.lang.String			https.proxyHost
	http-client.proxy.http.port	java.lang.Integer			http.proxyPort
	http-client.proxy.https.port	java.lang.Integer			https.proxyPort
	http-client.proxy.http.user	java.lang.String			http.proxyUser
	http-client.proxy.https.user	java.lang.String			https.proxyUser
	http-client.proxy.http.password	java.lang.String			http.proxyPassword
	http-client.proxy.https.password	java.lang.String			https.proxyPassword
HttpClientOptions	http-client.application-id	java.lang.String				client.application-id
	http-client.connect-timeout	java.time.Duration	10 sec		AZURE_REQUEST_CONNECT_TIMEOUT		Long millis
	http-client.write-timeout	java.time.Duration	60 sec		AZURE_REQUEST_WRITE_TIMEOUT		Long millis
	http-client.response-timeout	java.time.Duration	60 sec		AZURE_REQUEST_RESPONSE_TIMEOUT		Long millis
	http-client.read-timeout	java.time.Duration	60 sec		AZURE_REQUEST_READ_TIMEOUT		Long millis
	http-client.connection-idle-timeout	java.time.Duration	60 sec				Long millis
	http-client.maximum-connection-pool-size	java.lang.Integer	"impl concern: 500 for netty, 5 for okhttp"
RetryPolicy	http-retry.mode	java.lang.String	exponential				either 'fixed' or 'exponential'
	http-retry.retry-after-header	java.lang.String
	http-retry.retry-after-time-unit	TODO					TODO
ExponentialBackoff	http-retry.exponential.max-retries	java.lang.Integer	3		AZURE_REQUEST_RETRY_COUNT
	http-retry.exponential.base-delay	java.time.Duration	800 ms				Long millis
	http-retry.exponential.max-delay	java.time.Duration	8000 ms				Long millis
FixedDelay	http-retry.fixed.max-retries	java.time.Duration	required if mode is fixed				Long millis
	http-retry.fixed.delay	java.time.Duration	required if mode is fixed				Long millis
HttpLogOptions	http.logging.level	HttpLogDetailLevel			AZURE_HTTP_LOG_DETAIL_LEVEL		enum -> valueOf
	http.logging.pretty-print-body	java.lang.Boolean
	http.logging.headers	java.lang.String	list				comma separated set
ConfigurationClientBuilder	endpoint	java.lang.String		yes
	connection-string	java.lang.String		yes