ACI-agent-provisioning-messages

Some of the ACI agent provisioning messages from the Docker log of ci.jenkins.io:

2021-06-21 19:03:38.604+0000 [id=67]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-w1h2k
java.util.concurrent.TimeoutException: Deployment timeout
        at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:117)
        at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:128)
        at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:113)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2021-06-21 19:03:38.605+0000 [id=67]    WARNING c.m.j.c.aci.AciCloud#canProvision: Cannot provision: template for label maven is not available now, because it failed to provision last time.

After that, there were still successful ACI agent allocations (like agent aci-maven-h42nn).  For example:

2021-06-21 19:03:48.594+0000 [id=4966]  INFO    c.m.j.c.aci.AciCloud#lambda$provision$1: Add ACI node: aci-maven-h42nn
2021-06-21 19:04:29.275+0000 [id=4966]  INFO    c.m.j.c.aci.AciCloud#waitToOnline: Waiting agent aci-maven-h42nn to online
2021-06-21 19:04:38.536+0000 [id=67]    INFO    hudson.slaves.NodeProvisioner#lambda$update$6: aci-maven-h42nn provisioning successfully completed. We have now 40 computer(s)
2021-06-21 19:10:19.723+0000 [id=70]    INFO    c.m.j.c.s.ContainerOnceRetentionStrategy#done: terminating aci-maven-h42nn since PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=Plugins/graphql-server-plugin/PR-328 ..
2021-06-21 19:10:19.728+0000 [id=5553]  INFO    j.s.DefaultJnlpSlaveReceiver#channelClosed: Computer.threadPoolForRemoting [#563] for aci-maven-h42nn terminated: java.nio.channels.ClosedChannelException
2021-06-21 19:10:21.552+0000 [id=5662]  INFO    c.m.j.c.aci.AciService#deleteAciContainerGroup: Delete ACI Container Group: aci-maven-h42nn successfully

However, there were several deployment timeout failures interspered, including:

2021-06-21 19:03:38.604+0000 [id=67]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-w1h2k
2021-06-21 19:13:58.553+0000 [id=66]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-hg5zj
2021-06-21 20:16:18.582+0000 [id=29]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-mzsq2
2021-06-21 20:26:48.599+0000 [id=65]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-k7kw3


A new style of failure message appears at 17:12 that reads:

2021-06-21 17:12:45.463+0000 [id=916]   WARNING c.m.j.c.aci.AciCleanTask#cleanDeployments: AzureAciCleanUpTask: cleanDeployments: Failed to get/delete deployment
Also:   java.lang.Exception: #block terminated with an error
2021-06-21 17:12:45.627+0000 [id=916]   WARNING c.m.j.c.aci.AciCleanTask#cleanDeployments: AzureAciCleanUpTask: cleanDeployments: Failed to get/delete deployment
Also:   java.lang.Exception: #block terminated with an error
2021-06-21 17:12:45.817+0000 [id=916]   WARNING c.m.j.c.aci.AciCleanTask#cleanDeployments: AzureAciCleanUpTask: cleanDeployments: Failed to get/delete deployment
Also:   java.lang.Exception: #block terminated with an error
2021-06-22 04:15:47.776+0000 [id=39290] WARNING c.m.j.c.aci.AciCleanTask#cleanDeployments: AzureAciCleanUpTask: cleanDeployments: Failed to get/delete deployment
Also:   java.lang.Exception: #block terminated with an error

Each of those failures includes text like the following:1G

com.azure.core.management.exception.ManagementException: Status code 404, "{"error":{"code":"DeploymentNotFound","message":"Deployment 'aci-maven-9qfnkg3z' could not be found."}}": Deployment 'aci-maven-9qfnkg3z' could not be found.

Most recent failures look like this:

2021-06-22 04:27:31.887+0000 [id=40061] WARNING c.m.j.c.aci.AciCloud#canProvision: Cannot provision: template for label maven-11 is not available now, because it failed to provision last time.
2021-06-22 04:27:31.888+0000 [id=40061] WARNING c.m.j.c.aci.AciCloud#canProvision: Cannot provision: template for label maven-windows is not available now, because it failed to provision last time.
2021-06-22 04:27:37.851+0000 [id=29]    WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven-msxbj
java.lang.Exception: Deployment aci-maven-tww6tm7d status: Failed
        at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:126)
        at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:128)
        at com.microsoft.jenkins.containeragents.aci.AciCloud.lambda$provision$1(AciCloud.java:113)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)