nginx access.log
10.3.0.232 - - [25/Jul/2016:12:38:53 +1000] "POST /auth/reset HTTP/1.1" 502 172 "https://puppet.corporate.local/auth/reset?token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJhZG1pbiIsInN1YiI6IjFmMzQ1MjI5LTIxZjktNDNjYy04YmNmLWVmZGU4N2U0Y2M5MyIsImV4cCI6MTQ2OTUwMDYzNywiaWF0IjoxNDY5NDE0MjM3fQ.SuOA-_c3f3otyDF-5LfmED2JioYo3BH5P3Jz_4r9EF4Z80tuz-pw8Z9eidJ8X7myiL4VoQF-sOQMs89ysxD7vh35ifgLnJnV7-2u3K6Ep1FdHHas_yRjdYSdEELJgHByNyqnSYQa5511f9wDYMZN_wyhbqCuXzbdUELjvVj27-sS0jEh2RZMi9Znp68473gwQGxziQkp_bWU1IJ9YJvPEjm29EQSg4Lm1l0b3FcjjdYeWxCE2XPshy294G1BC0KqFBigOZBTmgFBDze6X-4H0y12gk4l_i_TDLC6qZkKQsBR2D3ym4x68TKewS7lKXPX8I4BTswAQ3IIhKyOD4llbQu7QT_ju6DixXgDcZMXek2l4QlLrMev1u7KtMLWf6PSvA61JaKpii4lobgTewdzWm-WIDuU0nAcZHAVDNRe_KBuKbvuwhc15dg2D87WausU2B_lGAg4248f9tcjRfVAsdM4YoSOP6_3HCWScybPz5ABefwvyGGj_Asbs5ydLPUg5SsePG19h5s6d3eWBW4YMR3tRyrR52Leyuo7t2-o5dGmxGq2QEQs-SD_0f_BHeajHKQFbINzDdAyLSmdOLiQl4OvL95SOWICsAop64U0rG2GOp84YpwVTZ7IRuOkPrG5fbIP1fXGm4z98BjoJ6S2gEqIzfRFkhWfvJCNXWTtTTg" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7" "-"
nginx error.log
2016/07/25 12:38:53 [error] 9989#0: *127 connect() failed (111: Connection refused) while connecting to upstream, client: 10.3.0.232, server: hplipuppt11.corporate.local, request: "POST /auth/reset HTTP/1.1", upstream: "http://127.0.0.1:4430/auth/reset", host: "puppet.corporate.local", referrer: "https://puppet.corporate.local/auth/reset?token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJhZG1pbiIsInN1YiI6IjFmMzQ1MjI5LTIxZjktNDNjYy04YmNmLWVmZGU4N2U0Y2M5MyIsImV4cCI6MTQ2OTUwMDYzNywiaWF0IjoxNDY5NDE0MjM3fQ.SuOA-_c3f3otyDF-5LfmED2JioYo3BH5P3Jz_4r9EF4Z80tuz-pw8Z9eidJ8X7myiL4VoQF-sOQMs89ysxD7vh35ifgLnJnV7-2u3K6Ep1FdHHas_yRjdYSdEELJgHByNyqnSYQa5511f9wDYMZN_wyhbqCuXzbdUELjvVj27-sS0jEh2RZMi9Znp68473gwQGxziQkp_bWU1IJ9YJvPEjm29EQSg4Lm1l0b3FcjjdYeWxCE2XPshy294G1BC0KqFBigOZBTmgFBDze6X-4H0y12gk4l_i_TDLC6qZkKQsBR2D3ym4x68TKewS7lKXPX8I4BTswAQ3IIhKyOD4llbQu7QT_ju6DixXgDcZMXek2l4QlLrMev1u7KtMLWf6PSvA61JaKpii4lobgTewdzWm-WIDuU0nAcZHAVDNRe_KBuKbvuwhc15dg2D87WausU2B_lGAg4248f9tcjRfVAsdM4YoSOP6_3HCWScybPz5ABefwvyGGj_Asbs5ydLPUg5SsePG19h5s6d3eWBW4YMR3tRyrR52Leyuo7t2-o5dGmxGq2QEQs-SD_0f_BHeajHKQFbINzDdAyLSmdOLiQl4OvL95SOWICsAop64U0rG2GOp84YpwVTZ7IRuOkPrG5fbIP1fXGm4z98BjoJ6S2gEqIzfRFkhWfvJCNXWTtTTg"
more:
[root@hplipuppt11 console-services]# netstat -an | grep LISTEN | grep 4430
tcp 0 0 127.0.0.1:4430 0.0.0.0:* LISTEN
pe-cons+ 62597 14.3 2.7 3985572 448828 ? Ssl 12:38 1:31 /opt/puppetlabs/server/bin/java -Xmx256m -Xms256m -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/console-services/console-services-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/console-services/conf.d -b /etc/puppetlabs/console-services/bootstrap.cfg
[root@hplipuppt11 console-services]# grep 4433 /etc/services
vop 4433/tcp # Versile Object Protocol
[root@hplipuppt11 console-services]# lsof -i | grep vop
java 62597 pe-console-services 81u IPv4 111019 0t0 TCP *:vop (LISTEN)
So, looks like pe-console-services was restarting when i was trying to reset the password. Why?
Yep:
2016-07-25 12:37:17,404 INFO [p.p.r.users] Successful request to reset user password via POST /api/rbac/users/e2e8f10a-04ce-4421-9e00-8fbb4581db85/password/reset
2016-07-25 12:37:17,423 INFO [p.p.a.routes] Request activity for rbac users e2e8f10a-04ce-4421-9e00-8fbb4581db85
2016-07-25 12:38:50,019 INFO [p.t.internal] Shutting down due to JVM shutdown hook.
2016-07-25 12:38:50,021 INFO [p.t.internal] Beginning shutdown sequence
OH yep, agent run post installation must have triggered a refresh:
Jul 25 12:39:27 hplipuppt11 puppet-agent[61263]: (/Stage[main]/Puppet_enterprise::Console_services/Service[pe-console-services]) Triggered 'refresh' from 1 events