puppetserver error

gistfile1.txt

Nov 12 17:45:47 fqdn.com systemd[1]: Starting puppetserver Service...
Nov 12 17:45:47 fqdn.com systemd[1]: Started puppetserver Service.
Nov 12 17:46:12 fqdn.com java[56495]: Failed to load feature test for posix: can't find user for 0
Nov 12 17:46:12 fqdn.com java[56495]: Cannot run on Microsoft Windows without the win32-process, win32-dir and win32-service gems: Win32API only supported on win32
Nov 12 17:46:12 fqdn.com java[56495]: Puppet::Error: Cannot determine basic system flavour
Nov 12 17:46:12 fqdn.com java[56495]: (root) at /usr/share/ruby/vendor_ruby/puppet/feature/base.rb:32
Nov 12 17:46:12 fqdn.com java[56495]: require at org/jruby/RubyKernel.java:1065
Nov 12 17:46:12 fqdn.com java[56495]: (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
Nov 12 17:46:12 fqdn.com java[56495]: require at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
Nov 12 17:46:12 fqdn.com java[56495]: Puppet at /usr/share/ruby/vendor_ruby/puppet.rb:75
Nov 12 17:46:12 fqdn.com java[56495]: require at org/jruby/RubyKernel.java:1065
Nov 12 17:46:12 fqdn.com java[56495]: (root) at /usr/share/ruby/vendor_ruby/puppet.rb:29
Nov 12 17:46:12 fqdn.com java[56495]: (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
Nov 12 17:46:12 fqdn.com java[56495]: require at org/jruby/RubyKernel.java:1065
Nov 12 17:46:12 fqdn.com java[56495]: require at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
Nov 12 17:46:12 fqdn.com java[56495]: (root) at file:/usr/share/puppetserver/puppet-server-release.jar!/puppet-server-lib/puppet/server/master.rb:1
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_core.clj:136
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_core.clj:149
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_core.clj:139
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_core.clj:251
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_core.clj:239
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_service.clj:33
Nov 12 17:46:12 fqdn.com java[56495]: invoke at internal.clj:246
Nov 12 17:46:12 fqdn.com java[56495]: invoke at internal.clj:230
Nov 12 17:46:12 fqdn.com java[56495]: shutdown_on_error at internal.clj:285
Nov 12 17:46:12 fqdn.com java[56495]: invoke at internal.clj:254
Nov 12 17:46:12 fqdn.com java[56495]: invoke at internal.clj:254
Nov 12 17:46:12 fqdn.com java[56495]: invoke at core.clj:619
Nov 12 17:46:12 fqdn.com java[56495]: doInvoke at core.clj:2396
Nov 12 17:46:12 fqdn.com java[56495]: invoke at jruby_puppet_service.clj:32
Nov 12 17:46:12 fqdn.com java[56495]: invoke at core.clj:1836
Nov 12 17:46:12 fqdn.com java[56495]: Exception in thread "main" java.lang.IllegalStateException: There was a problem adding a JRubyPuppet instance to the pool.
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.services.jruby.jruby_puppet_core$eval6764$prime_pools_BANG___6765$fn__6766.invoke(jruby_puppet_core.clj:257)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.services.jruby.jruby_puppet_core$eval6764$prime_pools_BANG___6765.invoke(jruby_puppet_core.clj:239)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.services.jruby.jruby_puppet_service$reify__6929$service_fnk__5254__auto___positional$reify__6940$fn__6941$fn__6942.invoke(jruby_puppet_service.clj:3
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.trapperkeeper.internal$shutdown_on_error_STAR_.invoke(internal.clj:246)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.trapperkeeper.internal$shutdown_on_error_STAR_.invoke(internal.clj:230)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.trapperkeeper.internal$shutdown_service$reify__5853$service_fnk__5254__auto___positional$reify__5858.shutdown_on_error(internal.clj:285)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.trapperkeeper.internal$eval5789$fn__5801$G__5775__5809.invoke(internal.clj:254)
Nov 12 17:46:12 fqdn.com java[56495]: at puppetlabs.trapperkeeper.internal$eval5789$fn__5801$G__5774__5818.invoke(internal.clj:254)
Nov 12 17:46:12 fqdn.com java[56495]: at clojure.lang.AFn.applyToHelper(AFn.java:167)

# java -cp puppet-server-release.jar org.jruby.Main -r syslog -e 'puts :syslog'
LoadError: Syslog not supported on this platform
   (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/syslog.rb:20
  require at org/jruby/RubyKernel.java:1065
   (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1

Kickstart base packages:

%packages --nobase
@core
chrony
ruby
puppet
open-vm-tools
open-vm-tools-deploypkg
finger
telnet
bind-utils
net-tools
krb5-workstation
sg3_utils
nfs-utils
lsof
vim-enhanced
sssd
sssd-common
sssd-client
sssd-tools
ldb-tools
openldap-clients
redhat-lsb-core
tree
policycoreutils-python
ksh
sysstat
perl
wget
rhn-setup
rhn-check
rhn-client-tools
%end

# java -cp puppet-server-release.jar org.jruby.Main -r etc -e'puts Etc.getpwuid(0).inspect'
nil

# ruby -r etc -e'puts Etc.getpwuid(0).inspect'
#<struct Struct::Passwd name="root", passwd="x", uid=0, gid=0, gecos="root", dir="/root", shell="/bin/bash">

Installed a local copy of jruby

# ./jruby -r etc -e'puts Etc.getpwuid(0).inspect'
#<struct Struct::Passwd name="root", passwd="x", uid=0, gid=0, gecos="root", dir="/root", shell="/bin/bash", change=0, uclass="", expire=2147483647>

# ./jruby -r syslog -e 'puts :syslog'
syslog

# ./jruby --version
jruby 1.7.15 (1.9.3p392) 2014-09-03 82b5cc3 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_72-b14 +jit [linux-amd64]

# java -cp puppet-server-release.jar -Djruby.native.verbose=true org.jruby.Main foo.rb
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
    at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:30)
    at jnr.ffi.LibraryLoader.load(LibraryLoader.java:265)
    at jnr.ffi.Library.loadLibrary(Library.java:123)
    at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:219)
    at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:223)
    at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:36)
    at jnr.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:18)
    at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:95)
    at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:69)
    at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:38)
    at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:35)
    at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:31)
    at jnr.posix.LazyPOSIX.allocateStat(LazyPOSIX.java:49)
    at org.jruby.util.RegularFileResource.isSymLink(RegularFileResource.java:81)
    at org.jruby.util.RegularFileResource.exists(RegularFileResource.java:66)
    at org.jruby.runtime.load.LibrarySearcher.findFileResourceWithLoadPath(LibrarySearcher.java:173)
    at org.jruby.runtime.load.LibrarySearcher.findResourceLibrary(LibrarySearcher.java:145)
    at org.jruby.runtime.load.LibrarySearcher.findLibrary(LibrarySearcher.java:81)
    at org.jruby.runtime.load.LibrarySearcher.findBySearchState(LibrarySearcher.java:70)
    at org.jruby.runtime.load.LoadService.findLibraryBySearchState(LoadService.java:997)
    at org.jruby.runtime.load.LoadService.findFileForLoad(LoadService.java:411)
    at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:556)
    at org.jruby.runtime.load.LoadService.requireCommon(LoadService.java:453)
    at org.jruby.runtime.load.LoadService.require(LoadService.java:417)
    at org.jruby.javasupport.Java.load(Java.java:118)
    at org.jruby.runtime.load.LoadService.reflectedLoad(LoadService.java:626)
    at org.jruby.ext.LateLoadingLibrary.load(LateLoadingLibrary.java:48)
    at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:50)
    at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:916)
    at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:569)
    at org.jruby.runtime.load.LoadService.requireCommon(LoadService.java:453)
    at org.jruby.runtime.load.LoadService.require(LoadService.java:417)
    at org.jruby.ext.jruby.JRubyLibrary.load(JRubyLibrary.java:76)
    at org.jruby.runtime.load.LoadService.reflectedLoad(LoadService.java:626)
    at org.jruby.ext.LateLoadingLibrary.load(LateLoadingLibrary.java:48)
    at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:50)
    at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:916)
    at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:569)
    at org.jruby.runtime.load.LoadService.requireCommon(LoadService.java:453)
    at org.jruby.runtime.load.LoadService.require(LoadService.java:417)
    at org.jruby.Ruby.init(Ruby.java:1254)
    at org.jruby.Ruby.newInstance(Ruby.java:330)
    at org.jruby.Main.internalRun(Main.java:258)
    at org.jruby.Main.run(Main.java:217)
    at org.jruby.Main.main(Main.java:197)
Caused by: java.lang.ExceptionInInitializerError
    at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
    at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at java.lang.Class.newInstance(Class.java:379)
    at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:60)
    at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:49)
    at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
    at jnr.ffi.Library.loadLibrary(Library.java:110)
    ... 42 more
Caused by: java.lang.IllegalStateException: Can't overwrite cause with java.lang.UnsatisfiedLinkError
    at java.lang.Throwable.initCause(Throwable.java:456)
    at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:252)
    at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
    at com.kenai.jffi.Type.resolveSize(Type.java:155)
    at com.kenai.jffi.Type.size(Type.java:138)
    at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
    at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
    at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
    at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
    at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
    ... 53 more
Caused by: java.lang.UnsatisfiedLinkError
    at com.kenai.jffi.Foreign.newLoadError(Foreign.java:72)
    at com.kenai.jffi.Foreign.access$300(Foreign.java:42)
    at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:98)
    at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
    at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
    ... 61 more
Caused by: java.lang.UnsatisfiedLinkError: /tmp/jffi1424102812055706119.tmp: /tmp/jffi1424102812055706119.tmp: failed to map segment from shared object: Operation not permitted
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
    at java.lang.Runtime.load0(Runtime.java:795)
    at java.lang.System.load(System.java:1062)
    at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:328)
    at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:235)
    at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:403)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at com.kenai.jffi.Init.load(Init.java:68)
    at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
    at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
    ... 63 more
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1065
  require at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
   (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1065
   (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
  require at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  require at org/jruby/RubyKernel.java:1065
   (root) at /usr/share/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/ffi.rb:1
   (root) at foo.rb:1

# sh ./META-INF/jruby.home/bin/jruby -r syslog -e 'puts :syslog'
LoadError: Syslog not supported on this platform
   (root) at /usr/share/puppetserver/puppet-server-release_exploded/META-INF/jruby.home/lib/ruby/shared/syslog.rb:20
  require at org/jruby/RubyKernel.java:1065
   (root) at /usr/share/puppetserver/puppet-server-release_exploded/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1

Final result /tmp was mounted noexec so had to

1. create /var/lib/puppet/tmp and change ownership to puppet
2. add to /etc/sysconfig/puppetserver javaargs -Djava.io.tmpdir=/var/lib/puppet/tmp

Thanks! You just saved me hours of work! :)

Thanks for sharing your result, saved me a lot of time too !

For clarification, you want to append '-Djava.io.tmpdir=/var/lib/puppet/tmp' to the JAVA_ARGS line. For RHEL8 and Puppet server 7.6 mine looks like this:

# Modify this if you'd like to change the memory allocation, enable JMX, etc
JAVA_ARGS="-Xms2g -Xmx2g -Djruby.logger.class=com.puppetlabs.jruby_utils.jruby.Slf4jLogger -Djava.io.tmpdir=/var/lib/puppet/tmp"