Skip to content

Instantly share code, notes, and snippets.

@rampage644
Created August 20, 2014 13:29
Show Gist options
  • Save rampage644/5ebeadd9aaad3f8a75b4 to your computer and use it in GitHub Desktop.
Save rampage644/5ebeadd9aaad3f8a75b4 to your computer and use it in GitHub Desktop.
Impala Pth
# Gnu Pth as thread library for impalad
In short, it's impossible to use Gnu Pth library with `impalad` "AS IS", i.e. without modification.
Gnu Pth:
* Gnu Pth can't fully replace `pthreads`. It lacks some functions, some entities.
* It doesn't provide versioned symbols
There are some `*.so` libraries (system/thirdparty) which come precompiled and they are linked against versioned symbols. Be prepared to recompile them replace somehow or just do anything. Example:
* libjvm.so (oracle jre)
* libhdfs.so (feel free to find sources)
* librt.so (system one)
* boost.so
They also may use (and actually, do so) some features Gnu Pth doesn't provide (semaphores as most striking example, there is pth fork with semaphores, though).
> Written with [StackEdit](https://stackedit.io/).

Gnu Pth as thread library for impalad

In short, it's impossible to use Gnu Pth library with impalad "AS IS", i.e. without modification.

Gnu Pth:

  • Gnu Pth can't fully replace pthreads. It lacks some functions, some entities.
  • It doesn't provide versioned symbols

There are some *.so libraries (system/thirdparty) which come precompiled and they are linked against versioned symbols. Be prepared to recompile them replace somehow or just do anything. Example:

  • libjvm.so (oracle jre)
  • libhdfs.so (feel free to find sources)
  • librt.so (system one)
  • boost.so

They also may use (and actually, do so) some features Gnu Pth doesn't provide (semaphores as most striking example, there is pth fork with semaphores, though).

25.08 update

Today i tried to force compile executor with pth instead of pthread. First, I was able to reduce the number of undefined reference's when linking with pth via this Version file:

	GLIBC_2.2.5 {
	    pthread_create;
	    pthread_join;
	    pthread_detach;
	    pthread_attr_setstacksize;
	    pthread_attr_getstacksize;
	    pthread_attr_getstack;
	    pthread_setspecific;
	    pthread_barrier_init;
	    pthread_cancel;
	    sem_destroy;
	    pthread_create;
	    pthread_getattr_np;
	    pthread_key_create;
	    pthread_key_delete;
	    pthread_sigmask;
	    sem_trywait;
	    __pthread_get_minstack;
	    pthread_attr_setguardsize;
	    sem_wait;
	    sem_post;
	    sem_timedwait;
	    pthread_condattr_setclock;
	    sem_init;
	    pthread_once;
	    pthread_mutex_trylock;
	    pthread_join;
	    pthread_kill;
	    __pthread_unwind;
	    pthread_barrier_wait;
	    pthread_getspecific;
	};

Also, i make pthread_detach as weak alias for __pthread_detach which is present in pth.

Remaining undefined reference's come from:

  • librt.so
  • libjvm.so
  • libhdfs.so
  • libthrift.a

They address features not present in pth.

I still have no idea about how version-ed symbols should work and created. I tried creating dummy shared library referencing pthread. It also reference symbols with @GLIBC suffix. It seems that they come from linker when linking target shared library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment