Skip to content

Instantly share code, notes, and snippets.

@parsa
Last active December 5, 2017 05:05
Show Gist options
  • Select an option

  • Save parsa/8e55bbd00ac6f84f87e84db95f63535f to your computer and use it in GitHub Desktop.

Select an option

Save parsa/8e55bbd00ac6f84f87e84db95f63535f to your computer and use it in GitHub Desktop.
Rebuilding glibc 2.3.x (more commonly known as "Playing with Fire")

From: http://oregontechsupport.com/articles/glibc.php

Rebuilding glibc 2.3.x (more commonly known as "Playing with Fire")

"Don't get suckered in by the comments -- they can be terribly misleading. Debug only code." -- Dave Storer

The GNU C Library (glibc) is more or less a collection of commonly used C-programming libraries. Many essential programs require these routines; in addition, you won't be able to compile any programs without it.

Disclaimer/Warning Messing with glibc can potentially damage your system, make your system inoperable, and is generally considered a bonehead idea unless you know what you're doing. Critical programs such as bash, agetty, ls, and so forth rely on and can be made inoperable by changes in the system's glibc library. Use the information contained here with great caution. Don't do this on a production machine until you've mastered the art of upgrading glibc libraries. The author makes no warranty or claims about this page, so be warned!

Why Rebuild

Many people think that having the "latest and greatest" version of anything is important. Frequently, this isn't true. With glibc, it's a mixed answer: upgrading for better (and less buggy) library routines versus potential system incompatibilities. If the upgrade is a minor version increase, it might be worth your while. For major version increases, it's not worth it unless you plan to recompile all your system software (and even that can be tricky when switching to a new library).

You can tell which version of glibc your system has by running:

$ ls /lib/libc-*
libc-2.3.2.so

The Danger

Binaries which are dynamically linked will fail if the required library is not present. Pretty much every modern program uses this method of operation, for better or for worse. The trick is to upgrade our glibc on a running system without making the system choke on us. Here's an example of what evil will happen with a botched glibc upgrade:

$ ls
bash: No such file or directory

Why is this? An examination of the ls command (prior to botching the system, of course) yields the answer:

$ file /bin/ls
/bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, dynamically linked (uses shared libs), stripped

As it turns out, our file is dynamically liked to shared libraries which are, you guessed it, part of the glibc libraries:

$ ldd /bin/ls
     librt.so.1 => /lib/librt.so.1 (0x40017000)
     libc.so.6 => /lib/libc.so.6 (0x4002b000)
     libpthread.so.0 => /lib/libpthread.so.0 (0x40159000)
     /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

If these libraries and/or their symlinks are missing or damaged, we're in for a world of hurt and pain.

The Theory of ldconfig(8)

In theory, the ldconfig(8) program allows you to upgrade minor version changes in your libraries while maintaining symlinks of the same name. For example, let's assume you have glibc-2.3.1 installed. As a result, one of the files you will have in your /lib directory will be:

-rwxr-xr-x     libnss_nisplus-2.3.1.so

Running ldconfig(8) will create symlinks using the *major* version number of each file located in /lib, /usr/lib, and in any directory listed in /etc/ld.so.conf. In this case, you will end up with:

-rwxr-xr-x     libnss_nisplus-2.3.1.so
lrwxrwxrwx     libnss_nisplus.so.2 -> libnss_nisplus-2.3.1.so

So, if you decide to upgrade to glibc-2.3.2, technically you'll have these files:

-rwxr-xr-x     libnss_nisplus-2.3.1.so
-rwxr-xr-x     libnss_nisplus-2.3.2.so
lrwxrwxrwx     libnss_nisplus.so.2 -> libnss_nisplus-2.3.1.so

And then running ldconfig would wipe out the symlink and replace it:

-rwxr-xr-x     libnss_nisplus-2.3.1.so
-rwxr-xr-x     libnss_nisplus-2.3.2.so
lrwxrwxrwx     libnss_nisplus.so.2 -> libnss_nisplus-2.3.2.so

Which works great assuming that:

  1. All programs that use this shared library refer to it by the symlink name only
  2. No backwards compatibility issues exist between upgrades in minor versions.

Usually, both assumptions are true. In reality, sometimes one or both are false. Thus the problem, and another reason to avoid playing with fire ;-)

Requirements

  • Download ftp://ftp.gnu.org/gnu/glibc-2.3.2.tar.bz2
  • Download ftp://ftp.gnu.org/gnu/glibc-linuxthreads-2.3.2.tar.bz2
  • Download ftp://ftp.kernel.org/pub/linux/kernel/2.4/linux-2.4.22.tar.bz2

Prepare the kernel headers

You'll need the most recent kernel headers in order to build glibc.

# make mrproper
# make menuconfig 
# make dep
# mkdir -p /usr/include/linux-2.4.22
# cp -a include/linux /usr/include/linux-2.4.22/
# cp -a include/asm* /usr/include/linux-2.4.22/

Note: When running "make menuconfig", you don't need to actually configure anything; just accept the defaults and then exit out of menuconfig. We don't want to overwrite our system's current kernel headers just yet as they may be incompatible with the new headers.

Preparing glibc

  • Extract the glibc archive
  • Extract the glibc-linuxthreads archive inside the glibc extracted directory from above
  • Create an empty dummy directory outside the glibc extracted directory in which to build glibc

Building glibc

$ ../glibc-2.3.2/configure --prefix=/usr \
     --disable-profile \
     --enable-add-ons \
     --with-headers=/usr/include/linux-2.4.22 \ 
     --libexecdir=/usr/bin
$ make
$ make check
# make install install_root=/tmp/glibc-build
# make localedata/install-locales install_root=/tmp/glibc-build
# rm -f /tmp/glibc-build/etc/ld.so.*

Our goal here is to initially build glibc without overwriting our existing glibc. We've done so by installing it temporarily into /tmp/glibc-build/ and removing the etc/ld.so* files so as not to overwrite ours later on. We need to make a few more changes to this directory to make it "safe" to install first, using the following script:

#!/bin/sh
cd /tmp/glibc-build/lib
for file in * ; do
     # Rename all non-symlink files
     if [ -f $file -a ! -h $file ]; then
          cp -a $file incoming/${file}.incoming
     fi
     # Delete all non-renamed files 
     rm $file 
done

Back up the existing glibc libraries

This is simply a precaution. How useful it is in real life, I'm not sure. But in practice, I always like to play it safe.

# cd / ; cp -fa lib lastlib 
# cd /usr ; cp -fa lib lastlib

Installing the new glibc

Take the system down to single-user level so as not to upset any currently running programs. If you're not sure what this is, then you probably shouldn't be attempting to use this document ;-) Then, run the following:

# cd /tmp/glibc-build
# cp -fa * /

Switch and Bait

Our system still has both the cache and links to our old library files. This is important as our system might not run otherwise. Next, let's carefully switch over to the new library. Note: that's a lower-case L option to ldconfig after the -v bit.

# cd /lib
# /sbin/ldconfig -v -l *.incoming 
# ls

If the result of that last command works, then so far, everything is working fine. We need to re-rename our libraries now without upsetting the system. Use the following script:

#!/bin/sh
cd /lib
for file in *.incoming ; do
     cp -fa $file `basename $file .incoming`
     /sbin/ldconfig -l `basename $file .incoming`
     mv $file ${file}.done 
done

You'll need to check and see if any broken symlinks remain; if so, you'll need to repair them by hand. Once done, you can delete the /lib/*.done files.

Last but not least, you'll need to switch your kernel headers to match your new glibc library:

# cd /usr/include/linux-2.4.22
# cp -fa linux asm* /usr/include/

In theory, everything should work fine now, and your new glibc library is installed correctly. By saying in theory is actually correct, as in practice many things can still go wrong. Be aware that you'll need to test your system and make sure that everything works properly. If you've come this far, rebuilding your system's programs and relinking to the new libraries is something to consider. ;-)

If you've gotten this far, congratulations! Otherwise, hopefully this document will help you get started in the right direction or help generate some new ideas about whatever tasks you are attempting with your glibc installation.

Symbol Stripping

One thing that happens (and is poorly documented) is the following error message when compiling new programs:

/usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 08048160

The number at the end isn't important. However, the hint is in the word symbol. If you've stripped the symbols off your glibc library (using strip(1) or the traditional LDFLAGS=-s variable), then you've gotten yourself into trouble. No symbols = nothing to link to. You'll need to recompile your glibc library without stripping any symbols this time. Of course, since we didn't strip any symbols in the steps in this document, one has to wonder where you got the idea from in the first place ;-)

If you feel that you must strip symbols to make your files smaller, use the following formula:

  1. For binaries, use "strip --strip-unneeded {,s}bin/\*"
  2. For libraries, use "strip --strip-debug lib/\*"
  3. For packages that are designed to be libraries only (e.g., glibc, zlib), don't strip anything. Trust me, you'll live longer and have more sanity. Find another creative outlet for your obsession ;-)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment