Instructions on getting through HDF compilation for Android

HDF5 Android How-to

How to build HDF5 for Android


References:
-----------

[1] http://developer.android.com/tools/sdk/ndk/index.html

[2] http://developer.android.com/sdk/index.html

[3] 
http://linux4hippos.blogspot.com/2013/05/cross-compile-hdf5-for-intel-xeon-phi.html

[4] http://danielpocock.com/building-existing-autotools-c-projects-on-android

Build Environment:
------------------

Ubuntu 12.04 LTS (64-bit)

Android NDK R9b x86_64 from [1]

HDF5 1.8.11

Steps:
------

0. Install the NDK in $ANDROID_NDK .

1. Create a standalone toolchain (GCC w/ the right sysroot) for Android 4.3 (API 
18) on ARM

$ANDROID_NDK/build/tools/make-standalone-toolchain.sh --platform=android-18

This creates a file called 
/tmp/ndk-<username>/arm-linux-androideabi-4.6.tar.bz2.
Unzip and set your environment variables. E.g.,

export PATH=/home/<username>/arm-linux-androideabi-4.6/bin:$PATH
export AR=arm-linux-androideabi-ar
export AS=arm-linux-androideabi-as
export CC=arm-linux-androideabi-gcc
export CPP=arm-linux-androideabi-cpp
export CXX=arm-linux-androideabi-g++
export LD=arm-linux-androideabi-ld
export NM=arm-linux-androideabi-nm
export RANLIB=arm-linux-androideabi-ranlib

2. Without some rather ugly hacks, configure is going nowhere. Following [3] a
few changes are necessary to get us off the ground:

a.) Edit test/Makefile.in and change the definition of TEST_SCRIPT from

TEST_SCRIPT = testerror.sh testlibinfo.sh testcheck_version.sh \
testlinks_env.sh $(am__append_1)

to

TEST_SCRIPT =

b.) Edit configure and remove the return statement 'as_fn_exit $as_status' from
the definition of as_fn_error.

See subtask HDFFV-8560.

3. ./configure -host=arm-linux-androideabi

See the attached config.log for the output. I couldn't figure out how to 
convince
configure to build the shared libraries. See subtask HDFFV-8561.

4. 'make' runs for a while and fails when compiling H5make_libsettings.c
See subtask HDFFV-8562.

We can fix that brute force by removing '#define H5_HAVE_GETPWUID 1' from 
H5pubconf.h

5. Running 'make' again tells us that we can't execute ARM binaries on our 
machine:

CC H5make_libsettings.o
CCLD H5make_libsettings
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo | \
sed -e 's/-L/:/g' -e 's/ //g'`" \
./H5make_libsettings > H5lib_settings.c || \
(test $HDF5_Make_Ignore && echo "*** Error ignored") || \
(rm -f H5lib_settings.c ; exit 1)
/bin/bash: line 1: ./H5make_libsettings: cannot execute binary file
make[2]: *** [H5lib_settings.c] Error 1
make[2]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make: *** [all-recursive] Error 1

See subtask HDFFV-8563.

5. Time to fire up the Android phone emulator! This is part of the Android SDK. 
[2]
Create a new Android Virtual Device (AVD).
Give it an name, e.g., 'HDF5emu'.
Device: Nexus S (4.0", 480x800: hdpi)
Target: Android 4.3 (API Level 18)
CPU/ABI: ARM (armeabi-v7a)
... (no changes of defaults)
Internal Storage: 1 GiB
SD Card: 256 MiB
Emulation Options: check 'Snapshot'

Click 'Ok' and start the new AVD. (click 'Start')

Make sure that the file system you are running the tests in is mounted 
read/write:

adb shell
su
mount -o rw,remount rootfs /
chmod 777 /mnt/sdcard
exit

6. Copy H5make_libsettings onto to AVD via the Android Debug Bridge:

Create a directory: adb shell mkdir /mnt/sdcard/hdf5
Copy the file: adb push src/H5make_libsettings /mnt/sdcard/hdf5
Copy the file: adb push src/libhdf5.settings /mnt/sdcard/hdf5
Make the file executable: adb shell chmod 777 
/mnt/sdcard/hdf5/H5make_libsettings
Run it: adb shell "cd /mnt/sdcard/hdf5 ; ./H5make_libsettings"

See subtask HDFFV-8564.

See the attachment H5lib_settings.c for the output.

Type 'make'.

7. 'make' cruises along until H5detect

CCLD H5detect
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo | \
sed -e 's/-L/:/g' -e 's/ //g'`" \
./H5detect > H5Tinit.c || \
(test $HDF5_Make_Ignore && echo "*** Error ignored") || \
(rm -f H5Tinit.c ; exit 1)
/bin/bash: line 1: ./H5detect: cannot execute binary file
make[2]: *** [H5Tinit.c] Error 1
make[2]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/gerd/Android/HDF5/hdf5-1.8.11/src'
make: *** [all-recursive] Error 1

Same fix as before: copy H5detect to AVD, run, and capture the output in 
H5Tinit.c (see attachment)

8. 'make' finishes building the library and starts making tests. It fails in
     compiling tcheck_version because of a global name conflict. See subtask
     HDFFV-8565.

     Android defines two global functions 'major' and 'minor'. For now, just
     rename the global variables in tcheck_version and move on!

9. 'make' continues and starts building the tools until it hits h5ls. See
     subtask HDFFV-8566.

    Brute force fix: delete H5_HAVE_IOCTL from H5pubconf.h .

10. H5make_libsettings is invoked a second time and fails. See subtask HDFFV-8563.

11. H5detect is invoked a second time and fails. See subtask HDFFV-8563.

12. After that tests and tools build fine to the end.

13. Copy all the tests and HDF5 files onto the AVD and run them. All pass, as
     far as I recall.

14. 'make install' works.