GDAL 2.0 on Amazon Linux

xlab commented Jan 10, 2017 •

edited

Loading

Lol, still in there.. what if somebody doesn't read comments? 🥇

python1981 commented Jul 4, 2017

Thanks, this is a useful reference. (Agree it might be a nice touch to remove the rm /usr/local/)

nbeuchat commented Apr 2, 2019

That's useful. What is the point of the last line tar zcvf ~/gdal-2.0.0-amz1.tar.gz *? My understanding is that you're basically putting everything from the /usr/local into the tar file instead of the gdal-specific files

nashmaniac commented May 3, 2019

I tried using this in ebextensions like this.. got Timeout error. My .ebextension is like this.

commands:
  01_yum_update:
    command: sudo yum -y update

files:
  "/etc/httpd/conf.d/wsgihacks.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      WSGIPassAuthorization On

packages:
  yum:
    git: []
    postgresql96-devel: []
    gettext: []
    libjpeg-turbo-devel: []
    libffi-devel: []

files:
 "/opt/elasticbeanstalk/hooks/appdeploy/pre/01_install_gdal_prerequisites.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
        #!/usr/bin/env bash
        sudo yum -y update
        sudo yum-config-manager --enable epel
        sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel
        cd /tmp
        curl -L http://download.osgeo.org/gdal/2.0.0/gdal-2.0.0.tar.gz | tar zxf -
        cd gdal-2.0.0/
        ./configure --prefix=/usr/local --without-python
        make -j4
        sudo make install
        cd /usr/local
        tar zcvf ~/gdal-2.0.0-amz1.tar.gz *

aaronsilverman commented Jun 6, 2019 •

edited

Loading

I'm doing this on a docker image (amazonlinux:2.0.20190228) and it looks like you need to install yum-utils and install and enable the EPEL repo as outlined here: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-enable-epel/

dliff commented Jun 28, 2019

We are using this to run GDAL on Amazon Linux.
Since compiling takes quite a long time, you probably won't want to recompile each time an EC2 loads up (we're using Elastic Beanstalk's autoscaling).
If you host the tar file you create as well as libproj.so (in /usr/lib64) you can add container commands to avoid recompiling each time.

Rough idea of what is working so far:

  04_install_gdal:
    command: |
      sudo wget [url]/gdal_extract.tgz
  05_install_gdal2:
    command: |
      sudo wget [url]libproj.so
  06_install_gdal3:
    command: |
      sudo tar -xvf gdal_extract.tgz -C /usr/local/
  07_install_gdal4:
    command: |
      sudo cp libproj.so /usr/lib64/

leogny commented Sep 30, 2019

We are using this to run GDAL on Amazon Linux.
Since compiling takes quite a long time, you probably won't want to recompile each time an EC2 loads up (we're using Elastic Beanstalk's autoscaling).
If you host the tar file you create as well as libproj.so (in /usr/lib64) you can add container commands to avoid recompiling each time.

Rough idea of what is working so far:
  04_install_gdal:
    command: |
      sudo wget [url]/gdal_extract.tgz
  05_install_gdal2:
    command: |
      sudo wget [url]libproj.so
  06_install_gdal3:
    command: |
      sudo tar -xvf gdal_extract.tgz -C /usr/local/
  07_install_gdal4:
    command: |
      sudo cp libproj.so /usr/lib64/

Can you share the full code for this with the commands to avoid recompiling each time?

dliff commented Oct 9, 2019 •

edited

Loading

We are using this to run GDAL on Amazon Linux.
Since compiling takes quite a long time, you probably won't want to recompile each time an EC2 loads up (we're using Elastic Beanstalk's autoscaling).
If you host the tar file you create as well as libproj.so (in /usr/lib64) you can add container commands to avoid recompiling each time.
Rough idea of what is working so far:
  04_install_gdal:
    command: |
      sudo wget [url]/gdal_extract.tgz
  05_install_gdal2:
    command: |
      sudo wget [url]libproj.so
  06_install_gdal3:
    command: |
      sudo tar -xvf gdal_extract.tgz -C /usr/local/
  07_install_gdal4:
    command: |
      sudo cp libproj.so /usr/lib64/
Can you share the full code for this with the commands to avoid recompiling each time?

This is essentially the full code (these are container commands for Elastic Beanstalk). I have removed the URL's because they are hosted on our public S3. You will need to follow the instructions in the original post to produce a tar.gz, then upload that tgz and paste the URL for the 04_install_gdal command. Then, you'll need to copy /usr/lib64/libproj.so and host that on S3 also, and paste that URL for 05_install_gdal2. Hope that helps.

Do the whole process on an EC2 instance that is identical to the ones that will be autoscaled.

essy2017 commented Jan 9, 2020

Thanks a bunch for this post, very helpful. I've used this successfully on Elastic Beanstalk. The only thing I'd add is that EB now has a "test" parameter for commands that can potentially save needless downloads (or maybe the parameter option has been around for awhile, I just discovered it):

04_install_gdal:  
    command: sudo wget [url]/gdal_extract.tgz  
    test: "[ ! -f gdal_extract.tgz ]"  
05_install_gdal2:
  command: sudo wget [url]libproj.so
  test: "[ ! -f /usr/lib64/libproj.so ]"
06_install_gdal3:
  command:  sudo tar -k -xvf gdal_extract.tgz -C /usr/local
  test: "[ ! -f /usr/local/lib/libgdal.so ]"
07_install_gdal4:
  command: sudo cp libproj.so /usr/lib64/
  test: "[ ! -f /usr/lib64/libproj.so ]"

So the commands will only run if the test evaluates to true (or returns 0). For 06_install_gdal3 I just check for the existence of libgdal.so, which presumably won't be there without a GDAL installation. Also I use the -k flag on the tar command to avoid replacing existing files and blowing up the existing /usr/local directory.

speedy250 commented Jan 14, 2020

Anyone willing to provide some support my way? I'm hitting an error deploying new ec2 instances, and any help would be greatly appreciated!

My Goal:
Precompiled gdal to run in elastic beanstalk so that spinning up a new server does not take 10-15 minutes (same goal as mentioned above)

I am currently trying the pre deploy method used by @nashmaniac and will likely hit a timeout error - the build time for gdal is quite long on smaller ec2 instances, hence the above goal.

I've created the 2 tar.gz files (per original post et al):

sudo yum -y update
sudo yum-config-manager --enable epel
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel s3cmd gdal-bin
cd /tmp
curl -L http://download.osgeo.org/gdal/2.4.4/gdal-2.4.4.tar.gz | tar zxf -
cd gdal-2.4.4/
./configure --prefix=/usr/local --without-python
make -j4
sudo make install
sudo ldconfig
cd /usr/local
tar zcvf ~/gdal-bundle.tar.gz *
cd /usr/lib64/
tar zcvf ~/libproj-bundle.tar.gz ./libproj.so

These are moved to S3 using s3cmd

Following the advice from @dliff and @essy2017, in Elastic Beanstalk .ebextensions I've got the container commands fetching and unpacking:

container_commands:
  03_set_cwd:
    command: cd /tmp/
  04_gdal_copy_s3:
    command: wget [s3-url]/gdal-full-bundle.tar.gz
  05_libproj_copy_s3:
    command: wget [s3-url]/libproj-bundle.tar.gz
  06_gdal_unpack:
    command: tar -xvf gdal-full-bundle.tar.gz -C /usr/local/
  07_libproj_unpack:
    command: tar -xvf libproj-bundle.tar.gz -C /usr/lib64/
  10_collectstatic:
    command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
  12_migrate:
    command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput"
    leader_only: true

But eb deploy keeps failing at 12_migrate throwing the error

  ...
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/db/models/fields.py", line 3, in <module>
    from django.contrib.gis import forms, gdal
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/forms/__init__.py", line 3, in <module>
    from .fields import (  # NOQA
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/forms/fields.py", line 2, in <module>
    from django.contrib.gis.gdal import GDALException
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module>
    from django.contrib.gis.gdal.datasource import DataSource
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/datasource.py", line 39, in <module>
    from django.contrib.gis.gdal.driver import Driver
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module>
    from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal
  File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/libgdal.py", line 46, in <module>
    lgdal = CDLL(lib_path)
  File "/usr/lib64/python3.6/ctypes/__init__.py", line 343, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libgdal.so.20: cannot open shared object file: No such file or directory

SSH into EC2 and run gdalinfo --version throws this:
gdalinfo: error while loading shared libraries: libgeos_c.so.1: cannot open shared object file: No such file or directory

Notes:

I had export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH as a container command to no avail. I've also confirmed the var's value in the shell to be /user/local/lib
adding sudo to each container command did not work
I tried ssh'ing in and linking libgdal.so.20 using ln -s /usr/lib/libgdal.so /usr/lib/libgdal.so.1 but it didn't help, same error

I seem to be going in circles for answers, and I'm hoping someone encountered what I'm seeing!

Also, is it just bad practice to copy over the whole /usr/local folder?

essy2017 commented Jan 14, 2020

@seanwhudson

I actually encountered errors myself after deployment. I'm not sure about your current instance, but in your last note creating the simlink for libgdal.so.20 did you have a typo? The OSError looks like you want to link to libgdal.so.20 not libgdal.so.1.

I have found a new solution after starting from scratch by modifying install and compile scripts I found at https://gist.github.com/hervenivon/fe3a327bc28b142e51beb38ef11844c0.

[1] Create a script install-gdal.sh:

#!/bin/bash

# Installation of GDAL and dependencies.
# Based on https://gist.github.com/hervenivon/fe3a327bc28b142e51beb38ef11844c0

# Update with your versions.
GEOS_VERSION=3.7.1
GDAL_VERSION=2.3.3
PROJ4_VERSION=4.9.3

sudo yum-config-manager --enable epel
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel

# Compilation work for geos.
mkdir -p "/tmp/geos-${GEOS_VERSION}-build"
cd "/tmp/geos-${GEOS_VERSION}-build"
curl -o "geos-${GEOS_VERSION}.tar.bz2" \
    "http://download.osgeo.org/geos/geos-${GEOS_VERSION}.tar.bz2" \
    && bunzip2 "geos-${GEOS_VERSION}.tar.bz2" \
    && tar xvf "geos-${GEOS_VERSION}.tar"
cd "/tmp/geos-${GEOS_VERSION}-build/geos-${GEOS_VERSION}"
./configure --prefix=/usr/local/geos

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Compilation work for proj4.
mkdir -p "/tmp/proj-${PROJ4_VERSION}-build"
cd "/tmp/proj-${PROJ4_VERSION}-build"
curl -o "proj-${PROJ4_VERSION}.tar.gz" \
    "http://download.osgeo.org/proj/proj-${PROJ4_VERSION}.tar.gz" \
    && tar xfz "proj-${PROJ4_VERSION}.tar.gz"
cd "/tmp/proj-${PROJ4_VERSION}-build/proj-${PROJ4_VERSION}"
./configure --prefix=/usr/local/proj4

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Compilation work for GDAL
mkdir -p "/tmp/gdal-${GDAL_VERSION}-build"
cd "/tmp/gdal-${GDAL_VERSION}-build"
curl -o "gdal-${GDAL_VERSION}.tar.gz" \
    "http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz" \
    && tar xfz "gdal-${GDAL_VERSION}.tar.gz"
cd "/tmp/gdal-${GDAL_VERSION}-build/gdal-${GDAL_VERSION}"
./configure --prefix=/usr/local/gdal \
            --with-static-proj4=/usr/local/proj4 \
            --without-python

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Bundle resources.
cd /usr/local/geos
tar zcvf "/tmp/geos-${GEOS_VERSION}.tar.gz" *

cd /usr/local/proj4
tar zcvf "/tmp/proj4-${PROJ4_VERSION}.tar.gz" *

cd /usr/local/gdal
tar zcvf "/tmp/gdal-${GDAL_VERSION}.tar.gz" *

[2] Run the script on an EC2 instance (takes a long time) after which you should have the three bundled files available:

/tmp/geos-3.7.1.tar.gz
/tmp/proj4-4.9.3.tar.gz
/tmp/gdal-2.3.3.tar.gz

[3] Upload these to your S3 bucket.

[4] Add the following to your .ebextensions. Each command tests for the existence of the directory on your instance and downloads and unzips the source as needed.

commands:

  01_install_geos:
    command: |
      sudo wget [s3-url]/geos-3.7.1.tar.gz
      sudo mkdir -p /usr/local/geos
      sudo tar -xvf geos-3.7.1.tar.gz -C /usr/local/geos
      sudo rm -f geos-3.7.1.tar.gz
    test: "[ ! -d /usr/local/geos ]"

  02_install_proj4:
    command: |
      sudo wget [s3-url]/proj4-4.9.3.tar.gz
      sudo mkdir -p /usr/local/proj4
      sudo tar -xvf proj4-4.9.3.tar.gz -C /usr/local/proj4
      sudo rm -f proj4-4.9.3.tar.gz
    test: "[ ! -d /usr/local/proj4 ]"

  03_install_gdal:
    command: |
      sudo wget [s3-url]/gdal-2.3.3.tar.gz
      sudo mkdir -p /usr/local/gdal
      sudo tar -xvf gdal-2.3.3.tar.gz -C /usr/local/gdal
      sudo rm -f gdal-2.3.3.tar.gz
    test: "[ ! -d /usr/local/gdal ]"

[5] Finally, add the following to your .ebextensions/options.config file:

option_settings:
  aws:elasticbeanstalk:application:environment:
    PATH: /usr/local/gdal/bin:$PATH
    LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
    GDAL_DATA: /usr/local/gdal/share/gdal

Seems like a lot of steps now that I've typed it all out but it has been working reliably for me on EB running Python 3.6 running on 64bit Amazon Linux/2.9.4. Seems much cleaner than bundling the entire /usr/local directory too.

speedy250 commented Jan 15, 2020

This is excellent, thank you @essy2017. I'll take a run and post any issues I work through as well.

I'm very green to the Linux (and aws's Centos) env, so I was unsure of how to bundle just geos/gdal/proj and transport them to a different system - and even if it was possible! From the script it looks like just taking the /usr/local/{lib} folders is sufficient, so that's good to know.

speedy250 commented Jan 15, 2020 •

edited

Loading

Okay, managed to get it working! Just two small fixes to your option_settings

option_settings:
  aws:elasticbeanstalk:application:environment:
    PATH: /usr/local/gdal/bin:$PATH
    LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/geos/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
    GDAL_LIBRARY_PATH: /usr/local/gdal/lib/libgdal.so

Note the addition of /usr/local/geos/lib: in LD_LIBRARY_PATH and changing GDAL_DATA to GDAL_LIBRARY_PATH - while the second may have worked, the first change to add the geos libs was what finally turned the corner for me.

One thing I would caution is that eb runs .ebextension files in alphabetical order, so it might be worth it to either merge the above two files, or prefix their names with 01 and 02 to avoid any strange behavior.

Lastly, bcs I never answered before re: typo - No; it was throwing an error looking for [...].1 shared object files, which helped me pin down the missing geos path.

Learned a ton here, thanks again for the assist.

serge-gaia commented Feb 13, 2020

Thank you @essy2017 and @vancityhuddy. You saved my day.

speedy250 commented Feb 17, 2020

For anyone else I've created a small example repo. Hope it helps!

https://github.com/vancityhuddy/aws-eb-gdal-example

Zalkota commented Feb 17, 2020 •

edited

Loading

I set up the script and .ebextension/01_options.conf file from the previous posts, but now I am getting the error:

from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal
File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/contrib/gis/gdal/libgdal.py", line 47, in <module>
lgdal = CDLL(lib_path)
File "/usr/lib64/python3.6/ctypes/__init__.py", line 343, in __init__
 self._handle = _dlopen(self._name, mode)
`OSError: libcrypto.so.1.1: cannot open shared object file: No such file or directory`

From my understanding, the path to that file is invalid. Here is my 01_options.conf:

...
option_settings:
  aws:elasticbeanstalk:application:environment:
    PATH: /usr/local/gdal/bin:$PATH
    LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/geos/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
    GDAL_LIBRARY_PATH: /usr/local/gdal/lib/libgdal.so

Vancityhuddy and essy2017, this is very much appreciated.

speedy250 commented Feb 17, 2020

I ran into something similar, and iirc it had to do with the gdal versions and/or the output names I was using. Don't go for latest osgeo libraries, I think that caused an issue for me. You'll see in my gdal-build-script from the linked repo that I set the versions to

GEOS_VERSION=3.6.4
GDAL_VERSION=2.4.4
PROJ4_VERSION=5.2.0

A quicker solution (bcs you don't have to recompile) is to verify your lib names. SSH into your ec2 instance, rummage around your /usr/local/ folders or search for which lib filenames exist on the machine. Then you could update your 01_options.conf to match the filenames.

Zalkota commented Feb 17, 2020 •

edited

Loading

I recompiled with those versions, but no luck.

I tried searching through the EC2 using ssh, but I only found the following:
Location: /usr/lib64/

libcrypto.so                       
libcrypto.so.10         
libcrypto.so.1.0.2k         
libcrypt.so

Location: /lib64/
libcrypt.so.1

I don't think Libcrypto.so.1.1 is installed on this EC.

My AWS EBS application seems to failing here when trying to execute this method in GeoDjango:
https://github.com/django/django/blob/86908785076b2bbc31b908781da6b6ad1779b18b/django/contrib/gis/gdal/libgdal.py#L46

Vancity, does your Amazon Linux have libcrypto.so.1.1?

hanayashiki commented Oct 14, 2020

this is very very very slow...

speedy250 commented Oct 19, 2020 •

edited

Loading

this is very very very slow...

@hanayashiki
It is very dependent on what type of machine you put into play. I'd recommend not going less than a T3.small - if you run a micro or nano it will take forever!

rahulkhairnarr commented Oct 19, 2020

@seanwhudson

I actually encountered errors myself after deployment. I'm not sure about your current instance, but in your last note creating the simlink for libgdal.so.20 did you have a typo? The OSError looks like you want to link to libgdal.so.20 not libgdal.so.1.

I have found a new solution after starting from scratch by modifying install and compile scripts I found at https://gist.github.com/hervenivon/fe3a327bc28b142e51beb38ef11844c0.

[1] Create a script install-gdal.sh:

#!/bin/bash

# Installation of GDAL and dependencies.
# Based on https://gist.github.com/hervenivon/fe3a327bc28b142e51beb38ef11844c0

# Update with your versions.
GEOS_VERSION=3.7.1
GDAL_VERSION=2.3.3
PROJ4_VERSION=4.9.3

sudo yum-config-manager --enable epel
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel

# Compilation work for geos.
mkdir -p "/tmp/geos-${GEOS_VERSION}-build"
cd "/tmp/geos-${GEOS_VERSION}-build"
curl -o "geos-${GEOS_VERSION}.tar.bz2" \
    "http://download.osgeo.org/geos/geos-${GEOS_VERSION}.tar.bz2" \
    && bunzip2 "geos-${GEOS_VERSION}.tar.bz2" \
    && tar xvf "geos-${GEOS_VERSION}.tar"
cd "/tmp/geos-${GEOS_VERSION}-build/geos-${GEOS_VERSION}"
./configure --prefix=/usr/local/geos

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Compilation work for proj4.
mkdir -p "/tmp/proj-${PROJ4_VERSION}-build"
cd "/tmp/proj-${PROJ4_VERSION}-build"
curl -o "proj-${PROJ4_VERSION}.tar.gz" \
    "http://download.osgeo.org/proj/proj-${PROJ4_VERSION}.tar.gz" \
    && tar xfz "proj-${PROJ4_VERSION}.tar.gz"
cd "/tmp/proj-${PROJ4_VERSION}-build/proj-${PROJ4_VERSION}"
./configure --prefix=/usr/local/proj4

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Compilation work for GDAL
mkdir -p "/tmp/gdal-${GDAL_VERSION}-build"
cd "/tmp/gdal-${GDAL_VERSION}-build"
curl -o "gdal-${GDAL_VERSION}.tar.gz" \
    "http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz" \
    && tar xfz "gdal-${GDAL_VERSION}.tar.gz"
cd "/tmp/gdal-${GDAL_VERSION}-build/gdal-${GDAL_VERSION}"
./configure --prefix=/usr/local/gdal \
            --with-static-proj4=/usr/local/proj4 \
            --without-python

# Make in parallel with 2x the number of processors.
make -j $(( 2 * $(cat /proc/cpuinfo | egrep ^processor | wc -l) )) \
 && sudo make install \
 && sudo ldconfig


# Bundle resources.
cd /usr/local/geos
tar zcvf "/tmp/geos-${GEOS_VERSION}.tar.gz" *

cd /usr/local/proj4
tar zcvf "/tmp/proj4-${PROJ4_VERSION}.tar.gz" *

cd /usr/local/gdal
tar zcvf "/tmp/gdal-${GDAL_VERSION}.tar.gz" *

[2] Run the script on an EC2 instance (takes a long time) after which you should have the three bundled files available:

/tmp/geos-3.7.1.tar.gz
/tmp/proj4-4.9.3.tar.gz
/tmp/gdal-2.3.3.tar.gz

[3] Upload these to your S3 bucket.

[4] Add the following to your .ebextensions. Each command tests for the existence of the directory on your instance and downloads and unzips the source as needed.

commands:

  01_install_geos:
    command: |
      sudo wget [s3-url]/geos-3.7.1.tar.gz
      sudo mkdir -p /usr/local/geos
      sudo tar -xvf geos-3.7.1.tar.gz -C /usr/local/geos
      sudo rm -f geos-3.7.1.tar.gz
    test: "[ ! -d /usr/local/geos ]"

  02_install_proj4:
    command: |
      sudo wget [s3-url]/proj4-4.9.3.tar.gz
      sudo mkdir -p /usr/local/proj4
      sudo tar -xvf proj4-4.9.3.tar.gz -C /usr/local/proj4
      sudo rm -f proj4-4.9.3.tar.gz
    test: "[ ! -d /usr/local/proj4 ]"

  03_install_gdal:
    command: |
      sudo wget [s3-url]/gdal-2.3.3.tar.gz
      sudo mkdir -p /usr/local/gdal
      sudo tar -xvf gdal-2.3.3.tar.gz -C /usr/local/gdal
      sudo rm -f gdal-2.3.3.tar.gz
    test: "[ ! -d /usr/local/gdal ]"

[5] Finally, add the following to your .ebextensions/options.config file:

option_settings:
  aws:elasticbeanstalk:application:environment:
    PATH: /usr/local/gdal/bin:$PATH
    LD_LIBRARY_PATH: /usr/local/proj4/lib:/usr/local/gdal/lib:$LD_LIBRARY_PATH
    GDAL_DATA: /usr/local/gdal/share/gdal

Seems like a lot of steps now that I've typed it all out but it has been working reliably for me on EB running Python 3.6 running on 64bit Amazon Linux/2.9.4. Seems much cleaner than bundling the entire /usr/local directory too.

Hey,

I am using your solution with Amazon Linux 2 and Python 3.7 but facing issue in deployment. Can you please help for Amazon Linux 2

essy2017 commented Oct 19, 2020

@rahulkhairnarr What is the issue/can you share error messages? Also, facing issues on EB deployment?

hanayashiki commented Oct 19, 2020

this is very very very slow...
@hanayashiki
It is very dependent on what type of machine you put into play. I'd recommend not going less than a T3.small - if you run a micro or nano it will take forever!

Not forever, just a night's sleep

jmchaves commented Dec 16, 2020 •

edited

Loading

It worked for me with gdal v2.2.0, django 2.1
I had to change the GDAL path: GDAL_LIBRARY_PATH = /usr/local/lib/libgdal.so in settings.py

robmarkcole commented Jan 13, 2021

Attempting install on a sagemaker notebook instance, make -j4 completes fine but the following sudo make installresults in a massive volume of errors like:

/opt/conda/include/unicode/localpointer.h:224:5: warning: identifier ‘noexcept’ is a keyword in C++11 [-Wc++0x-compat]
     LocalPointer(LocalPointer<T> &&src) U_NOEXCEPT : LocalPointerBase<T>(src.ptr) {

cvargas-xbrein commented Dec 20, 2021

Hello, thank you very much for the information I wanted to ask a favor if it is possible to share these 3 files to do a test with sagemaker please

mojodna/README.md

xlab commented Jan 10, 2017 • edited Loading

python1981 commented Jul 4, 2017

nbeuchat commented Apr 2, 2019

nashmaniac commented May 3, 2019

aaronsilverman commented Jun 6, 2019 • edited Loading

dliff commented Jun 28, 2019

leogny commented Sep 30, 2019

dliff commented Oct 9, 2019 • edited Loading

essy2017 commented Jan 9, 2020

speedy250 commented Jan 14, 2020

essy2017 commented Jan 14, 2020

speedy250 commented Jan 15, 2020

speedy250 commented Jan 15, 2020 • edited Loading

serge-gaia commented Feb 13, 2020

speedy250 commented Feb 17, 2020

Zalkota commented Feb 17, 2020 • edited Loading

speedy250 commented Feb 17, 2020

Zalkota commented Feb 17, 2020 • edited Loading

hanayashiki commented Oct 14, 2020

speedy250 commented Oct 19, 2020 • edited Loading

rahulkhairnarr commented Oct 19, 2020

essy2017 commented Oct 19, 2020

hanayashiki commented Oct 19, 2020

jmchaves commented Dec 16, 2020 • edited Loading

robmarkcole commented Jan 13, 2021

cvargas-xbrein commented Dec 20, 2021

xlab commented Jan 10, 2017 •

edited

Loading

aaronsilverman commented Jun 6, 2019 •

edited

Loading

dliff commented Oct 9, 2019 •

edited

Loading

speedy250 commented Jan 15, 2020 •

edited

Loading

Zalkota commented Feb 17, 2020 •

edited

Loading

Zalkota commented Feb 17, 2020 •

edited

Loading

speedy250 commented Oct 19, 2020 •

edited

Loading

jmchaves commented Dec 16, 2020 •

edited

Loading