Skip to content

Instantly share code, notes, and snippets.

View phaustin's full-sized avatar

Philip Austin phaustin

61 followers · 6 following
  • University of British Columbia
  • Vancouver, BC Canada
View GitHub Profile
@phaustin
phaustin / debugging_conda.md
Last active January 24, 2019 18:39
debugging conda

@dopplershift, a good way to take advantage of the existing recipe work for a package is to add exit 1 to the end of build.sh, then when the build fails, all the artefacts are in place for you and a script is placed in the work directory that you can source and that script will activate the envs and set all the env vars needed to allow you to hack further. A good tip is to use -m conda_build_config-dbg.yaml so that things are built with reasonable debugging flags (pointing at https://github.com/AnacondaRecipes/aggregate/blob/master/conda_build_config-dbg.yaml => I'm about to push a small fix/update for this file though)

@abellgithub for macOS, sysroots are done a bit differently because clang and llvm are messed up in that regard. We added CONDA_BUILD_SYSROOT env. var. This is documented as we've significantly deviated from Apple's way If you are using conda-forge's python and have the compilers installed in your active env. and have CONDA_BUILD_SYSROOT set to point to a good, old macOS SDK then it should wo

@phaustin
phaustin / pangeo_docs.md
Last active January 24, 2019 05:17
docs for pangeo

@martindurant, although i've been using dask daily for a while this was my first time digging deep on a large cluster and pushing to s3. There's a lot to learn and while I don't want to hold the tools responsible for my ignorance any time tools can smooth my path i'm enormously grateful. I experienced a large stumbling block first writing with parquet. As I was very familiar with parquet locally this was somewhat easier to figure out although it was complicated by other memory issues (dask/dask#4410). Time passed and I switched to xarray and zarr. A whole new stack to get my head around. The particular error message lead me to believe I was configuring zarr incorrectly although i now understand why it's just a permissions error. I found myself switching between a few documentation pages:

http://xarray.pydata.org/en/stable/io.html#cloud-storage-buckets;

https://zarr.readthedocs.io/en/stable/tutorial.html#distributed-cloud-storage;

http://docs.dask.org/en/latest/remote-data-services.html#optional-paramete

@phaustin
phaustin / post-commit
Created January 2, 2019 15:54
Pre and post commit hooks to automatically generate production ready css and add it to the current commit. git add doesn't add the files to the current commit so a .commit file is created to keep track and execute an amending commit in a post-commit hook
#!/bin/sh
#
# To enable this hook place this file in .git/hooks and make it executable.
if [ -a .commit ]
then
rm .commit
git commit --amend -C HEAD --no-verify
fi
exit
@phaustin
phaustin / git_tags.md
Last active December 24, 2018 16:07
git tags in code

(https://github.com/pybind/python_example/blob/master/conda.recipe/meta.yaml)

Does it provide meaningful tags?
# Basic versioneer, I don't want to run the setup.py before things are required
{% if environ.get('GIT_DESCRIBE_NUMBER') == '0' %}
{% set version = environ.get('GIT_DESCRIBE_TAG') %}
{% else %}
{% set version = environ.get('GIT_DESCRIBE_TAG', '') + '+' + environ.get('GIT_DESCRIBE_NUMBER', '') %}
{% endif %}
@phaustin
phaustin / conda_script.yaml
Created December 21, 2018 13:22
conda script example
Marius van Niekerk @mariusvniekerk 05:14
script:
- {{ PYTHON }} -m pip install . --no-deps -vv
- chmod +x *.py # [unix]
- cp *.py ${PREFIX}/bin/ # [unix]
@phaustin
phaustin / conda_build_config.yaml
Created December 20, 2018 14:43
conda build example for xeus-python
channel_sources:
- conda-forge/label/gcc7,defaults # [unix]
- conda-forge,defaults # [win]
channel_targets:
- conda-forge gcc7 # [unix]
- conda-forge main # [win]
cxx_compiler:
- gxx # [linux]
- clangxx # [osx]
@phaustin
phaustin / headless_xserver.txt
Created December 20, 2018 14:36
headless xserver for testing
Running a GUI's testsuite in headless mode for your regular CI service in two steps:
1) apt-get install xvfb
2) xvfb-run --server-args="-screen 0 1024x768x24" python3 -m unittest discover
@phaustin
phaustin / black_ci_links.txt
Created December 14, 2018 14:32
how to call black in ci
@phaustin
phaustin / ci_conda_forge.yaml
Created December 14, 2018 02:33
ci for conda-forge
build_number_decrement:
- '0'
c_compiler:
- gcc
channel_sources:
- conda-forge/label/gcc7,defaults
channel_targets:
- conda-forge gcc7
cxx_compiler:
- gxx
@phaustin
phaustin / pybind11_embed.cpp
Created December 13, 2018 19:04
pybind 11 embed
#include <pybind11/embed.h>
#include <pybind11/pybind11.h>
int main(int argc, char *argv[]) {
pybind11::scoped_interpreter x{};
pybind11::object y = pybind11::none{};
y = pybind11::int_(5);
y = pybind11::none{};
// prints 1
printf("%d\n", y.is(pybind11::none{}));
return 0;