CircleCi - only build features that has changed

buildWhenAffected.sh

##################
### config.yml ###
##################

version: 2
jobs:
    build:
        docker:
            - image: circleci/python:3.6
        steps:
            - checkout
            - run:
                command: |
                    .circleci/commit_check.sh



#######################
### commit_check.sh ###
#######################

set -e

# latest commit
LATEST_COMMIT=$(git rev-parse HEAD)

# latest commit where path/to/folder1 was changed
FOLDER1_COMMIT=$(git log -1 --format=format:%H --full-diff path/to/folder1)

# latest commit where path/to/folder2 was changed
FOLDER2_COMMIT=$(git log -1 --format=format:%H --full-diff path/to/folder2)

if [ $FOLDER1_COMMIT = $LATEST_COMMIT ];
    then
        echo "files in folder1 has changed"
        .circleci/do_something.sh
elif [ $FOLDER2_COMMIT = $LATEST_COMMIT ];
    then
        echo "files in folder2 has changed"
        .circleci/do_something_else.sh
else
     echo "no folders of relevance has changed"
     exit 0;
fi

Looks like CircleCI used with pipelines disables CIRCLE_COMPARE_URL variable. This might get handy then https://github.com/iynere/compare-url

@ondrejsevcik With pipelines there are those now, which are even better: https://circleci.com/docs/2.0/pipeline-variables/#pipeline-values. Thank you everyone for the ideas discussed here.

The original solution does not work correctly if you have multiple commits in a branch. The scenario is: edit a file in the path you are checking, commit it, then make a change to another path, commit it, then push both commits to Circleci (if you have it set up to only run for the last commit in a branch). It will check the last commit which does not have the changes to the path you're checking.

In order to know if something changed when there are multiple commits, you need to have a base commit/sha to compare to (called merge base). You could assume all changes eventually get to origin/main (this may not true for all cases, like a shared topic/feature branches), and use something like this bash script:

#!/bin/bash

REF=HEAD
SINCE=origin/main
DIR_TO_CHECK=path/to/your-directory

MERGE_BASE=$(git merge-base ${SINCE} ${REF})
FILES_CHANGED=$(git diff --name-only ${MERGE_BASE}..${REF} -- ${DIR_TO_CHECK})
printf "Files changed:\n${FILES_CHANGED}\n"

if [[ -n $FILES_CHANGED ]]; then
  echo "Found changes"
else
  echo "No changes"
fi

Circle CI has since came up with their own recommended way to do this:
https://circleci.com/docs/2.0/configuration-cookbook/?section=examples-and-guides#execute-specific-workflows-or-steps-based-on-which-files-are-modified

The circlci example at https://circleci.com/docs/2.0/configuration-cookbook/?section=examples-and-guides#execute-specific-workflows-or-steps-based-on-which-files-are-modified returns below error:

[#/jobs/check-updated-files] 0 subschemas matched instead of one
1. [#/jobs/check-updated-files] only 0 subschema matches out of 2
|   1. [#/jobs/check-updated-files] no subschema matched out of the total 2 subschemas
|   |   1. [#/jobs/check-updated-files] 0 subschemas matched instead of one
|   |   |   1. [#/jobs/check-updated-files] expected type: Mapping, found: Sequence
|   |   |   |   SCHEMA:
|   |   |   |     type: object
|   |   |   |   INPUT:
|   |   |   |     - filter:
|   |   |   |         mapping: |
|   |   |   |           service1/.* run-build-service-1-job true
|   |   |   |           service2/.* run-build-service-2-job true
|   |   |   |         base-revision: master
|   |   |   |         config-path: .circleci/config.yml

i get the same ^