How to trigger a build only if changes happen on particular set of files

Question

How do I tell Jenkins/Hudson to trigger a build only for changes on a particular project in my Git tree?

Answer 1

The Git plugin has an option (excluded region) to use regexes to determine whether to skip building based on whether files in the commit match the excluded region regex.

Unfortunately, the stock Git plugin does not have a "included region" feature at this time (1.15). However, someone posted patches on GitHub that work on Jenkins and Hudson that implement the feature you want.

It is a little work to build, but it works as advertised and has been extremely useful since one of my Git trees has multiple independent projects.

https://github.com/jenkinsci/git-plugin/pull/49

Update: The Git plugin (1.16) now has the 'included' region feature.

Answer 2

If you are using a declarative syntax of Jenkinsfile to describe your building pipeline, you can use changeset condition to limit stage execution only to the case when specific files are changed. This is now a standard feature of Jenkins and does not require any additional configruation/software.

stages {
    stage('Nginx') {
        when { changeset "nginx/*"}
        steps {
            sh "make build-nginx"
            sh "make start-nginx"
        }
    }
}

You can combine multiple conditions using anyOf or allOf keywords for OR or AND behaviour accordingly:

when {
    anyOf {
        changeset "nginx/**"
        changeset "fluent-bit/**"
    }
}
steps {
    sh "make build-nginx"
    sh "make start-nginx"
}

Answer 3

Basically, you need two jobs. One to check whether files changed and one to do the actual build:

Job #1

This should be triggered on changes in your Git repository. It then tests whether the path you specify ("src" here) has changes and then uses Jenkins' CLI to trigger a second job.

export JENKINS_CLI="java -jar /var/run/jenkins/war/WEB-INF/jenkins-cli.jar"
export JENKINS_URL=http://localhost:8080/
export GIT_REVISION=`git rev-parse HEAD`
export STATUSFILE=$WORKSPACE/status_$BUILD_ID.txt

# Figure out, whether "src" has changed in the last commit
git diff-tree --name-only HEAD | grep src

# Exit with success if it didn't
$? || exit 0

# Trigger second job
$JENKINS_CLI build job2 -p GIT_REVISION=$GIT_REVISION -s

Job #2

Configure this job to take a parameter GIT_REVISION like so, to make sure you're building exactly the revision the first job chose to build.

Answer 4

While this doesn't affect single jobs, you can use this script to ignore certain steps if the latest commit did not contain any changes:

/*
 * Check a folder if changed in the latest commit.
 * Returns true if changed, or false if no changes.
 */
def checkFolderForDiffs(path) {
    try {
        // git diff will return 1 for changes (failure) which is caught in catch, or
        // 0 meaning no changes 
        sh "git diff --quiet --exit-code HEAD~1..HEAD ${path}"
        return false
    } catch (err) {
        return true
    }
}

if ( checkFolderForDiffs('api/') ) {
    //API folder changed, run steps here
}

Answer 5

You can use Generic Webhook Trigger Plugin for this.

With a variable like changed_files and expression $.commits[*].['modified','added','removed'][*].

You can have a filter text like $changed_files and filter regexp like "folder/subfolder/[^"]+?" if folder/subfolder is the folder that should trigger builds.

Answer 6

If the logic for choosing the files is not trivial, I would trigger script execution on each change and then write a script to check if indeed a build is required, then triggering a build if it is.

Answer 7

I answered this question in another post:

How to get list of changed files since last build in Jenkins/Hudson

#!/bin/bash

set -e

job_name="whatever"
JOB_URL="http://myserver:8080/job/${job_name}/"
FILTER_PATH="path/to/folder/to/monitor"

python_func="import json, sys
obj = json.loads(sys.stdin.read())
ch_list = obj['changeSet']['items']
_list = [ j['affectedPaths'] for j in ch_list ]
for outer in _list:
  for inner in outer:
    print inner
"

_affected_files=`curl --silent ${JOB_URL}${BUILD_NUMBER}'/api/json' | python -c "$python_func"`

if [ -z "`echo \"$_affected_files\" | grep \"${FILTER_PATH}\"`" ]; then
  echo "[INFO] no changes detected in ${FILTER_PATH}"
  exit 0
else
  echo "[INFO] changed files detected: "
  for a_file in `echo "$_affected_files" | grep "${FILTER_PATH}"`; do
    echo "    $a_file"
  done;
fi;

You can add the check directly to the top of the job's exec shell, and it will exit 0 if no changes are detected... Hence, you can always poll the top level for check-in's to trigger a build.

Answer 8

I wrote this script to skip or execute tests if there are changes:

#!/bin/bash

set -e -o pipefail -u

paths=()
while [ "$1" != "--" ]; do
    paths+=( "$1" ); shift
done
shift

if git diff --quiet --exit-code "${BASE_BRANCH:-origin/master}"..HEAD ${paths[@]}; then
    echo "No changes in ${paths[@]}, skipping $@..." 1>&2
    exit 0
fi
echo "Changes found in ${paths[@]}, running $@..." 1>&2

exec "$@"

So you can do something like:

./scripts/git-run-if-changed.sh cmd vendor go.mod go.sum fixtures/ tools/ -- go test

Answer 9

For Bitbucket Repository users (and other people using Source-Control Management hosts whom webhook payload doesn't seem to indicate file changes).

It seems that Git plugin 'included regions' fail whatever I do, and always trigger the job. My setup is Jenkins 2.268, run in a Docker container, and it was purgatory to find a correct way to achieve building jobs depending on file changes, but here's one below.

Required Jenkins plugins:

• Groovy
• Bitbucket (or, if you're on another SCM host: a plugin which can trigger builds on this host's webhooks)

Create a new Freestyle job called 'Switch':

1. Source Code Management: indicate your SCM information (make sure the 'Branches to build' are the right ones.
2. Build triggers > Build when a change is pushed to Bitbucket: checked
3. Build step > Execute system Groovy script (not just Execute Groovy script!), leaving Use Groovy sandbox unchecked.

The script:

import jenkins.*;
import jenkins.model.*;

// CONFIGURATION
// Links between changed file patterns and job names to build
def jobsByPattern = [
  "/my-project/": "my-project-job",
  "my-super-project/":"super-job",
]

// Listing changes files since last build
def changeLogSets = build.changeSets
def changedFiles = []
for (int i = 0; i < changeLogSets.size(); i++) {
   def entries = changeLogSets[i].items
   for (int j = 0; j < entries.length; j++) {
    def entry = entries[j]
    def files = new ArrayList(entry.affectedFiles)
    for (int k = 0; k < files.size(); k++) {
       def file = files[k]
       changedFiles.add(file.path)
    }
  }
}

// Listing ad hoc jobs to build
jobsToBuild = [:] // declare an empty map
for(entry in jobsByPattern ) {
  def pattern = entry.key
  println "Check pattern: $pattern"
  for (int i = 0; i < changedFiles.size(); i++) {
    def file = changedFiles[i]
    println "Check file: $file"
    if( file.contains( pattern ) ) {
      def jobName = entry.value
      jobsToBuild[ jobName ] = true
      break
    }
  }
}

// Building appropriate jobs
jobsToBuild.each{
  def jobName = it.key
  println "$jobName must be built!"
  def job = Jenkins.instance.getJob(jobName)
  def cause = new hudson.model.Cause.UpstreamCause(build)
  def causeAction = new hudson.model.CauseAction(cause)
  Jenkins.instance.queue.schedule(job, 0, causeAction)
}

I believe this method can handle multiple commits since the last build, so it seems to answer the need. Any suggestion of enhancement welcomed.