How can you remove and then restore non index changes using the stash without ever having merge conflicts in git?

Question

I want to be able to run my tests for my project with the current state of the index, ignoring non-committed working changes (I later plan to add this to a pre-commit hook). However, I am having trouble figuring out how to remove and then restore the non index changes in a way that never causes merge conflicts. I need this because it is being run by a script, so it shouldn't alter the repository state when finished.

git stash --include-untracked --keep-index and git stash pop come close, but in many cases it results in merge conflicts, even if no changes where made between the two commands.

For example:

mkdir project; cd project; git init .;

# setup the initial project with file.rb and test.rb
cat > file.rb <<EOF
def func()
  return 42
end
EOF

cat > test.rb <<EOF
#!/usr/bin/env ruby
load './file.rb'
if (func() == 42)
  puts "Tests passed"
  exit 0
else
  puts "Tests failed"
  exit 1
end
EOF

chmod +x test.rb
git add .
git commit -m "Initial commit"

# now change file.rb and add the change
cat > file.rb <<EOF
def func()
  return 10 + 32
end
EOF
git add file.rb

# now make a breaking change to func, and don't add the change
cat > file.rb <<EOF
def func()
  return 20 + 32 # not 42 anymore...
end 
EOF

From here I want to run the tests against the current state of the index, and the restore the uncommitted changes. The expected result is for the tests to pass, as the breaking change wasn't added to the index.

The following commands do NOT work:

git commit --include-untracked --keep-index
./test.rb
git stash pop

The the problem occurs in the git stash pop - a merge conflict occurs.

The only other solution I could think of was to make a temporary commit, then stash the remaining changes, then rollback the commit with git reset --soft HEAD~, then pop the stash. However that is both cumbersome, and I'm not sure how safe that is to run in a pre-commit hook.

Is there a better solution to this problem?

Answer 1

Like you, I run

git stash --keep-index --include-untracked

I can then run tests and so on.

The next part is tricky. These are some things I tried:

• git stash pop can fail with conflicts, which is unacceptable.
• git stash pop --index can fail with conflicts, which is unacceptable.
• git checkout stash -- . applies all tracked changes (good), but also stages them (unacceptable), and does not restore untracked files from the stash (unacceptable). The stash remains (fine -- I can git stash drop).
• git merge --squash --strategy-option=theirs stash can fail with conflicts, which is unacceptable, and even when it doesn't conflict it does not restore untracked files from the stash (unacceptable).
• git stash && git stash pop stash@{1} && git stash pop (trying to apply the changesets in reverse order) can fail with conflicts, which is unacceptable.

But I found a set of commands which does what we want:

# Stash what we actually want to commit
git stash
# Unstash the original dirty tree including any untracked files
git stash pop stash@{1}
# Replace the current index with that from the stash which contains only what we want to commit
git read-tree stash
# Drop the temporary stash of what we want to commit (we have it all in working tree now)
git stash drop

For less output, and condensed into one line:

git stash --quiet && git stash pop --quiet stash@{1} && git read-tree stash && git stash drop --quiet

As far as I'm aware, the only thing this doesn't restore is files which were added in the index and then deleted from the working tree (they'll end up added and present) and files which were renamed in the index and then deleted from the working tree (same outcome). For this reason we need to look for files which match these two cases with a line like git status -z | egrep -z '^[AR]D' | cut -z -c 4- | tr '\0' '\n' before the initial stash, and then loop through and delete them after restoring.

Obviously you should only be running the initial git stash --keep-index --include-untracked if the working tree has any untracked files or unstaged changes. To check for that you can use the test git status --porcelain | egrep --silent '^(\?\?|.[DM])' in your script.

I believe this is better than the existing answers -- it doesn't need any intermediate variables (other than whether the tree was dirty or not, and a record of which files need to be deleted after restoring the stash), has fewer commands and doesn't require garbage collection to be switched off for safety. There are intermediate stashes, but I'd argue this this exactly the kind of thing they're for.

Here's my current pre-commit hook, which does everything mentioned:

#!/bin/sh

# Do we need to tidy up the working tree before tests?
# A --quiet option here doesn't actually suppress the output, hence redirection.
git commit --dry-run >/dev/null
ret=$?
if [ $ret -ne 0 ]; then
    # Nothing to commit, perhaps. Bail with success.
    exit 0
elif git status --porcelain | egrep --silent '^(\?\?|.[DM])'; then
    # There are unstaged changes or untracked files
    dirty=true

    # Remember files which were added or renamed and then deleted, since the
    # stash and read-tree won't restore these
    #
    # We're using -z here to get around the difficulty of parsing
    # - renames (-> appears in the string)
    # - files with spaces or doublequotes (which are doublequoted, but not when
    #   untracked for unknown reasons)
    # We're not trying to store the string with NULs in it in a variable,
    # because you can't do that in a shell script.
    todelete="$(git status -z | egrep -z '^[AR]D' | cut -z -c 4- | tr '\0' '\n')"
else
    dirty=false
fi

if $dirty; then
    # Tidy up the working tree
    git stash --quiet --keep-index --include-untracked
    ret=$?

    # Abort if this failed
    if [ $ret -ne 0 ]; then
        exit $ret
    fi
fi

# Run tests, remember outcome
make precommit
ret=$?

if $dirty; then
    # Restore the working tree and index
    git stash --quiet && git stash pop --quiet stash@{1} && git read-tree stash && git stash drop --quiet
    restore_ret=$?

    # Delete any files which had unstaged deletions
    if [ -n "$todelete" ]; then
        echo "$todelete" | while read file; do
            rm "$file"
        done

        # Abort if this failed
        if [ $restore_ret -ne 0 ]; then
            exit $restore_ret
        fi
    fi
fi

# Exit with the exit status of the tests
exit $ret

Any improvements welcome.

Answer 2

$ git config gc.auto 0   # safety play
$ INDEX=`git write-tree`
$ git add -f .
$ WORKTREE=`git write-tree`
$ git read-tree $INDEX
$ git checkout-index -af
$ git clean -dfx
$ # your tests here
$ git read-tree $WORKTREE
$ git checkout-index -af
$ git clean -dfx
$ git read-tree $INDEX
$ git config --unset gc.auto
$ # you're back.

The git clean manpage for -x rather elliptically suggests this solution

Answer 3

Here is the best solution I have figured out so far. It works in the git hooks pre-commit and commit-msg. I've tested it when there is:

• Only staged changes
• Only unstaged changes
• Both staged and unstaged changes
• No changes

In all three cases it works correctly. The big downside is it is pretty hacky, creating and removing temp commits and stashes.

#!/bin/bash

git commit -m "FOR_COMMIT_MSG_HOOK" -n
commit_status=$?
# git stash save always returns 0, even if it
# failed creating a stash. So compare the new and old git stash list
# output to see if a stash was created
current_stash_count="$(git stash list)"
git stash save -q -u "FOR_COMMIT_MSG_HOOK"
new_stash_count="$(git stash list)"

echo "##### Running tests #####"
# put testing code here


if [[ $commit_status -eq 0 ]]; then
    git reset --soft 'HEAD~'
fi

if [[ "$current_stash_count" != "$new_stash_count" ]]; then
  git stash pop -q
fi

exit $result