I need the timestamps of files on my local and on my server to be in sync. This is accomplished with Subversion by setting use-commit-times=true in the config so that the last modified of each file is when it was committed.
Each time I clone my repository, I want the timestamps of files to reflect when they were last changed in the remote repository, not when I cloned the repo.
Is there any way to do this with git?
If, however you really want to use commit times for timestamps when checking out then try using this script and place it (as executable) in the file $GIT_DIR/.git/hooks/post-checkout:
#!/bin/sh -e
OS=${OS:-`uname`}
old_rev="$1"
new_rev="$2"
get_file_rev() {
git rev-list -n 1 "$new_rev" "$1"
}
if [ "$OS" = 'Linux' ]
then
update_file_timestamp() {
file_time=`git show --pretty=format:%ai --abbrev-commit "$(get_file_rev "$1")" | head -n 1`
touch -d "$file_time" "$1"
}
elif [ "$OS" = 'FreeBSD' ]
then
update_file_timestamp() {
file_time=`date -r "$(git show --pretty=format:%at --abbrev-commit "$(get_file_rev "$1")" | head -n 1)" '+%Y%m%d%H%M.%S'`
touch -h -t "$file_time" "$1"
}
else
echo "timestamp changing not implemented" >&2
exit 1
fi
IFS=`printf '\t\n\t'`
git ls-files | while read -r file
do
update_file_timestamp "$file"
done
Note however, that this script will cause quite a large delay for checking out large repositories (where large means large amount of files, not large file sizes).
UPDATE: My solution is now packaged into Debian/Ubuntu/Mint, Fedora, Gentoo and possibly other distros:
https://github.com/MestreLion/git-tools#install
sudo apt install git-restore-mtime # Debian/Ubuntu/Mint
yum install git-tools # Fedora/ RHEL / CentOS
emerge dev-vcs/git-tools # Gentoo
IMHO, not storing timestamps (and other metadata like permissions and ownership) is a big limitation of git.
Linus' rationale of timestamps being harmful just because it "confuses make" is lame:
make clean is enough to fix any problems.
Applies only to projects that use make, mostly C/C++. It is completely moot for scripts like Python, Perl, or documentation in general.
There is only harm if you apply the timestamps. There would be no harm in storing them in repo. Applying them could be a simple --with-timestamps option for git checkout and friends (clone, pull etc), at the user's discretion.
Both Bazaar and Mercurial stores metadata. Users can apply them or not when checking out. But in git, since original timestamps are not even available in the repo, there is no such option.
So, for a very small gain (not having to re-compile everything) that is specific to a subset of projects, git as a general DVCS was crippled, some information from about files is lost, and, as Linus said, it's INFEASIBLE to do it now. Sad.
That said, may I offer 2 approaches?
1 - http://repo.or.cz/w/metastore.git , by David Härdeman. Tries to do what git should have done in the first place: stores metadata (not only timestamps) in the repo when commiting (via pre-commit hook), and re-applies them when pulling (also via hooks).
2 - My humble version of a script I used before for generating release tarballs. As mentioned in other answers, the approach is a little different: to apply for each file the timestamp of the most recent commit where the file was modified.
Below is a really bare-bones version of the script, as a proof-of-concept, on Python 2.7. For actual usage I strongly recommend the full version above:
#!/usr/bin/env python
# Bare-bones version. Current dir must be top-level of work tree.
# Usage: git-restore-mtime-bare [pathspecs...]
# By default update all files
# Example: to only update only the README and files in ./doc:
# git-restore-mtime-bare README doc
import subprocess, shlex
import sys, os.path
filelist = set()
for path in (sys.argv[1:] or [os.path.curdir]):
if os.path.isfile(path) or os.path.islink(path):
filelist.add(os.path.relpath(path))
elif os.path.isdir(path):
for root, subdirs, files in os.walk(path):
if '.git' in subdirs:
subdirs.remove('.git')
for file in files:
filelist.add(os.path.relpath(os.path.join(root, file)))
mtime = 0
gitobj = subprocess.Popen(shlex.split('git whatchanged --pretty=%at'),
stdout=subprocess.PIPE)
for line in gitobj.stdout:
line = line.strip()
if not line: continue
if line.startswith(':'):
file = line.split('\t')[-1]
if file in filelist:
filelist.remove(file)
#print mtime, file
os.utime(file, (mtime, mtime))
else:
mtime = long(line)
# All files done?
if not filelist:
break
Performance is pretty impressive, even for monster projects wine, git or even the linux kernel:
bash
# 0.27 seconds
# 5,750 log lines processed
# 62 commits evaluated
# 1,155 updated files
git
# 3.71 seconds
# 96,702 log lines processed
# 24,217 commits evaluated
# 2,495 updated files
wine
# 13.53 seconds
# 443,979 log lines processed
# 91,703 commits evaluated
# 6,005 updated files
linux kernel
# 59.11 seconds
# 1,484,567 log lines processed
# 313,164 commits evaluated
# 40,902 updated files
I am not sure this would be appropriate for a DVCS (as in "Distributed" VCS)
The huge discussion had already took place in 2007 (see this thread)
And some of Linus's answer were not too keen on the idea. Here is one sample:
I'm sorry. If you don't see how it's WRONG to set a datestamp back to something that will make a simple "make" miscompile your source tree, I don't know what defintiion of "wrong" you are talking about.
It's WRONG.
It's STUPID.
And it's totally INFEASIBLE to implement.
(Note: small improvement: after a checkout, timestamps of up-to-date files are no longer modified (Git 2.2.2+, January 2015): "git checkout - how can I maintain timestamps when switching branches?".)
The long answer was:
I think you're much better off just using multiple repositories instead, if this is something common.
Messing with timestamps is not going to work in general. It's just going to guarantee you that "make" gets confused in a really bad way, and does not recompile enough instead of recompiling too much.
Git does make it possible to do your "check the other branch out" thing very easily, in many different ways.
You could create some trivial script that does any of the following (ranging from the trivial to the more exotic):
- just create a new repo:
git clone old new cd new git checkout origin/<branch>
and there you are. The old timestamps are fine in your old repo, and you can work (and compile) in the new one, without affectign the old one at all.
Use the flags "-n -l -s" to "git clone" to basically make this instantaneous. For lots of files (eg big repos like the kernel), it's not going to be as fast as just switching branches, but havign a second copy of the working tree can be quite powerful.
- do the same thing with just a tar-ball instead, if you want to
git archive --format=tar --prefix=new-tree/ <branchname> | (cd .. ; tar xvf -)
which is really quite fast, if you just want a snapshot.
- get used to "
git show", and just look at individual files.
This is actually really useful at times. You just do
git show otherbranch:filename
in one xterm window, and look at the same file in your current branch in another window. In particular, this should be trivial to do with scriptable editors (ie GNU emacs), where it should be possible to basically have a whole "dired mode" for other branches within the editor, using this. For all I know, the emacs git mode already offers something like this (I'm not an emacs user)
- and in the extreme example of that "virtual directory" thing, there was at least somebody working on a git plugin for FUSE, ie you could literally just have virtual directories showing all your branches.
and I'm sure any of the above are better alternatives than playing games with file timestamps.
Linus
I took Giel's answer and instead of using a post-commit hook script, worked it into my custom deployment script.
Update: I've also removed one | head -n following @eregon's suggestion, and added support for files with spaces in them:
# Adapted to use HEAD rather than the new commit ref
get_file_rev() {
git rev-list -n 1 HEAD "$1"
}
# Same as Giel's answer above
update_file_timestamp() {
file_time=`git show --pretty=format:%ai --abbrev-commit "$(get_file_rev "$1")" | head -n 1`
sudo touch -d "$file_time" "$1"
}
# Loop through and fix timestamps on all files in our CDN directory
old_ifs=$IFS
IFS=$'\n' # Support files with spaces in them
for file in $(git ls-files | grep "$cdn_dir")
do
update_file_timestamp "${file}"
done
IFS=$old_ifs
we were forced to invent yet another solution, because we needed specifically modification times and not commit times, and the solution also had to be portable (i.e. getting python working in windows's git installations really is not a simple task) and fast. It resembles the David Hardeman's solution, which I decided not to use because of lack of documentation (from repository I was not able to get idea what exactly his code does).
This solution stores mtimes in a file .mtimes in git repository, updates them accordingly on commits (jsut selectively the mtimes of staged files) and applies them on checkout. It works even with cygwin/mingw versions of git (but you may need to copy some files from standard cygwin into git's folder)
The solution consists of 3 files:
pre-commit:
#!/bin/bash
mtimestore -s
git add .mtimes
post-checkout
#!/bin/bash
mtimestore -r
mtimestore - bash:
#!/bin/bash
function usage
{
echo "Usage: mtimestore (-a|-s|-r)"
echo "Option Meaning"
echo " -a save-all - saves state of all files in a git repository"
echo " -s save - saves mtime of all staged files of git repository"
echo " -r restore - touches all files saved in .mtimes file"
exit 1
}
function echodate
{
echo "$(stat -c %Y "$1")|$1" >> .mtimes
}
IFS=$'\n'
while getopts ":sar" optname
do
case "$optname" in
"s")
echo "saving changes of staged files to file .mtimes"
if [ -f .mtimes ]
then
mv .mtimes .mtimes_tmp
pattern=".mtimes"
for str in $(git diff --name-only --staged)
do
pattern="$pattern\|$str"
done
cat .mtimes_tmp | grep -vh "|\($pattern\)\b" >> .mtimes
else
echo "warning: file .mtimes does not exist - creating new"
fi
for str in $(git diff --name-only --staged)
do
echodate "$str"
done
rm .mtimes_tmp 2> /dev/null
;;
"a")
echo "saving mtimes of all files to file .mtimes"
rm .mtimes 2> /dev/null
for str in $(git ls-files)
do
echodate "$str"
done
;;
"r")
echo "restorim dates from .mtimes"
if [ -f .mtimes ]
then
cat .mtimes | while read line
do
timestamp=$(date -d "1970-01-01 ${line%|*} sec GMT" +%Y%m%d%H%M.%S)
touch -t $timestamp "${line##*|}"
done
else
echo "warning: .mtimes not found"
fi
;;
":")
usage
;;
*)
usage
;;
esac
mtimestore - c++
#include <time.h>
#include <utime.h>
#include <sys/stat.h>
#include <iostream>
#include <cstdlib>
#include <fstream>
#include <string>
#include <cerrno>
#include <cstring>
#include <sys/types.h>
#include <ctime>
#include <map>
void changedate(int time, const char* filename)
{
try
{
struct utimbuf new_times;
struct stat foo;
stat(filename, &foo);
new_times.actime = foo.st_atime;
new_times.modtime = time;
utime(filename, &new_times);
}
catch(...)
{}
}
bool parsenum(int& num, char*& ptr)
{
num = 0;
if(!isdigit(*ptr))
return false;
while(isdigit(*ptr))
{
num = num*10 + (int)(*ptr) - 48;
ptr++;
}
return true;
}
//splits line into numeral and text part - return numeral into time and set ptr to the position where filename starts
bool parseline(const char* line, int& time, char*& ptr)
{
if(*line == '\n' || *line == '\r')
return false;
time = 0;
ptr = (char*)line;
if( parsenum(time, ptr))
{
ptr++;
return true;
}
else
return false;
}
//replace \r and \n (otherwise is interpretted as part of filename)
void trim(char* string)
{
char* ptr = string;
while(*ptr != '\0')
{
if(*ptr == '\n' || *ptr == '\r')
*ptr = '\0';
ptr++;
}
}
void help()
{
std::cout << "version: 1.4" << std::endl;
std::cout << "usage: mtimestore <switch>" << std::endl;
std::cout << "options:" << std::endl;
std::cout << " -a saves mtimes of all git-versed files into .mtimes file (meant to be done on intialization of mtime fixes)" << std::endl;
std::cout << " -s saves mtimes of modified staged files into .mtimes file(meant to be put into pre-commit hook)" << std::endl;
std::cout << " -r restores mtimes from .mtimes file (that is meant to be stored in repository server-side and to be called in post-checkout hook)" << std::endl;
std::cout << " -h show this help" << std::endl;
}
void load_file(const char* file, std::map<std::string,int>& mapa)
{
std::string line;
std::ifstream myfile (file, std::ifstream::in);
if(myfile.is_open())
{
while ( myfile.good() )
{
getline (myfile,line);
int time;
char* ptr;
if( parseline(line.c_str(), time, ptr))
{
if(std::string(ptr) != std::string(".mtimes"))
mapa[std::string(ptr)] = time;
}
}
myfile.close();
}
}
void update(std::map<std::string, int>& mapa, bool all)
{
char path[2048];
FILE *fp;
if(all)
fp = popen("git ls-files", "r");
else
fp = popen("git diff --name-only --staged", "r");
while(fgets(path, 2048, fp) != NULL)
{
trim(path);
struct stat foo;
int err = stat(path, &foo);
if(std::string(path) != std::string(".mtimes"))
mapa[std::string(path)]=foo.st_mtime;
}
}
void write(const char * file, std::map<std::string, int>& mapa)
{
std::ofstream outputfile;
outputfile.open(".mtimes", std::ios::out);
for(std::map<std::string, int>::iterator itr = mapa.begin(); itr != mapa.end(); ++itr)
{
if(*(itr->first.c_str()) != '\0')
{
outputfile << itr->second << "|" << itr->first << std::endl;
}
}
outputfile.close();
}
int main(int argc, char *argv[])
{
if(argc >= 2 && argv[1][0] == '-')
{
switch(argv[1][1])
{
case 'r':
{
std::cout << "restoring modification dates" << std::endl;
std::string line;
std::ifstream myfile (".mtimes");
if (myfile.is_open())
{
while ( myfile.good() )
{
getline (myfile,line);
int time, time2;
char* ptr;
parseline(line.c_str(), time, ptr);
changedate(time, ptr);
}
myfile.close();
}
}
break;
case 'a':
case 's':
{
std::cout << "saving modification times" << std::endl;
std::map<std::string, int> mapa;
load_file(".mtimes", mapa);
update(mapa, argv[1][1] == 'a');
write(".mtimes", mapa);
}
break;
default:
help();
return 0;
}
} else
{
help();
return 0;
}
return 0;
}
more info may be found here https://github.com/kareltucek/git-mtime-extension some out of date information are at http://www.ktweb.cz/blog/index.php?page=page&id=116
//edit - c++ version updated:
//edit see github for up-to-date version
The following script incorporates the -n 1 and HEAD suggestions, works in most non-Linux environments (like Cygwin), and can be run on a checkout after the fact:
#!/bin/bash -e
OS=${OS:-`uname`}
get_file_rev() {
git rev-list -n 1 HEAD "$1"
}
if [ "$OS" = 'FreeBSD' ]
then
update_file_timestamp() {
file_time=`date -r "$(git show --pretty=format:%at --abbrev-commit "$(get_file_rev "$1")" | head -n 1)" '+%Y%m%d%H%M.%S'`
touch -h -t "$file_time" "$1"
}
else
update_file_timestamp() {
file_time=`git show --pretty=format:%ai --abbrev-commit "$(get_file_rev "$1")" | head -n 1`
touch -d "$file_time" "$1"
}
fi
OLD_IFS=$IFS
IFS=$'\n'
for file in `git ls-files`
do
update_file_timestamp "$file"
done
IFS=$OLD_IFS
git update-index --refresh
Assuming you named the above script /path/to/templates/hooks/post-checkout and/or /path/to/templates/hooks/post-update, you can run it on an existing repository via:
git clone git://path/to/repository.git
cd repository
/path/to/templates/hooks/post-checkout
This solution should run pretty quickly. It sets atimes to committer times and mtimes to author times. It uses no modules so should be reasonably portable.
#!/usr/bin/perl
# git-utimes: update file times to last commit on them
# Tom Christiansen <tchrist@perl.com>
use v5.10; # for pipe open on a list
use strict;
use warnings;
use constant DEBUG => !!$ENV{DEBUG};
my @gitlog = (
qw[git log --name-only],
qq[--format=format:"%s" %ct %at],
@ARGV,
);
open(GITLOG, "-|", @gitlog) || die "$0: Cannot open pipe from `@gitlog`: $!\n";
our $Oops = 0;
our %Seen;
$/ = "";
while (<GITLOG>) {
next if /^"Merge branch/;
s/^"(.*)" // || die;
my $msg = $1;
s/^(\d+) (\d+)\n//gm || die;
my @times = ($1, $2); # last one, others are merges
for my $file (split /\R/) { # I'll kill you if you put vertical whitespace in our paths
next if $Seen{$file}++;
next if !-f $file; # no longer here
printf "atime=%s mtime=%s %s -- %s\n",
(map { scalar localtime $_ } @times),
$file, $msg,
if DEBUG;
unless (utime @times, $file) {
print STDERR "$0: Couldn't reset utimes on $file: $!\n";
$Oops++;
}
}
}
exit $Oops;
I saw some requests for a Windows version, so here it is. Create the following two files:
C:\Program Files\Git\mingw64\share\git-core\templates\hooks\post-checkout
#!C:/Program\ Files/Git/usr/bin/sh.exe
exec powershell.exe -NoProfile -ExecutionPolicy Bypass -File "./$0.ps1"
C:\Program Files\Git\mingw64\share\git-core\templates\hooks\post-checkout.ps1
[string[]]$changes = &git whatchanged --pretty=%at
$mtime = [DateTime]::Now;
[string]$change = $null;
foreach($change in $changes)
{
if($change.Length -eq 0) { continue; }
if($change[0] -eq ":")
{
$parts = $change.Split("`t");
$file = $parts[$parts.Length - 1];
if([System.IO.File]::Exists($file))
{
[System.IO.File]::SetLastWriteTimeUtc($file, $mtime);
}
}
else
{
#get timestamp
$mtime = [DateTimeOffset]::FromUnixTimeSeconds([Int64]::Parse($change)).DateTime;
}
}
This utilizes git whatchanged, so it runs through all the files in one pass instead of calling git for each file.
Here is a Go program:
import "bufio"
import "log"
import "os/exec"
func check(e error) {
if e != nil {
log.Fatal(e)
}
}
func popen(name string, arg ...string) (*bufio.Scanner, error) {
cmd := exec.Command(name, arg...)
pipe, e := cmd.StdoutPipe()
if e != nil {
return nil, e
}
return bufio.NewScanner(pipe), cmd.Start()
}
import "os"
import "strconv"
import "time"
func main() {
gitLs, e := popen("git", "ls-files")
check(e)
files := map[string]bool{}
for gitLs.Scan() {
files[gitLs.Text()] = true
}
gitLog, e := popen(
"git", "log", "-m",
"--name-only", "--relative", "--pretty=format:%ct", ".",
)
check(e)
for len(files) > 0 {
gitLog.Scan()
sec, e := strconv.ParseInt(gitLog.Text(), 10, 64)
check(e)
unix := time.Unix(sec, 0)
for gitLog.Scan() {
name := gitLog.Text()
if name == "" {
break
}
if ! files[name] {
continue
}
os.Chtimes(name, unix, unix)
delete(files, name)
}
}
}
It is similar to this answer. It builds up a file list like that answer,
but it builds from git ls-files instead of just looking in the working
directory. This solves the problem of excluding .git, and it also solves the
problem of untracked files. Also, that answer fails if the last commit of a
file was a merge commit, which I solved with git log -m. Like the other
answer, will stop once all files are found, so it doesnt have to read all the
commits. For example with git/git, as of this posting it only had to read
182 commits. Also it ignores old files from the history as needed, and it
wont touch a file that has already been touched. Finally, it is faster than the
other solution. Results with git/git repo:
PS C:\git> Measure-Command { ..\git-touch }
Milliseconds : 470