Move Git LFS tracked files under regular Git

Question

I have a project where I stored video files with Git LFS. Now I ran into some complications with my build server that doesn't yet support Git LFS. As it's an external service, I can't really affect the build process, and thus would want to move the files from under Git LFS back to "regular" Git. I managed to untrack the file types with git lfs untrack '<file-type>' but git lfs ls-files still gives a list of the files previously added.

I imagine I could remove the files, push the changes and then manually re-add them, but is this really the recommended way of doing things?

score 113 · Accepted Answer · edited Mar 14 '19 at 19:58

113

I have just recently run into this problem where assets were accidentally added to git-lfs on one branch that shouldn't have been. My solution was:

git lfs untrack '<file-type>'
git rm --cached '<file-type>'
git add '<file-type>'
git commit -m "restore '<file-type>' to git from lfs"

The result is a rewrite of the git-lfs oid sha256 pointers with the standard file contents.

(Edit 2019-03): The accepted answer was changed to provide an easy solution for simpler cases. See also the edits in the answer by VonC for alternate solutions in case you have a more complex case on hand.

edited Mar 14 '19 at 19:58

Olli Niskanen

1,155
1
9
18

answered Jan 31 '17 at 15:36

mred

1,254
1
9
5

6

With `.gitattributes` is set up correctly to track the files I want, this solution worked beautifully. Much simpler than some of the other solutions posted. – Josh Rickert May 17 '17 at 21:15
1

This worked wonderfully. Should be listed as the true answer. – Aditya May 20 '18 at 12:11
2

worked for me in '18. tip for others: 'file-type' was '**.jpg' – oligofren Dec 28 '18 at 17:19
1

Based on the upvotes, I'll change this to the accepted answer. – Olli Niskanen Mar 13 '19 at 11:40
3

@OlliNiskanen while it is gratifying to be marked as the accepted answer, I feel it important to note that while this is a good and simple solution to the specific question asked, for more complex cases readers should consult the batching methods suggested in the edits by tstephens619 and ttaylorr to VonC's answer. – mred Mar 14 '19 at 13:22
2

@mred, Agreed. I suggested an edit that links to the previous accepted answer. This answer was pretty buried and people found it helpful, so I feel this should stay as the first thing visitors see. – Olli Niskanen Mar 14 '19 at 17:34

Tom Hebb · Answer 2 · 2019-08-02T18:54:37.400

As of Git 2.16 (released Jan 17th, 2018), you can do this easily with the --renormalize flag of git add:

git lfs untrack '<pattern>'
git add --renormalize .
git commit -m 'Restore file contents that were previously in LFS'

From Git's documentation:

--renormalize: Apply the "clean" process freshly to all tracked files to forcibly add them again to the index. This is useful after changing core.autocrlf configuration or the text attribute in order to correct files added with wrong CRLF/LF line endings. This option implies -u.

The key part here is "all tracked files". Normally, filters are only run when a Git operation changes a file in the work tree. Changing the LFS whitelist in .gitattributes isn't a Git operation, and so the index ends up in an inconsistent state after you run git lfs untrack. Running git add --renormalize . tells Git to re-run filters on every file in the repository, which ensures that all files which should be in LFS are—and that all files which shouldn't be aren't.

The good thing about this method is that if you checkout from the past or another branch, the old lfs stuff is still there. But going forward it is not using lfs any more. The one thing I would add, is to remove the lfs stuff now from .gitattributes (or in my case deleting the whole file) and checking that in too. — David Casper, Jun 27 '19 at 01:25
It should be noted that this won't change the history, meaning that if you check out an older commit, you might get pointers to LFS instead of the actual files. And if you have since moved the repo or deleted the LFS, you won't get those old large files back this way. — Thomas Tempelmann, May 11 '20 at 14:03

VonC · Answer 3 · 2018-04-24T09:29:30.983

Issue 641 mentions the same issue.

I tried to stop using Git LFS, but found no way to revert my previous tracked pointer files using git lfs uninit, git lfs untrack, git rm... after I move those files back it still lists as tracked by Git LFS with git lfs ls-files, how can I opt out the whole Git LFS stuff from my repo?

The answer was:

Remove all filter.lfs.* git config entries with git lfs uninit.

Clear any any attributes that use the lfs filter in .gitattributes by running git lfs untrack for each file type, or deleting .gitattributes if LFS is all you ever used it for.

After this, any added files will go straight to git.

But this was not so simple:

I later end up LFS pointer files in my working directory and have to recover all my pictures from .git/lfs using the sha1 hash stored in those pointers manually.

Update March 2016, the issue 957 illustrates a possible solution by tstephens619:

I made the same mistake of including several small graphics formats into my git lfs tracking list.
I was able to move this files back into git by doing the following:
Create a list of all of the files currently being tracked by git-lfs, filter out *.gz and *.rpm (I want to still track those extensions with git-lfs)
git lfs ls-files | grep -vE "\.gz|\.rpm$" | cut -d ' ' -f 3 > ~/temp/lfs-files.txt
Stop tracking the small graphics files
git lfs untrack "*.tts"
git lfs untrack "*.bfx"
git lfs untrack "*.ttf"
git lfs untrack "*.xcf"
git lfs untrack "*.pkm"
git lfs untrack "*.png"
Temporarily uninit git-lfs
git lfs uninit
# Git LFS 2.x+
git lfs uninstall
Use the file list to touch each file:
cat ~/temp/lfs-files.txt | xargs touch
git status will now show each file as modified
Add the changes to git index (I did this via git gui)
commit the changes and then re-init git-lfs
git commit
git lfs init

The maintainer ttaylorr adds:

One way to do this would be:

for file in $FILES_TO_REVERT; do
  git lfs untrack "$file";
  git rm --cached "$file";
  git add --force "$file";
done

git commit -m "..."

My preference would be not to add a command to Git LFS to the above effect, since it is possible in a number of different way with the porcelain commands provided by Git and Git LFS

Thanks for finding the correct issue from the project. Still, even this solution requires moving the files manually. Seems sort of odd when the workflow from Git to Git LFS is basically `git rm --cached ` -> `git add ` -> `git commit` as long as you have the tracking set up correctly. — Olli Niskanen, Jan 28 '16 at 16:11
@Klipi I agree: it seems the untrack scenario is still perfectible. — VonC, Jan 28 '16 at 16:17
My version of LFS (2.0.2) doesn't have a `uninit` command. Instead of `git lfs uninit` I had to use `git lfs uninstall`. — endavid, Apr 29 '17 at 09:23

Joker · Answer 4 · 2019-01-28T12:43:43.887

3

I had problems doing steps in Windows. To remove all git lfs tracked files and restore original file I did the following in git bash:

Removed .gitattributes
git lfs ls-files | cut -d ' ' -f 3 > lfs-files.txt
Execute following snippet:

Snippet:

while read file; do
  git lfs untrack "$file";
  git rm --cached "$file";
  git add --force "$file";
done <lfs-files.txt

edited Jan 28 '19 at 12:43

answered May 28 '18 at 14:57

Joker

1,439
17
26

score -3 · Answer 5 · answered Jul 28 '17 at 15:03

-3

You can't really remove anything from GIT LFS, and although the solutions presented here may work (with modifications), they require a lot of effort and may have side effects on your repository.

If you arrived here, it is time to ask yourself whether you want to manage your large files with GIF LFS and whether GIT itself (which is inherently bad at managing large files, because it is a distributed version control system) was a good choice.

If you have many large files and you are a single organization working on your project, something like Subversion may work better for you.

answered Jul 28 '17 at 15:03

Florian Winter

3,381
1
33
52

4

@ericfrazer Every single visit to this question proves my point, as following the official documentation of GIT LFS, removing files from LFS should be a no-brainer. Except, it isn't, because it does not work. People are struggling, because of a combination of a) it is hard to fine-tune what files to add and b) adding many small files has performance issues on Windows because GIT LFS runs as a sub process, and starting sub processes is expensive without fork(). Bottom line: For your own mental sanity, don't use LFS before all of this is fixed. Which hasn't happened for years now... – Florian Winter Dec 18 '18 at 11:53

Move Git LFS tracked files under regular Git

5 Answers5

Linked