git encoding trouble: recursively removing a directory where git prints out a different name than it accepts
Posted by jpluimers on 2017/05/11
The story so far:
A few years back I put all my conferences material in a GitHub repository https://github.com/jpluimers/Conferences/. There were a lot directories and files so I didn’t pay much attention to the initial check-in list. The files had been part of copy.com syncing between Windows and Mac machines.
Often git on a Mac is a bit easier than on Windows (on a Mac you can install them with the
xcode-select --installtrick which installs only the Command Line Tools without having to install the full Xcode [WayBack]).
I choose a Mac because it is closer to a Linux machine than Widows so I expected no encoding trouble (as git has a Linux origin: it “was created by Linus Torvalds in 2005 for development of the Linux kernel“).
Boy I was wrong:
Recently I cloned the repository in a different place and found out a few strange things:
- Directories with accented characters had been duplicated, for instance in https://github.com/jpluimers/Conferences/tree/master/2011
- Beyond Compare would show the same content
- After a check-out git would not understand the %FC encoded directory name (%FC is IEC_8859-1 encoding for ü and \374 is the octal representation of 0xFC [WayBack]) and a
git statuswould show stuff like this:-
Untracked files: (use "git add ..." to include in what will be committed) EKON15-Renaissance-Hotel-D%FCsseldorf/or
deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Debugging/BO-EKON15-Delphi-XE2-Debugging.pdf"
-
- A
git rm -r --cachedcall [WayBack] would not work, as both these would fail:-
$ git rm -r --cached EKON15-Renaissance-Hotel-D%FCsseldorf fatal: pathspec 'EKON15-Renaissance-Hotel-D%FCsseldorf' did not match any files
and
$ git rm -r --cached "EKON15-Renaissance-Hotel-D\374sseldorf" fatal: pathspec 'EKON15-Renaissance-Hotel-D\374sseldorf' did not match any files
-
- a
So git could:
- detect the directories and files
- display the names of the detected directories and files
- not translate back the specified names into directories and files
All if this was with:
$ git --version
git version 1.9.5 (Apple Git-50.3)
This is how I fixed it
First I created an alias:
alias git-config="echo global: ; git config --list --global ; echo local: ; git config --lis --local ; echo system: ; git config --list --system"
That allowed me to view the git settings on various levels in my system.
It revealed I didn’t have the core.precomposeunicode setting at all (valid values are true or false). I also read various stories about one or both being the correct value: osx – Git and the Umlaut problem on Mac OS X – Stack Overflow [WayBack].
–jeroen
Result of git status:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ git status . | |
| On branch master | |
| Your branch is up-to-date with 'origin/master'. | |
| Changes not staged for commit: | |
| (use "git add/rm <file>…" to update what will be committed) | |
| (use "git checkout — <file>…" to discard changes in working directory) | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Debugging/BO-EKON15-Delphi-XE2-Debugging.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Unit-Testing/BO-EKON15-Delphi-XE2-Unit-Testing.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-0-sample-code.txt" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-1-Delphi-64bit.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-2-LiveBindings-DataBinding.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-3-Delphi-VCL Styles.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-4-Delphi-FireMonkey.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-Workshop/BO-EKON15-2011-XE2-Wokshop-5-Delphi-FireMonkey-xPlatform.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/Delphi-XE2-and-XML/BO-EKON15-2011-Delphi-XE2-and-XML.pdf" | |
| deleted: "EKON15-Renaissance-Hotel-D\374sseldorf/XSL-transforming-XML/BO-EKON15-2011-XSL-transforming-XML.pdf" |
The Wiert Corner - irregular stream of stuff 
Leave a Reply