Here's a way to get from a starting point of two or more peer directory trees not tracked by git or git-annex, to the point where they can be synced in the manner described above: syncing non-git trees with git-annex

Yes, it is really a minor point. And indeed, "git log --summary" is pretty good already. But I’d still think that at least the title could deserves some love. Including the Hostname or the name of the repository there is a good idea as well.

Took me a minute to see this is not about wishlist: more descriptive commit messages in git-annex branch, but about the "git-annex automatic sync" message that is used when committing any changes currently on the master branch before doing the rest of the sync.

So.. It would be pretty easy to ls-files the relevant files before the commit and make a message. Although this would roughly double the commit time in a large tree, since that would walk the whole tree again (git commit -a already does it once). Smarter approaches could be faster.. perhaps it could find unstaged files, stage them, generate the message, and then git commit the staged changes.

But, would this really be useful? It's already easy to get git log to show a summary of the changes made in such a commit. So it's often seen as bad form to unnecessarily mention which files a commit changes in the commit message.

Perhaps more useful would be to expand the current message with details like where the sync is being committed, or what remotes it's going to sync from, or something like that.

The symlinks are in the git repository. So if the rsync damanged one, git would see the change. And nothing that happens to the symlinks can affect fsck.

git-annex does not use hard links at all.

fsck corrects mangled file permissions. It is possible to screw up the permissions so badly that it cannot see the files at all (ie, chmod 000 on a file under .git/annex/objects), but then fsck will complain and give up, not move the files to bad. So I don't see how a botched rsync could result in fsck moving a file with correct content to bad.

OK, thanks. I was just wondering - since there are links in git(-annex), and a hard links too, that maybe the issue has been caused by rsync.

I will keep my eye on that and run checks with my own checksum and fsck from time to time, and see what happens. I will post my results here, but the whole run (fsck or checksum) takes almost 2 days, so I will not do it too often... ;)

All that git annex fsck does is checksum the file and move it away if the checksum fails.

If bad data was somehow read from the disk that one time, what you describe could occur. I cannot think of any other way it could happen.

Thanks, joey, but I still do not know, why the file that has been (and is) OK according to separate sha1 and sha256 checks, has been marked 'bad' by fsck and moved to .git/annex/bad. What could be a reason for that? Could have rsync caused it? I know too little about internal workings of git-annex to answer this question.

But one thing I know for certain - the false positives should not happen, unless something is wrong with the file. Otherwise, if it is unreliable, if I have to check twice, it is useless. I might as well just keep checksums of all the files and do all checks by hand...

Well, it should only move files to .git/annex/bad/ if their filesize is wrong, or their checksum is wrong.

You can try moving a file out of .git/annex/bad/ and re-run fsck and see if it fails it again. (And if it does, paste in a log!)

To do that -- Suppose you have a file .git/annex/bad/SHA256-s33--5dc45521382f1c7974d9dbfcff1246370404b952 and you know that file foobar was supposed to have that content (you can check that foobar is a symlink to that SHA value). Then reinject it:

git annex reinject .git/annex/bad/SHA256-s33--5dc45521382f1c7974d9dbfcff1246370404b952 foobar

Yes, contents are still considered used while tags or refs refer to them. Including remote tracking branches like remotes/origin/master

Sorry for commmenting on my own question ... I think I just figured out that git annex unused does in fact do what I want. When I tried it, it just didn't show the obsolete versions of the files I edited because I hadn't yet synchronized all repositories, so that was why the obsolete versions were still considered used.