Detecting binary files in the history of a git repository

Git, VCSes and binary files

Git is famous and has become popular even in the enterprise/commercial environments. But Git is also infamous regarding storage of large and/or binary files that change often, in spite of the fact they can be efficiently stored. For large files there have been several attempts to fix the issue, with varying degree of success, the most successful being git-lfs and git-annex.

My personal view is that, contrary to many practices, is a bad idea to store binaries in any VCS. Still, this practice has been and still is in use in may projects, especially in closed source projects. I won't go into the reasons, and how legitimate they are, let's say that we might finally convince people that binaries should be removed from the VCS, git, in particular.

Since the purpose of a VCS is to make sure all versions of the stored objects are never lost, Linus designed git in such a way that knowing the exact hash of the tip/head of your git branch, it is guaranteed the whole history of that branch hasn't changed even if the repository was stored in a non-trusted location (I will ignore hash collisions, for practical reasons).

The consequence of this is that if the history is changed one bit, all commit hashes and history after that change will change also. This is what people refer to when they say they rewrite the (git) history, most often, in the context of a rebase.

But did you know that you could use git rebase to traverse the history of a branch and do all sorts of operations such as detecting all binary files that were ever stored in the branch?

Detecting any binary files, only in the current commit

As with everything on *nix, we start with some building blocks, and construct our solution on top of them. Let's first find all files, except the ones in .git:

find . -type f -print | grep -v '^\.\/\.git\/'

Then we can use the 'file' utility to list for non-text files:

(find . -type f -print | grep -v '^\.\/\.git\/' | xargs file )| egrep -v '(ASCII|Unicode) text'

And if there are any such file, then it means the current git commit is one that needs our attention, otherwise, we're fine.

(find . -type f -print | grep -v '^\.\/\.git\/' | xargs file )| egrep -v '(ASCII|Unicode) text' && (echo 'ERROR:' && git show --oneline -s) || echo OK

 Of course, we assume here, the work tree is clean.

Checking all commits in a branch

Since we want to make this an efficient process and we only care if the history contains binaries, and branches are cheap in git, we can use a temporary branch that can be thrown away after our processing is finalized.
Making a new branch for some experiments is also a good idea to avoid losing the history, in case we do some stupid mistakes during our experiment.

Hence, we first create a new branch which points to the exact same tip the branch to be checked points to, and move to it:

git checkout -b test_bins

Git has many commands that facilitate automation, and my case I want to basically run the chain of commands on all commits. For this we can put our chain of commands in a script:

cat > ../check_file_text.sh

#!/bin/sh

(find . -type f -print | grep -v '^\.\/\.git\/' | xargs file )| egrep -v '(ASCII|Unicode) text' && (echo 'ERROR:' && git show --oneline -s) || echo OK

then (ab)use 'git rebase' to execute that for us for all commits:

git rebase --exec="sh ../check_file_text.sh" -i $startcommit

After we execute this, the editor window will pop up, just save and exit. Assuming $startcommit is the hash of the first commit we know to be clean or beyond which we don't care to search for binaries, this will look in all commits since then.

Here is an example output when checking the newest 5 commits:

$ git rebase --exec="sh ../check_file_text.sh" -i HEAD~5
Executing: sh ../check_file_text.sh
OK
Executing: sh ../check_file_text.sh
OK
Executing: sh ../check_file_text.sh
OK
Executing: sh ../check_file_text.sh
OK
Executing: sh ../check_file_text.sh
OK
Successfully rebased and updated refs/heads/test_bins.

Please note this process can change the history on the test_bins branch, but that is why we used a throw-away branch anyway, right? After we're done, we can go back to another branch and delete the test branch.

$ git co master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'

$ git branch -D test_bins
Deleted branch test_bins (was 6358b91).

Enjoy!

Suppressing color output of the Google Repo tool

On Windows, in the cmd shell, the color control caracters generated by the Google Repo tool (or its windows port made by ESRLabs) or git appear as garbage. Unfortunately, the Google Repo tool, besides the fact it has a non-google-able name, lacks documentation regarding its options, so sometimes the only way to find out what is the option I want is to look in the code.
To avoid repeatedly look over the code to dig up this, future self, here is how you disable color output in the repo tool with the info subcommand:

repo --color=never info

Other options are 'auto' and 'always', but for some reason, auto does not do the right thing (tm) in Windows and garbage is shown with auto.

HOWTO: git - change branch without touching working copy (at all)

Did you ever had the need in a git repository to change to another branch without altering AT ALL the working copy and ever wondered how that's done?

Usual use cases might be when you mde some changes to the working copy thinking you were on another branch, or you double-track in git a directory which is also tracked by another VCS (e.g. ClearCase).

What you need, in fact, is to update the index and not touch the working copy. The command that does that is

git read-tree otherbranch

If you also need to commit the state of your working tree to the otherbranch, you also need to tell git to actually associate the curent HEAD with the branch you just switched to:

git symbolic-ref HEAD refs/heads/otherbranch

I use this approach at my work place* to develop/experiment with possible code improvements on my machine before considering the merge into the official code.

* The preferred VCS is (Base) ClearCase, and I keep a git repository over the relevant part of the project in the ClerCase Dynamic View, so for synchronisations, the files in the working copy are updated by ClearCase and I have to resync my git branch (clearcaseint) following the latest official code from time to time, so I can pull in my local disk git repository the clearcaseint branch and merge it with my experimental changes in my git feature branches. 

If people are curious about how I work with ClearCase and git, I can expand on this in another post.

HOWTO: git - remove those pesky remotes/origin/

Often people create branches to fix various bugs or implement various features, and publish those to a public repository to be pulled in the official repository (e.g. pull requests in the github model).

The problem is that after the fix is merged the local repo still has the branch that tracks the origin (your public repository).

To remove the local branch it is simple:

git branch -d feature_x

To remove the actual branch in the remote repository:

git push origin --delete feature_x

But, still the local tracking branches remain, those pesky 'remotes/origin/feature_x'. Here is an example from one of my repos:

$ git branch -a
  activestate
  master
  next
* py.test
  sysplatformtest
  remotes/activestate/master
  remotes/origin/HEAD -> origin/master
  remotes/origin/appauthor-fallback
  remotes/origin/linux-fixes
  remotes/origin/master
  remotes/origin/next
  remotes/origin/py.test
  remotes/origin/root-directories
  remotes/origin/sysplatformtest
  remotes/origin/wrapper-defaults

All of those can be removed with this command:

git remote prune origin

 And the output for my appdirs repository is:

Pruning origin
URL: git@github.com:eddyp/appdirs.git
 * [pruned] origin/appauthor-fallback
 * [pruned] origin/linux-fixes
 * [pruned] origin/next
 * [pruned] origin/root-directories
 * [pruned] origin/sysplatformtest
 * [pruned] origin/wrapper-defaults

Which, finally removes those damned refs:

$ git branch -a
  activestate
* master
  py.test
  remotes/activestate/master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/py.test

Great!

Update 2013-07-24:  There have been also other solutions proposed in the comments section. I am quoting them here for better visibility.

Besides the solution I proposed, people commented about other alternatives for pruning remote branches, by using git fetch --prune or by using git-push's --prune option.

Brice provided a method to remove specific branches one by one with git branch -dr command:

git branch -dr origin/foo

What is ironic is that in the past I have used this format, and before writing the original article I was looking for this, but haven't found it that easily. I hope this article will help some other poor soul or myself in the future.

Interesting talk on the scalability of git

http://vimeo.com/m/64716825

HOWTO: sudo + cowbuilder (+git-buildpackage)

In case you tried to use git-buildpackage and wanted to use cowbuilder as a builder, you might have ran into the error

sudo: sorry, you are not allowed to preserve the environment

This is due to a change in sudo default configuration in version 1.7.4p4-2 (I know, is true since 2010) which doesn't allow the execution of commands via sudo using the parameter '-E' which means 'setenv'.

The news item even explicitly states pbuilder can be affected by this because it wants to port over to the pbuilder environment the HOME environment variable and suggests using

Defaults env_keep += HOME

But adding such a line to your /etc/sudoers.d/01_pbuilders file (/etc/sudoers is recommended to be touched only by the package) will do the  same for all commands and users ran via sudo, which is, according to my preferences, too permissive.

The irony is that running git-buildpackage --git-pbuilder  will invoke sudo -E cowbuilder so the env_keep suggested fix will not work because for -E to be allowed, setenv needs to be set in sudoers. This would defeat the purpose of env_reset, if done for all commands, but we can do a better job if we allow this kind of change only for the cowbuilder and pbuilder commands. You do have different commands that are allowed explicitly stated, don't you?

On my system I have made a group especially for people allowed to do packaging work, the group is called 'pack'. The only account in that group is my own.

Also, I have defined a command alias named PBUILDERS which looks something like this:

Cmnd_Alias PBUILDERS = /usr/sbin/pbuilder, /usr/sbin/cowbuilder

Running PBUILDERS is already restricted to the pack group. Here is an example that requires password on run:

%pack ALL=PBUILDERS

So all that needs to be done is to allow setenv for the PBUILDERS commands. Reading through the sudoers manpage and after some trial and error (use visudo for editting sudoers files - visudo -f /etc/sudoers.d/01_pbuilders) I found out that the symbols that distinguish between commands, users and groups in a 'Defaults' line needs to be right next to the 'Defaults' word. For command the sign is an exclamation sign '!' (for user lists it's ':') so since we want to link the exception to the command, not the user list, we'll use 'Defaults!':

Defaults! PBUILDERS setenv

Assuming you also have a PBUILDERS commands alias, this is what you need to be able to use git-buildpackage in conjunction with cowbuilder and sudo.

If there is a non-intrusive way to prevent sudo to use -E when invoking cowbuilder, please add your comment, I would be interested to know it.

HOWTO fix git-buildpackage signs with wrong signature

If you have two or more available gpg keys that can be used for the same identity (e.g.: 'John Doe <john@doe.com>') that you might use to add entries to changelogs of packages, you might end up in situations where git-buildpackage or other similar tool might want to sign packages with the wrong key.

It seems debsign (the tool that actually does the signing) just picks up the first key that is still valid (I also have a revoked key) and matches the used identity.

There are many ways to fix this, but the one that will work for most cases is to run this command:

echo 'DEBSIGN_KEYID=0x0123ABCD' >> ~/.devscripts

Of course, you should replace  0x0123ABCD with the keyid which you prefer.

This will create a ~/.devscripts file (if it didn't exist) which will contain the DEBSIGN_KEY variable with the desired value. This file will be sourced by debsign before doing its actual work, so it will do the right thing when ran.

There are also git-buildpackage specific fixes like defining builder in the [DEFAULT] section of ~/.gbp.conf and passing '-k0x0123ABCD' to debuild something like:

[DEFAULT]
builder = debuild -i -I -k1234ABCD
...

But that probably gets ugly if you use a custom builder.


P.S.: I have just deleted my old key 0xDD1F1F9F since I won't be using it anymore. I don't remember where I put the revocation certificate, but I'll revoke the key, once and if I find the revocation certificate. Otherwise, it will expire in July, next year.

From now on, I am going to use only the key 0xE3E083A1 to which I even added a photo and some newer identities and updated with a few signatures I got during DebConf 9.

HOWTO: Git - reauthor/fix author and committer email and author name after a git cvsimport

You might find yourself at some moment when your git repository imported from CVS does not contain all the correct names and email addresses of the commits which were once in CVS but are now part of your project history in your git repo. Or you might do a cvsimport which missed a few authors.

Let's suppose you first import the cvs repo into git, but then you realise you missed some authors.

Before being able to do a git cvsimport, you need a checkout of the module or cvs subdir that you want to turn into its own git repo.

For ease of use I defined CVSCMD as

cvs -z9 -d :pserver:my_cvs_id@cvs.server.com:/root_dir

You will need to replace the items written in italics according to you situation, more exactly, you need to define 'my_cvs_id', 'cvs.server.com' and 'root_dir'. If your acces method to the server is not pserver, you should change that accordingly. This information should be available from your project admin or pages.


Check out the desired module or even subdir of a module

$CVSCMD checkout -d localdirname MODULE/path/to/subdir

git cvsimport -A ../authors -m -z 600 -C ../new-git-repo -R

How to find out the commits which do need rewriting

The way to limit yourself only to the commits that had no cvs-git author and commit information on git-cvsimport time is to use a filter like this:

git log -E --author='^[^@]*$' --pretty=format:%h

This tells git log to print only the abbreviated hashes (%h) for the commits that have NO '@' sign in the 'Author:', which happens if no cvs user id to git author and email was provided in the authors file and git cvsimport time.

We will use this command's output to tell later git filter-branch which commits need rewriting. *

But before that...

How do we find if our authors file is complete?

For this task we'll use a slighly modified form of the previous command and some shell script magic.

git log -E --author='^[^@]*$' --pretty=format:%an | sort -u > all-leftout-cvs-authors

And now in all-leftout-cvs-authors we'll have a sorted list of all cvs id's which were not handled in the original git-cvsimport. In my case there are only 19 such ids:

$ wc -l all-leftout-cvs-authors
19 all-leftout-cvs-authors

Nice, that will be easy to fix. Now edit your all-leftout-cvs-authors file to add the relevant information in a format similar to this:

john = John van Code <john@code.temple.tld>
jimmy = Jimmy O'Document <jimmy@documenting.com>

In case you can't make a complete cvs-user-to-name-and-email map, you might want to use stubs of the following form in order to be able to easily identify later such commits, if you prefer (or you could let them unaltered at al ;-):

cvsid=cvsid <cvsid@cvs.server.com>

How to actually do the filtering to fix history (using git-filter-branch and a script)

After this is done, we'll need just one more piece, the command to do the altering itself which reads as follow (note that my final authors file is called new-authors and that I placed this in a script in order to be able to easily run it without trying to escape all spaces and such madness):

[ "$authors_file" ] || export authors_file=$HOME/new-authors

#git filter-branch -f --remap-cvs --env-filter '
git filter-branch -f --env-filter '

get_name () {
    grep "^$1=" "$authors_file" | sed "s/^.*=\(.*\)\ .*$/\1/"
}

get_email () {
    grep "^$1=" "$authors_file" | sed "s/^.*\ <\(.*\)>$/\1/"
}

if grep -q "^$GIT_COMMITTER_NAME" "$authors_file" ; then
    GIT_AUTHOR_NAME=$(get_name "$GIT_COMMITTER_NAME") &&
    GIT_AUTHOR_EMAIL=$(get_email "$GIT_COMMITTER_NAME") &&
    GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" &&
    GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL" &&
    export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL &&
    export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL
fi
' -- --all

You might wonder what's up with the commented git filter-branch line with the --remap-cvs option. This script will NOT work for you as long as you have the stock git-filter-branch script and keep the option --remap-cvs while not patching your git-filter-branch script (/usr/lib/git-core/git-filter-branch), but that option will provide a file with the mappings from the old to the new commit ids. If you want that function, too, you'll want to apply this patch to git-filter-branch:

diff --git a/git-filter-branch b/git-filter-branch
old mode 100644
new mode 100755
index ae602e3..d1f7ef6
--- a/git-filter-branch
+++ b/git-filter-branch
@@ -149,6 +149,11 @@ do
   prune_empty=t
   continue
   ;;
+ --remap-cvs)
+  shift
+  remap_cvs=t
+  continue
+  ;;
  -*)
   ;;
  *)
@@ -368,6 +373,33 @@ while read commit parents; do
    die "could not write rewritten commit"
 done <../revs
 
+# Rewrite the cvs-revisions file, if requested and the file exists
+
+ORIG_CVS_REVS_FILE="${GIT_DIR}/cvs-revisions"
+if [ -f "$ORIG_CVS_REVS_FILE" ]; then
+ if [ "$remap_cvs" ]; then
+  printf "CVS remapping requested\n"
+
+  CVS_REVS_FILE="$tempdir/cvs-revisions"
+  cp "$ORIG_CVS_REVS_FILE" "$CVS_REVS_FILE"
+  printf "\nFound $ORIG_CVS_REVS_FILE; will copy and alter it as $CVS_REVS_FILE\n"
+  cvs_remap__commit_count=0
+  newcommits="$(ls ../map/ | wc -l)"
+  for commit in ../map/* ; do
+   cvs_remap__commit_count=$(($cvs_remap__commit_count+1))
+   printf "\rRemap CVS commit $commit ($cvs_remap__commit_count/$newcommits)"
+
+   oldsha1="$(basename $commit)"
+   read newsha1 < $commit
+   sed -i "s@$oldsha1\$@$newsha1@" "$CVS_REVS_FILE"
+  done
+ else
+  warn "\nNo CVS remapping requested, but cvs-revisions file found. All CVS mappings will be lost.\n"
+ fi
+elif [ "$remap_cvs" ]; then
+ warn "\nWARNING: CVS remap was ignored, since no original cvs-revisions file was found\n"
+fi
+
 # If we are filtering for paths, as in the case of a subdirectory
 # filter, it is possible that a specified head is not in the set of
 # rewritten commits, because it was pruned by the revision walker.
@@ -491,6 +523,11 @@ if [ "$filter_tag_name" ]; then
  done
 fi
 
+if [ "$remap_cvs" -a -f "$CVS_REVS_FILE" ]; then
+ mv "$ORIG_CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE.original"
+ cp "$CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE"
+fi
+
 cd ../..
 rm -rf "$tempdir"
  

Then, after running this script, let's call it filter, you should have a brand new git repo with the appropriate authors and their emails set.


P.S.: I have started writing this post some time ago but stopped just before the last part, the one with the filter script. I realise I might be missing something in the explanation, but if you have problems, please comment so I can help you fixing them.

P.P.S.: * I realised in the filter script at some point I wanted to do something like:

for R in $(git log -E --author='^[^@]*$' --pretty=format:%H | head -n 2) ; do
[the same git filter branch command above but ending in ...]
' $R
done

But I think I remember that $R didn't work on the whole history, but only on that revision, or some other weird of that sort. I know I ended up not filtering explicitly those revisions, but the entire history. I hope this helps.

Picking up the pieces

Update: After trying KiBi's suggestion to take advantage of this information, I looked for more info on the issue and found this conversation. A git upgrade to the backported git 1:1.7.1-1.1~bpo50+1 version, and git svn rebase started pulling the right stuff in. Yes, I left the svn-remote.svn.rewriteRoot stuff set to the old value and the svn-remote.svn.url to the new value.

Keywords:

Unable to determine upstream SVN information from working tree history

when running git svn reabase

---

As I said yesterday, I am going to come back to being active in Debian.

I remember looking a little at my page on the Debian Wiki and it was clear that it was stale. (I sometimes find it amusing how I presented myself „my name is Eddy Petrisor”.) For some reason now the wiki seems to be down.

Two days ago I tried to get the Wormux/Warmux upstream source but git svn appears not to like the rename although I even modified the .git/config and .git/svn/.metadata, but it still wasn't satisfied and answered in a bad mood with this message (after a long waiting period):

$ git svn rebase
Unable to determine upstream SVN information from working tree history

And, yes, I am aware of the compromise and I changed my password for the project on gna.org.

I guess is better if I try to take a look at the RC bugs for the moment.

How to: git push via ssh; alternative users on the same host

Let's assume you're part of a project X that uses git for version control on a server host.it. Due to historical reasons your login on machine host.it is 'eddyp'. You're also part of another project Y which is hosted on server host.me. In this project you have always used 'eddy' as your ssh login for the git push.

Now, project Y decides to change hosting and moves the repo on server host.it. Since the logins were also migrated, now you have login 'eddyp' for project X and login 'eddy' for project B.

You changed the URI in remote.origin.url, but you are faced with this error when pushing from project Y to server host.it:

Counting objects: 7, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 476 bytes, done.
Total 4 (delta 2), reused 0 (delta 0)
error: unable to create temporary sha1 filename ./objects/56: Permission denied

fatal: failed to write object
error: unpack failed: unpacker exited with error code

The problem is that you are probably trying to login with 'eddyp' on server host.it from project Y since you ~/.ssh/config contains a 'user eddyp' clause for host host.it.


You can avoid this problem and identify to server host.it with different logins depending on the project via a small nice trick using fake hostnames in ~/.ssh/config:


Host eddyp.host.com
User eddyp
Hostname host.it

Host eddy.host.com
User eddy
Hostname host.it

And setting the remote.origin.url setting accordingly in the repo for project Y and project X:


eddy@projx $ git config remote.origin.url ssh://eddyp.host.it/srv/git/projx.git

eddy@projy $ git config remote.origin.url ssh://eddy.host.it/srv/git/projy.git

MSI PR200WX-058EU sleep - 91a6c462b02d8dc02dbe95e5a407d78078a38d01 is first bad commit

Nailed it!

After a rocky start, I managed to find the commit that broke sleep for my laptop.


91a6c462b02d8dc02dbe95e5a407d78078a38d01 is first bad commit
commit 91a6c462b02d8dc02dbe95e5a407d78078a38d01
Author: H. Peter Anvin
Date: Wed Jul 11 12:18:57 2007 -0700

Use the new x86 setup code for x86-64; unify with i386

This unifies arch/*/boot (except arch/*/boot/compressed) between
i386 and x86-64, and uses the new x86 setup code for x86-64 as well.

Signed-off-by: H. Peter Anvin
Signed-off-by: Linus Torvalds


Simply reverting this commit wouldn't fix the problem entirely since the screen was always blank after the successful resumes; OTOH, the script used for testing was supposed to do stuff after resume, stuff which would have effects visible on the hard-disk, so it was visible on the next reboot if the last sleep/resume cycle was successful or not.


Great. Oh, and git bisect rules!




Now, if you're interested in finding a regression in the kernel and you might be interested in how I automated the thing, here are some small scripts I used:

• linux-build - a wrapper script around make-kpkg to build .deb packages of the linux kernels I build; I used it way before this bisect, but now I modified it in such a way the kernels are clearly versioned and indicate the commit to which they correspond, too
• sleepit - a script that automated the actions needed for a linux kernel to be tested; is really trivial and highly specialized on sleep/resume debugging; it assumes to be ran in the directory where you'd later want to grab dmesg-s outputs from
• sleeptest - a wrapper script that is smart enough to detect if the current kernel is a kernel to be tested or a stable (regular kernel) one
  
• if the kernel is a stable one:
  • looks for the signs left by the last test kernel and depending on them, mark the kernel bad or good in the bisect; this would result in a new checkout which would be processed or, if the bad commit was identified, the script would stop
    
  • in the case of a new bisect, the new checkout is cleaned up, patched, built, then the script installs the new linux-image .deb[1] and update-grub[2], leaving the reboot command at my discretion for the eventual case something went awry; a failure to compile the kernel in an automated fashion would have dropped me in an interactive console which meant I had to manually do the steps necessary to be ready to boot into the next kernel
    
• if the kernel is a test kernel run the sleepit script

The main script was the sleeptest script which is ran as root to allow sleep commands, installation of the kernel and update-grub; when building, the build is done via su to my user.

As a supplemental speed up, I configured libpam-usb to authenticate root and myself through a USB storage device, which is quite cool. I am still pondering if I should keep this enabled or migrate to something like libpam-rsa[*].

Of course, the scripts contain stuff hard-coded into them (my user name for one), but they can easily be modified to remove those limitations (generally they use variables).


linux-build

#!/bin/sh
# License: GPLv2+/MIT
# Author: Eddy Petrișor
#
# Acest script trebuie rulat din directorul nucleului cu comanda:
#    linux-build [--no-headers] [--rebuild]
#
# This script must be ran from the kernel tree directory with
#    linux-build [--no-headers] [--rebuild]

FATTEMPT=../attempt

TARGETS="kernel-image kernel-headers modules_config modules"
[ "$1" = "--no-headers" ] && shift && TARGETS="$(echo $TARGETS | sed 's#kernel-headers ##')"

if [ -f $ATTEMPT ]
then
 ATT=`cat "${FATTEMPT}"`
 if [ $# -eq 0 ]
 then
     ATT=`expr $ATT + 1`
     make-kpkg clean
 else
     if [ $# -eq 1 ] && [ $1 = '--rebuild' ]
     then
         # nothing to do, we are  already set
         echo 'Preparing for rebuild'
     else
         echo 'Illegal parameters'
         exit 2
     fi
 fi
else
 ATT=1
fi

# no problem if is rewritten on rebuild
echo "$ATT" >$FATTEMPT

# must define MODULE_LOC for mol module compilation
DIR=`pwd`
cd ..
MODULE_LOC=$(pwd)/modules
# this didn't work
# export ALL_PATCH_DIR=$(pwd)/linux-patches
cd ${DIR}

echo "Modules should be here: ${MODULE_LOC}"
echo "Stop by ctrl+c, if the independent modules aren't there"

# press ctrl+c, if needed -- disabled for now
#read

export MODULE_LOC
export CONCURRENCY_LEVEL=$(grep -c 'processor' /proc/cpuinfo)

[ -d .git ] && PREFIX="g$(git log --pretty=oneline --max-count=1 | cut -c 1-8)-" || PREFIX=""
APPEND=$PREFIX$(hostname)

#time make-kpkg --rootcmd fakeroot --revision ${ATT} --stem linux --append-to-version -`hostname` --config menuconfig --initrd --uc --us kernel-image kernel-headers modules_config modules
#time make-kpkg --rootcmd fakeroot --revision ${ATT} --stem linux --append-to-version -`hostname` --added-patches 'ata_piix-ich8-fix-map-for-combined-mode.patch,ata_piix-ich8-fix-native-mode-slave-port.patch' --config silentoldconfig --initrd --uc --us kernel-image kernel-headers modules_config modules
time make-kpkg --rootcmd fakeroot --revision ${ATT} --stem linux --append-to-version -$APPEND --config silentoldconfig --initrd --uc --us $TARGETS

sleepit

#!/bin/sh

FAILEDRESUME=/failed-resume
RESUMED=/resumed

modprobe i915
invoke-rc.d acpid stop
echo "$(uname -r)" > $FAILEDRESUME
dmesg >dmesg_before_$(uname -r); echo mem > /sys/power/state; dmesg >dmesg_after_$(uname -r); sync
echo 'resumed, oh my god' > resumed
echo "$(uname -r)" >> $RESUMED
rm -f $FAILEDRESUME
sync
sleep 10
reboot

sleeptest

#!/bin/sh

RESULTSDIR=/root/var/debug/sleep/regression
UNAMER="$(uname -r)"
FAILEDSLEEPFILE=/failed-resume
RESUMED=/resumed
SOURCEDIR=/home/eddy/usr/src/linux/linux-2.6

check_same_commit ()
{
local COMMIT
COMMIT=$(git log --pretty=oneline --max-count=1 | cut -c 1-8)
[ "$COMMIT" = "$1" ] && return 0 || return 1
}

get_rev_from_unamer ()
{
echo "$1" | sed 's#.*-g\([0-9a-f]*\)-heidi#\1#'
}

mark_bad ()
{
cd $SOURCEDIR
su -c 'git reset --hard HEAD' eddy
su -c 'git bisect bad' eddy
cd -
}

mark_good ()
{
cd $SOURCEDIR
su -c 'git reset --hard HEAD' eddy
su -c 'git bisect good' eddy
cd -
}

compile_next ()
{
cd $SOURCEDIR
if [ -f $FAILEDSLEEPFILE ] ; then
 LKVER=$(cat $FAILEDSLEEPFILE)
else
 LKVER=$(tail -n 1 $RESUMED)
fi
PREVCOMMIT=$(get_rev_from_unamer "$LKVER")

if check_same_commit "$PREVCOMMIT" ; then
 echo "It looks like you got your result!"
 exit 1337  # of course $? isn't 1337, but anyways
fi

su -c 'make clean && rm -fr debian && git reset --hard HEAD && patch -p1 < lkver="$(cat">>> BAD <<< $LKVER ($(get_rev_from_unamer $LKVER))"   mark_bad  else   LKVER=$(tail -n 1 $RESUMED)   echo "Marking >>> good <<< $LKVER ($(get_rev_from_unamer $LKVER))"   mark_good  fi  compile_next && \  cd $SOURCEDIR/.. && \  echo 'Installing the linux-image and running update-grub && reboot' && \  dpkg -i $(ls linux-image-*_$(cat attempt)_*.deb) && \  update-grub  fi 

You have my permission to use, modify and redistribute these scripts or modified versions based on these under the terms of the MIT license.


[*] because the libpam-rsa package seems to be unmaintained (especially upstream), while libpam-usb seems to inactive (maybe is considered finished by upstream?)

[1] I didn't automate the removal of the previous test kernel, but that could have been done easily

[2] I haven't made a custom grub section for the test kernels in such a way they would boot by default at the next reboot since I considered that to be too cumbersome for the moment (although I had /vmlinuz symlinks) and it was simpler to select manually the kernel

Coordinating localization via git

Note: this is quite long and might not be interesting for people not involved with /in either of: debian i18n, debian l10n, git, shell scripting.


For some time now I have been the de-facto coordinator of the Romanian localization team. During this time I was faced multiple times with problems related to motivation of the team members, setting goals for a release, coordination of the changes inside the team, integration of new translators in the team, loosing team members, my lack of time on some occasions and other problems.

I thought a lot about how to improve on those points and I was never satisfied with the answers I got. What I think was the worst problem in the team was the inability to set clear goals, especially during lenny's development cycle.

During sarge's and etch's development cycles we were in an infancy state and setting as the goal to have a fully translated installation process was really enough to keep people motivated. For sarge we missed by a small margin, but the translations were of poor quality, while for etch we worked more on improving the translations.


But during lenny it was bad, really bad. My free time had been shrinking for a while, starting with etch's release, and we were unable to set clear goal, since for etch we managed to have the installation process fully translated and a few other translations. There was no way for us to reach 100% translations during lenny's development cycle, so setting that as a goal was really unrealistic. Percents by themselves don't mean anything for people, and as long as there's no substance to those numbers, there's no motivation to reach for one arbitrary percent.

I tried to set as a goal translating the packages installed by default in a new installation, but that hit the eternal question How do we know easily which are those packages?. This remained sometimes unanswered or got an unsatisfying answer. Also, there was a goal to have correct diacritics for Romanian in lenny, to have aspell-ro that uses the correct diacritics.

I even got to a point that I, myself, lost my motivation and set myself a personal goal of overrunning the level the translations that the language just above Romanian had in the po-debconf l10n statistics (ranking between languages). This was nice way for me to keep myself motivated, but I had my reserves in making this motivation public out of fear of being misinterpreted, because, by a strange coincidence, that next language was Hungarian, and in Romania's history there was some friction between Hungaria and Romania, while there are still some tensions with the Hungarian minority in Romania, in areas where they represent the local majority.

I managed to reach my personal goal, but this wasn't addressing the big picture.



So, sometime around the start of this year I started thinking about ways to coordinate the Romanian localization team in order to have:

• a clear goal at any given time
• a way to always be able to change that goal as we go
• a way to sync with eventual calls for translations, or the current sid translations
  
• stats immediately available
• automated checks for spelling, correct diacritics usage and other checks that might be useful (e.g. translation completeness)
• an easy way to assign somebody else as a language coordinator (I would appreciate some help or I might even consider stepping down)
• easy integration of new translators (by providing immediate answer to the question "What can I do to help with the translations?")
  

For short, a tool that would allow the team to work more efficiently while having the possibility to set clear goals in order to keep people motivated.



So after some pondering, I thought that creating a repository with the translations and the helper tools that would do the funky sync, checks, stats would be the best way to do that. So I started hacking on that somewhere around July-August and I published the result, but without much publicity, since it is still incomplete.

Some of the technical details are still in a haze, but I have a general idea and I got some basic functionality.


Today I decided that I should announce this semi-officially though my blog, maybe I get some input, ideas, or even contributions (I really should write a TODO).




I give you the Debian L10n Romanian coordination repository.

This is a git repository that has some tools to facilitate translation coordination and the translations that are current in the distro for the team.


Can be cloned with:

git clone git://git.debian.org/git/users/eddyp-guest/debian-ro-repo.git

or, if you're behind a restrictive firewall:

git clone http://git.debian.org/git/users/eddyp-guest/debian-ro-repo.git



Currently the work flow for updating a translation is as follows:

1. source _bin/polibs (. _bin/polibs)
2. cd foo
3. po_refresh
4. complete the translation
5. po_rearrange "ro.po"
6. git add "ro.po" && git commit -m "updated translation for foo"
7. send the translation ("git format-patch origin" and send the patches by mail, or, alternatively, just "git push")
   

Features:

• provides a po_refresh function that can import material directly from http://i18n.debian.net/material, but can also allow manual imports (template.pot from a call from translation)
• for a new translation: source _bin/polibs (. _bin/polibs), make a directory with the name of the source package, cd into it, and run po_refresh
  
• po_rearrange - beautify and unify the layout of PO files (facilitating compact and sane diff-ing for PO files)
• po_merge uses compendium, if present
  

Planned features:

• sync translations/templates from package VCS-es (Vcs-* headers and debcheckout should be the means to the end)
• po_rearrange should be called as a pre-commit hook; should either reject the commit if the po file was not rearranged, or automatically rearranged before the commit
• generate stats
• add commands for "what's outdated", "what needs review", "submit translation", and maybe "reserve translation for offline use"
• conflict merges should be done via po_merge (.gitattributes is key here)
• support other file types (?) - does this make any sense?
• periodic and automated sync with sid for all translations
  

Problems:

• security - running tests automatically from files within the repo doesn't seem too wise, but looks like the only way to get automated testing on any translator machine; maybe keeping the code in a submodule might address this issue?
• entry level translators still have a hard time - UI sucks now; there should be a wrapper command that should use the library functions and should provide a useful help
  
• is git too difficult ? - git backend usage maybe should be cloaked?
• still in development/alpha stage - I still haven't figured some of the issues
• central repo or really distributed - should there be a central git repo where the coordinator(s) do the pushes? it seems the central repo with a small pushers team for new translators (which can't commit directly) might actually facilitate interactions between experienced and new translators to instruct/bring up to speed the rookies

I was hoping that the release notes for lenny would facilitate from this infrastructure, but unfortunately I was lately in a really inactive period wrt Debian.



Questions, suggestions, ideas are welcome.

migrating from svn to git for wormux's maintainance

I have been thinking for quite some time about migrating the maintenance of the wormux package to git, but in a really smart way so that:

• we preserve history currently stored in svn
• a new integration branch should exist - we used the incomplete package model with svn-buildpackage, but now it makes more sense to have the complete history, including the upstream code at the time
  
• all previous orig tarballs should exist in the git repo in pristine-tar format
• upstream's svn repo should be included entirely so that we can cherry-pick any patches we might want or need

Since both the package repo and the upstream repo are in SVN, you can't simply use git-svn for both in a single repo, so I started thinking of some scheme to be able to join both repos and the (uncommitted) orig tarballs (which are backed up on DGT's webspace).


Tonight I have experimented a little with my git-svn clones which I made previously and eventually thought that:

• the main repo should track the upstream repo via git-svn
• I should properly tag the debian package versions in the svn.d.o repo, so svn tags no longer look like branches in git
  
• ideally all the branches in the package repo should be reproduced in some corresponding branch in the final repo
• the integration branch (that should contain the real debian complete packages) should be created with a non-existant merge strategy which I call overlay (basically, for two branches A and B, the strategy resumes to: all files as in A are present, any files unique to B are present)
• using pristine-tar forces me to keep the 'upstream' branch reserved so I need to avoid using that

It seems the last point could be obtained in some way via this small set of commands (I am sure it needs some modification to make the integration work in another branch C):

git merge --no-commit -s ours B
git checkout B -- .
git checkout A -- .
git commit


Note: Looking back, I think I can do a better job next time and I will se if I can use the resulting repo to create a new repo and drop the svn package maintainance for wormux. If this works, I might be able to create a script to allow migration of svn-bp maintained packages to git, at least for the Debian Games Team.


Since you can't fix a problem if you don't take things one at a time, I started by cloning the initial git-svn generated repos.

So after waiting some time (both git-svn fetch commands took long enough), I had to git-svn repos:

• repo A - the package repo
• repo B - the upstream svn clone repo

I tried to clone both A and B, to avoid distroying the original A and B clones and noticed that the svn branches were not visible from the cloned repo since the origins didn't had any other local branches than master - the svn trunk for each of them. So I added tracking branches in A for the experimental and sarge versions of the package. I also added some tags in A so that the svn tags don't get lost in between.

In the end I got my A' repo that had:

0 eddy@heidi ~/usr/src/wormux/tt/debian-bare $ git br -a
debian-experimental
debian-sarge
debian-sid
* master
origin/debian-experimental
origin/debian-sarge
origin/master

Yes, and having master in A' is not a good idea, but since I made A' a bare repo I had to suffer. Well, one more thing to better in iteration 2.


Excellent, now I can go in B and add the new remote repo which is A' which I should have called pkgsvn, but I ended up call it 'origin'. Since git-svn doesn't treat the svn repo as a true remote, this was possible.



Due to some confusion I also added 'ups-' as a prefix to the svn remote trunk and tags, but I am still unsure if this was necessary - I'll have to see more clearly into the pristine-tar restrictions area.

After all of this I got this branch configuration:
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch
* master
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch -r
0.8-final
0.8beta4
origin/debian-experimental
origin/debian-sarge
origin/debian-sid
origin/master
ups-tags/wormux-0.7
ups-tags/wormux-0.7.2
ups-tags/wormux-0.7.3
ups-tags/wormux-0.7.4
ups-tags/wormux-0.7.9
ups-tags/wormux-0.7.9rc1
ups-tags/wormux-0.7beta3
ups-tags/wormux-0.8
ups-tags/wormux-0.8alpha1
ups-tags/wormux-0.8beta1
ups-tags/wormux-0.8beta2
ups-tags/wormux-0.8beta3
ups-tags/wormux-0.8beta4
ups-trunk
wormux-0.7
wormux-0.7.9
wormux-0.8beta1
wormux-0.8beta2

So more tracking branches were necessary; I fired 'gitk --all' and got a nice picture, after adding tracking branches for branches originating from A':


This is not yet complete and now I am feeling sleepy, so I'll continue once I experiment more.

git-core backport still useless

The versioned depends on tk8.5 has been removed in 1:1.5.6.3-1~bpo40+4, but still, this isn't enough:

$ gitk
/usr/bin/gitk: line 3: /usr/bin/wish8.5: No such file or directory
/usr/bin/gitk: line 3: exec: /usr/bin/wish8.5: cannot execute: No such file or directory

The irony is that you can simply workaround with:

# ln -sf /usr/bin/wish8.4 /usr/bin/wish8.5


But the proper fix would be to apply this patch:


diff --git a/debian/rules b/debian/rules
index 7a1e771..7bd25cd 100755
--- a/debian/rules
+++ b/debian/rules
@@ -7,7 +7,7 @@ CFLAGS =-g -Wall
STRIP =strip
TEST =test
OPTS =NO_OPENSSL=1 prefix=/usr mandir=/usr/share/man INSTALLDIRS=vendor \
- WITH_P4IMPORT=1 PYTHON_PATH=/usr/bin/python TCLTK_PATH=/usr/bin/wish8.5 \
+ WITH_P4IMPORT=1 PYTHON_PATH=/usr/bin/python TCLTK_PATH=/usr/bin/wish \
THREADED_DELTA_SEARCH=1

ifneq (,$(findstring nostrip,$(DEB_BUILD_OPTIONS)))
--
1.5.6.3

git-core etch backport is useless

Update:

It seems that the lenny version (1:1.5.6.3-1) plus some small changes to allow building and running on etch does not have this problem.

The changes include:

* backported to etch
- compile against tcl8.4, instead of tcl8.5
- gtik and git-gui depend on tk8.4, instead of tk8.5
- drop the versioned dep on asciidoc and docbook-xsl

---

It s really nice that git-core is backported to etch, but is kind of hard to convince people to use it when gitk needs tk8.5, which is unavailable in etch.

This was the reason why previosely I made a local backport (1:1.5.6-1~bpo40+1~local) thinking the missing/extra[*] dep was a temporary glitch. According to bug 456423, this should be safe.


Does backports.org have a buildd network? IIRC, it uses the experimental buildd network, but that doesn't explain the desynchronisation of the packages in etch-backports.


Oddly enough, the build now fails on my etch machine during the tests in t9400-git-cvsserver-server.sh (looks like it is not related to the tk8.5 change):

* FAIL 23: cvs update (create new file)
echo testfile1 >testfile1 &&
git add testfile1 &&
git commit -q -m "Add testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep testfile1 CVS/Entries|cut -d/ -f2,3,5))" = "testfile1/1.1/" &&
diff -q testfile1 ../testfile1

* expecting success: echo line 2 >>testfile1 &&
git add testfile1 &&
git commit -q -m "Append to testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep testfile1 CVS/Entries|cut -d/ -f2,3,5))" = "testfile1/1.2/" &&
diff -q testfile1 ../testfile1
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 282 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
0879984..e6bd37b master -> master
cvs update: Updating .
cvs update: New directory `master'
* FAIL 24: cvs update (update existing file)
echo line 2 >>testfile1 &&
git add testfile1 &&
git commit -q -m "Append to testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep testfile1 CVS/Entries|cut -d/ -f2,3,5))" = "testfile1/1.2/" &&
diff -q testfile1 ../testfile1

* checking known breakage:
mkdir test &&
echo >test/empty &&
git add test &&
git commit -q -m "Single Subdirectory" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test ! -d test

Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 388 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
e6bd37b..4a13ee9 master -> master
cvs update: Updating .
cvs update: New directory `master'
* FIXED 25: cvs update w/o -d doesn't create subdir (TODO)

* expecting success: (for dir in A A/B A/B/C A/D E; do
mkdir $dir &&
echo "test file in $dir" >"$dir/file_in_$(echo $dir|sed -e "s#/# #g")" &&
git add $dir;
done) &&
git commit -q -m "deep sub directory structure" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update -d &&
(for dir in A A/B A/B/C A/D E; do
filename="file_in_$(echo $dir|sed -e "s#/# #g")" &&
if test "$(echo $(grep -v ^D $dir/CVS/Entries|cut -d/ -f2,3,5))" = "$filename/1.1/" &&
diff -q "$dir/$filename" "../$dir/$filename"; then
:
else
echo >failure
fi
done) &&
test ! -f failure
Counting objects: 13, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (12/12), 754 bytes, done.
Total 12 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (12/12), done.
To gitcvs.git
4a13ee9..f573218 master -> master
cvs update: Updating .
cvs update: New directory `master'
grep: A/CVS/Entries: No such file or directory
grep: A/B/CVS/Entries: No such file or directory
grep: A/B/C/CVS/Entries: No such file or directory
grep: A/D/CVS/Entries: No such file or directory
grep: E/CVS/Entries: No such file or directory
* FAIL 26: cvs update (subdirectories)
(for dir in A A/B A/B/C A/D E; do
mkdir $dir &&
echo "test file in $dir" >"$dir/file_in_$(echo $dir|sed -e "s#/# #g")" &&
git add $dir;
done) &&
git commit -q -m "deep sub directory structure" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update -d &&
(for dir in A A/B A/B/C A/D E; do
filename="file_in_$(echo $dir|sed -e "s#/# #g")" &&
if test "$(echo $(grep -v ^D $dir/CVS/Entries|cut -d/ -f2,3,5))" = "$filename/1.1/" &&
diff -q "$dir/$filename" "../$dir/$filename"; then
:
else
echo >failure
fi
done) &&
test ! -f failure

* expecting success: git rm testfile1 &&
git commit -q -m "Remove testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test -z "$(grep testfile1 CVS/Entries)" &&
test ! -f testfile1
rm 'testfile1'
Counting objects: 3, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 233 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (2/2), done.
To gitcvs.git
f573218..2ed3834 master -> master
cvs update: Updating .
cvs update: New directory `master'
* ok 27: cvs update (delete file)

* expecting success: echo readded testfile >testfile1 &&
git add testfile1 &&
git commit -q -m "Re-Add testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep testfile1 CVS/Entries|cut -d/ -f2,3,5))" = "testfile1/1.4/" &&
diff -q testfile1 ../testfile1
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 300 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
2ed3834..3f3e927 master -> master
cvs update: Updating .
cvs update: New directory `master'
* FAIL 28: cvs update (re-add deleted file)
echo readded testfile >testfile1 &&
git add testfile1 &&
git commit -q -m "Re-Add testfile1" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep testfile1 CVS/Entries|cut -d/ -f2,3,5))" = "testfile1/1.4/" &&
diff -q testfile1 ../testfile1

* expecting success: echo Line 0 >expected &&
for i in 1 2 3 4 5 6 7
do
echo Line $i >>merge
echo Line $i >>expected
done &&
echo Line 8 >>expected &&
git add merge &&
git commit -q -m "Merge test (pre-merge)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep merge CVS/Entries|cut -d/ -f2,3,5))" = "merge/1.1/" &&
diff -q merge ../merge &&
( echo Line 0; cat merge ) >merge.tmp &&
mv merge.tmp merge &&
cd "$WORKDIR" &&
echo Line 8 >>merge &&
git add merge &&
git commit -q -m "Merge test (merge)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
sleep 1 && touch merge &&
GIT_CONFIG="$git_config" cvs -Q update &&
diff -q merge ../expected
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 302 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
3f3e927..28c1b4a master -> master
cvs update: Updating .
cvs update: New directory `master'
* FAIL 29: cvs update (merge)
echo Line 0 >expected &&
for i in 1 2 3 4 5 6 7
do
echo Line $i >>merge
echo Line $i >>expected
done &&
echo Line 8 >>expected &&
git add merge &&
git commit -q -m "Merge test (pre-merge)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
test "$(echo $(grep merge CVS/Entries|cut -d/ -f2,3,5))" = "merge/1.1/" &&
diff -q merge ../merge &&
( echo Line 0; cat merge ) >merge.tmp &&
mv merge.tmp merge &&
cd "$WORKDIR" &&
echo Line 8 >>merge &&
git add merge &&
git commit -q -m "Merge test (merge)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
sleep 1 && touch merge &&
GIT_CONFIG="$git_config" cvs -Q update &&
diff -q merge ../expected

* expecting success: ( echo LINE 0; cat merge ) >merge.tmp &&
mv merge.tmp merge &&
git add merge &&
git commit -q -m "Merge test (conflict)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
diff -q merge ../expected.C
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 303 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
28c1b4a..1f0f9c8 master -> master
cvs update: Updating .
cvs update: New directory `master'
diff: merge: No such file or directory
* FAIL 30: cvs update (conflict merge)
( echo LINE 0; cat merge ) >merge.tmp &&
mv merge.tmp merge &&
git add merge &&
git commit -q -m "Merge test (conflict)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update &&
diff -q merge ../expected.C

* expecting success: cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update -C &&
diff -q merge ../merge
cvs update: Updating .
cvs update: New directory `master'
diff: merge: No such file or directory
* FAIL 31: cvs update (-C)
cd cvswork &&
GIT_CONFIG="$git_config" cvs -Q update -C &&
diff -q merge ../merge

* expecting success: echo Line 9 >>merge &&
cp merge cvswork/merge &&
git add merge &&
git commit -q -m "Merge test (no-op)" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
sleep 1 && touch merge &&
GIT_CONFIG="$git_config" cvs -Q update &&
diff -q merge ../merge
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 303 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
1f0f9c8..e6c5fc1 master -> master
cvs update: Updating .
cvs update: New directory `master'
* ok 32: cvs update (merge no-op)

* expecting success:
touch really-empty &&
echo Line 1 > no-lf &&
echo -n Line 2 >> no-lf &&
git add really-empty no-lf &&
git commit -q -m "Update -p test" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs update &&
rm -f failures &&
for i in merge no-lf empty really-empty; do
GIT_CONFIG="$git_config" cvs update -p "$i" >$i.out
diff $i.out ../$i >>failures 2>&1
done &&
test -z "$(cat failures)"

Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 305 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To gitcvs.git
e6c5fc1..4d0f85b master -> master
cvs update: Updating .
cvs update: New directory `master'
cvs update: Updating .
cvs update: New directory `master'
cvs update: Updating .
cvs update: New directory `master'
cvs update: Updating .
cvs update: New directory `master'
cvs update: Updating .
cvs update: New directory `master'
* FAIL 33: cvs update (-p)

touch really-empty &&
echo Line 1 > no-lf &&
echo -n Line 2 >> no-lf &&
git add really-empty no-lf &&
git commit -q -m "Update -p test" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs update &&
rm -f failures &&
for i in merge no-lf empty really-empty; do
GIT_CONFIG="$git_config" cvs update -p "$i" >$i.out
diff $i.out ../$i >>failures 2>&1
done &&
test -z "$(cat failures)"


* expecting success:
mkdir status.dir &&
echo Line > status.dir/status.file &&
echo Line > status.file &&
git add status.dir status.file &&
git commit -q -m "Status test" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs update &&
GIT_CONFIG="$git_config" cvs status | grep "^File: status.file" >../out &&
test $(wc -l <../out) = 2 Counting objects: 5, done. Compressing objects: 100% (2/2), done. Writing objects: 100% (4/4), 366 bytes, done. Total 4 (delta 1), reused 0 (delta 0) Unpacking objects: 100% (4/4), done. To gitcvs.git 4d0f85b..a146cf3 master -> master
cvs update: Updating .
cvs update: New directory `master'
Invalid module '' at /home/eddy/usr/src/tools/git-backports/1.5.6.3-1~bpo40+3~local/git-core-1.5.6.3/t/../git-cvsserver line 2895, line 18.
cvs [status aborted]: end of file from server (consult above messages if any)
* FAIL 34: cvs status

mkdir status.dir &&
echo Line > status.dir/status.file &&
echo Line > status.file &&
git add status.dir status.file &&
git commit -q -m "Status test" &&
git push gitcvs.git >/dev/null &&
cd cvswork &&
GIT_CONFIG="$git_config" cvs update &&
GIT_CONFIG="$git_config" cvs status | grep "^File: status.file" >../out &&
test $(wc -l <../out) = 2 * expecting success: cd cvswork && GIT_CONFIG="$git_config" cvs status -l | grep "^File: status.file" >../out &&
test $(wc -l <../out) = 1 Invalid module '' at /home/eddy/usr/src/tools/git-backports/1.5.6.3-1~bpo40+3~local/git-core-1.5.6.3/t/../git-cvsserver line 2895, line 19.
cvs [status aborted]: end of file from server (consult above messages if any)
* FAIL 35: cvs status (nonrecursive)

cd cvswork &&
GIT_CONFIG="$git_config" cvs status -l | grep "^File: status.file" >../out &&
test $(wc -l <../out) = 1 * expecting success: cd cvswork && GIT_CONFIG="$git_config" cvs status | grep ^File: >../out &&
! grep / <../out Invalid module '' at /home/eddy/usr/src/tools/git-backports/1.5.6.3-1~bpo40+3~local/git-core-1.5.6.3/t/../git-cvsserver line 2895, line 18.
cvs [status aborted]: end of file from server (consult above messages if any)
* FAIL 36: cvs status (no subdirs in header)

cd cvswork &&
GIT_CONFIG="$git_config" cvs status | grep ^File: >../out &&
! grep / <../out * fixed 1 known breakage(s) * failed 11 among 36 test(s) make[2]: *** [t9400-git-cvsserver-server.sh] Error 1 make[2]: Leaving directory `/home/eddy/usr/src/tools/git-backports/1.5.6.3-1~bpo40+3~local/git-core-1.5.6.3/t' make[1]: *** [test] Error 2 make[1]: Leaving directory `/home/eddy/usr/src/tools/git-backports/1.5.6.3-1~bpo40+3~local/git-core-1.5.6.3' make: *** [build-arch-stamp] Error 2


[*] depends on how you look at it

Debian Games Team - have source, but do not import :-/

JoeyH wrote a while back some thoughts on why he hates the DGT SVN repo (mainly our policy to import just the incomplete source of packages - usually just the debian/ directory).

He updated that blog entry and latter said:

Update: I realized after posting this that while the space gains on alioth might be illusory, there is certianly a space gain for people checking the whole repo out. What's really needed is a way to keep all the upstream sources in svn, but check them out only when wanted, and check out all the debian directories for cross-game work.

I was thinking, how about having a different area of the repo where the full source is present and which pulls the debian directory as an external from the current location?

I know it has the huge disadvantage that anything outside of the debian/ directory, but part of the diff, is, for sure, lost. (Still, there is also another team policy to not track anything outside debian directly, except through patches[1], except for really exceptional cases, but even then, there is a need for a strong justification).


Would that work? (I know git-svn doesn't support externals properly, yet, so there's not much gain).


[1] I know Joey, you hate those, too