Mailing List Archive: Gentoo: Dev

Portage Git migration - clean cut or git-cvsserver

1 2 3 4 5 6 7

View All

Index | Next | Previous | View Threaded

robbat2 at gentoo

May 31, 2012, 9:49 AM

Post #101 of 163 (190 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Wed, May 30, 2012 at 10:31:06PM +0200, Dirkjan Ochtman wrote:
> On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson <robbat2 [at] gentoo> wrote:
> > No, the last mock conversion is still live and updating fairly often:
> > http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary
> Since you seem to know most about this project, can you give a short
> summary on what *you* think are hard blockers for the migration?
There are only two items from my perspective.
1.
Discussion on merge policy. Originally I thought we would disallow merge
commits, so that we would get a cleaner history. However, it turns out that if
the repo ends up being pushed to different places with slightly different
histories, merges are absolutely going to be required to prevent somebody from
having to rebase at least one of their sets of commits that are already pushed.

2.
Git-SVN breakage. Why does this matter you're wondering?
We need the newer Git for the commit signing, but it comes with a
price, the git-svn binary has some major failures with SVN 1.7.
Git since 1.7.8 has been broken this way.

You can see them best with the testsuite. I haven't keyworded those ebuilds for
git at all, because git-svn in some of my tests ended up being destructive of
the repository - it actually lost data.

The error sometimes turns up like this.

Initialized empty Git repository in /dev/shm/portage/dev-vcs/git-9999/work/git-9999/t/t d.t9155/git_project/.git/
svn: E235000: In file 'subversion/libsvn_subr/dirent_uri.c' line 2291: assertion failed (svn_uri_is_canonical(url, pool))

This is seemingly due to a change in SVN 1.7 that is stricter about all paths
(both URIs and to files/directories) being properly escaped.

Upstream Git has made no progress on it in more than 6 months.
Both ferringb and I put several weeks of work into it without succeeding.

If you want to reproduce it:
- Upgrade your SVN to 1.7
- Build & run the git testsuite
- Most of the t91xx tests will fail, because the working dir is 'trash directory'.
- If you patch the working dir to be 'trash_directory', you'll be left with
failures in at least: t9100 t9118 t9120
Because those are the tests that have whitespace or other characters that
need escaping.

The last version of my patch is files/git-1.7.8-git-svn-1.7-canonical-path.patch
but it caused more problems than it fixed.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2 [at] gentoo
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

robbat2 at gentoo

May 31, 2012, 9:50 AM

Post #102 of 163 (189 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 08:04:10AM -0400, Aaron W. Swenson wrote:
> On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote:
> > On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson
> > <robbat2 [at] gentoo> wrote:
> >> No, the last mock conversion is still live and updating fairly
> >> often:
> >> http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary
> The 6 hours it takes to clone the repo.
To directly clone that repo yes.

Which you should NEVER be doing in reality.

The final conversion repo we have already stated will only be 40MB, with
no history. History will be available separately to graft on if you need
it, just like the Linux kernel did with historical data.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2 [at] gentoo
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

rich0 at gentoo

May 31, 2012, 10:48 AM

Post #103 of 163 (188 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson <robbat2 [at] gentoo> wrote:
> 1.
> Discussion on merge policy. Originally I thought we would disallow merge
> commits, so that we would get a cleaner history. However, it turns out that if
> the repo ends up being pushed to different places with slightly different
> histories, merges are absolutely going to be required to prevent somebody from
> having to rebase at least one of their sets of commits that are already pushed.

Not sure I'm following, but I will be the first to admit that I'm a
git novice. Would this be aided by a convention, like only committing
to master on the gentoo official repository, and any on-the-side work
on places like github/etc stays in branches? Those repositories would
just keep getting fed commits on master from the official repository.

>
> 2.
> Git-SVN breakage. Why does this matter you're wondering?
> We need the newer Git for the commit signing, but it comes with a
> price, the git-svn binary has some major failures with SVN 1.7.
> Git since 1.7.8 has been broken this way.

To clarify - these won't be issues for gentoo per se, but there is a
sense that we can't stabilize the latest git because it will break it
for people using git-svn on non-gentoo work?

I think the general conclusion was that we would not be supporting any
mixed git+cvs/svn/etc workflows for gentoo itself.

If that is the case, what is our sense of how important this feature
even is to gentoo developers? They're the only ones who really have
to have the latest git to commit to the official tree.

Rich

robbat2 at gentoo

May 31, 2012, 12:13 PM

Post #104 of 163 (189 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
> On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson <robbat2 [at] gentoo> wrote:
> > 1.
> > Discussion on merge policy. Originally I thought we would disallow merge
> > commits, so that we would get a cleaner history. However, it turns out that if
> > the repo ends up being pushed to different places with slightly different
> > histories, merges are absolutely going to be required to prevent somebody from
> > having to rebase at least one of their sets of commits that are already pushed.
> Not sure I'm following, but I will be the first to admit that I'm a
> git novice. Would this be aided by a convention, like only committing
> to master on the gentoo official repository, and any on-the-side work
> on places like github/etc stays in branches? Those repositories would
> just keep getting fed commits on master from the official repository.
Ok, let me try and reword my statement.

- You have a commit, that you want to put into the Gentoo tree.
- You have already pushed it to your github, signed
- It needs to be merged/rebased so that it applies on the Gentoo tree.
- If you force it to be a rebase so it applies on the tip, then you may
have changed the history of your github tree, and broken any further
forks.
- If you permit a merge instead, nobody gets broken.

> > 2.
> > Git-SVN breakage. Why does this matter you're wondering?
> > We need the newer Git for the commit signing, but it comes with a
> > price, the git-svn binary has some major failures with SVN 1.7.
> > Git since 1.7.8 has been broken this way.
> To clarify - these won't be issues for gentoo per se, but there is a
> sense that we can't stabilize the latest git because it will break it
> for people using git-svn on non-gentoo work?
As the Git maintainer, I will not keyword it for anybody until I know
it's not going to lose/corrupt data, regardless of what they are using
it for.

I don't think there are many SVN repos left in Gentoo that haven't
converted to using Git directly, so it's probably not a problem from
that side.

> If that is the case, what is our sense of how important this feature
> even is to gentoo developers? They're the only ones who really have
> to have the latest git to commit to the official tree.
You'd be excluding me entirely, I need to use git-svn for other work
projects, and emerging between two different versions of git would be
very annoying (I switch constantly between the sides of work as they
overlap).

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2 [at] gentoo
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

williamh at gentoo

May 31, 2012, 12:18 PM

Post #105 of 163 (189 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

ciaran.mccreesh at googlemail

May 31, 2012, 12:23 PM

Post #106 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, 31 May 2012 14:18:04 -0500
William Hubbs <williamh [at] gentoo> wrote:
> > Not sure I'm following, but I will be the first to admit that I'm a
> > git novice. Would this be aided by a convention, like only
> > committing to master on the gentoo official repository, and any
> > on-the-side work on places like github/etc stays in branches?
> > Those repositories would just keep getting fed commits on master
> > from the official repository.
>
> Iagree with this; I think we should ban merge commits on master. That
> would force everyone to rebase their work on current master before
> they commit to master which would make the history clean.

So what's the point of switching to git if you want to ban the main
reason git exists?

--
Ciaran McCreesh

Attachments:

signature.asc (0.19 KB)

rich0 at gentoo

May 31, 2012, 12:29 PM

Post #107 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 3:13 PM, Robin H. Johnson <robbat2 [at] gentoo> wrote:
> - You have a commit, that you want to put into the Gentoo tree.
> - You have already pushed it to your github, signed
> - It needs to be merged/rebased so that it applies on the Gentoo tree.
> - If you force it to be a rebase so it applies on the tip, then you may
> �have changed the history of your github tree, and broken any further
> �forks.
> - If you permit a merge instead, nobody gets broken.

Maybe the best compromise is to tell people that if you push to
"master" on other repositories, you get to deal with the mess. If we
try to keep side overlays/etc working on branches and not on master
then there will be no history to rewrite, as the merge will be rebased
when it hits the official master, and from there it will get pulled by
other repositories.

We can perhaps allow merge commits on other branches, where the
continuity of history is less important.

Does that make sense?

> You'd be excluding me entirely, I need to use git-svn for other work
> projects, and emerging between two different versions of git would be
> very annoying (I switch constantly between the sides of work as they
> overlap).

I'm a big proponent of letting the people doing the work scratch their
own itches first! However, this does make us dependent on upstream -
is there any sense of when they'll be ready, or what their own
priority is for this issue. If this is becoming a deprecated feature
then I'm not sure we can tie our future to it.

I wasn't sure if any of the existing git-svn bugs pertained to this
issue. Either way we should add this as a blocker to the git
migration tracker.

I think that even if we made a big push it would still take us a month
or two to be ready with docs/infra/etc, and that might be optimistic.
So, this might not be rate-limiting if upstream is actively working on
it.

Rich

mgorny at gentoo

May 31, 2012, 12:33 PM

Post #108 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, 31 May 2012 14:18:04 -0500
William Hubbs <williamh [at] gentoo> wrote:

> On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
> > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson
> > <robbat2 [at] gentoo> wrote:
> > > 1.
> > > Discussion on merge policy. Originally I thought we would
> > > disallow merge commits, so that we would get a cleaner history.
> > > However, it turns out that if the repo ends up being pushed to
> > > different places with slightly different histories, merges are
> > > absolutely going to be required to prevent somebody from having
> > > to rebase at least one of their sets of commits that are already
> > > pushed.
> >
> > Not sure I'm following, but I will be the first to admit that I'm a
> > git novice. Would this be aided by a convention, like only
> > committing to master on the gentoo official repository, and any
> > on-the-side work on places like github/etc stays in branches?
> > Those repositories would just keep getting fed commits on master
> > from the official repository.
>
> Iagree with this; I think we should ban merge commits on master. That
> would force everyone to rebase their work on current master before
> they commit to master which would make the history clean.

What would git signing work with rebased commits? Would all of them
have to be signed once again?

--
Best regards,
Michał Górny

Attachments:

signature.asc (0.31 KB)

alexxy at gentoo

May 31, 2012, 12:52 PM

Post #109 of 163 (181 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

Michał Górny писал 2012-05-31 23:33:
> On Thu, 31 May 2012 14:18:04 -0500
> William Hubbs <williamh [at] gentoo> wrote:
>
>> On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
>> > On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson
>> > <robbat2 [at] gentoo> wrote:
>> > > 1.
>> > > Discussion on merge policy. Originally I thought we would
>> > > disallow merge commits, so that we would get a cleaner history.
>> > > However, it turns out that if the repo ends up being pushed to
>> > > different places with slightly different histories, merges are
>> > > absolutely going to be required to prevent somebody from having
>> > > to rebase at least one of their sets of commits that are already
>> > > pushed.
>> >
>> > Not sure I'm following, but I will be the first to admit that I'm
>> a
>> > git novice. Would this be aided by a convention, like only
>> > committing to master on the gentoo official repository, and any
>> > on-the-side work on places like github/etc stays in branches?
>> > Those repositories would just keep getting fed commits on master
>> > from the official repository.
>>
>> Iagree with this; I think we should ban merge commits on master.
>> That
>> would force everyone to rebase their work on current master before
>> they commit to master which would make the history clean.
>
> What would git signing work with rebased commits? Would all of them
> have to be signed once again?

Commits itsels still will be signed
--
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute,
Gatchina, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum [at] gmail
mailto:alexxy [at] gentoo
mailto:alexxy [at] omrb

williamh at gentoo

May 31, 2012, 12:54 PM

Post #110 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 07:13:42PM +0000, Robin H. Johnson wrote:
> - You have a commit, that you want to put into the Gentoo tree.
> - You have already pushed it to your github, signed

If I have a github tree, that would probably be because I didn't have
push access to the official tree, so signing the commit probably
isn'tgoing to matter; I would expect that a gentoo dev who has push
access to the tree would sign the commit when it is put into the gentoo
tree.

> - It needs to be merged/rebased so that it applies on the Gentoo tree.
> - If you force it to be a rebase so it applies on the tip, then you may
> have changed the history of your github tree, and broken any further
> forks.
> - If you permit a merge instead, nobody gets broken.

If you do this:

git rebase master mybranch
git checkout master
git merge mybranch <-- this is a fast-forward merge
git pull --rebase
git push

I think that covers this concern doesn't it?

> > > 2.
> > > Git-SVN breakage. Why does this matter you're wondering?
> > > We need the newer Git for the commit signing, but it comes with a
> > > price, the git-svn binary has some major failures with SVN 1.7.
> > > Git since 1.7.8 has been broken this way.
> > To clarify - these won't be issues for gentoo per se, but there is a
> > sense that we can't stabilize the latest git because it will break it
> > for people using git-svn on non-gentoo work?
> As the Git maintainer, I will not keyword it for anybody until I know
> it's not going to lose/corrupt data, regardless of what they are using
> it for.

Why not keyword it and use package.use.mask for the git-svn flag?

William

rich0 at gentoo

May 31, 2012, 12:58 PM

Post #111 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo> wrote:
> What would git signing work with rebased commits? Would all of them
> have to be signed once again?
>

The whole point of rebasing is to throw away history (which is either
good or bad based on your perspective).

So, if 14 devs spend 3 years and 2000 commits working on something in
a branch, and I commit it to master using a rebase, then all you'll
see in the master history is that rich0 committed 20k lines of code to
master on May 31st, and that would be signed by me.

I think that rebasing before merging is a pretty typical workflow
anyway - when you submit a patch to Linus, he doesn't really care that
you spent six months tweaking it - he just is getting a big blob of
code that either works or doesn't. Does all that sub-history really
matter? You could always push the branch to the repository if you
wanted to keep it on the side.

Rich

mgorny at gentoo

May 31, 2012, 1:09 PM

Post #112 of 163 (184 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, 31 May 2012 15:58:43 -0400
Rich Freeman <rich0 [at] gentoo> wrote:

> On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo>
> wrote:
> > What would git signing work with rebased commits? Would all of them
> > have to be signed once again?
> >
>
> The whole point of rebasing is to throw away history (which is either
> good or bad based on your perspective).
>
> So, if 14 devs spend 3 years and 2000 commits working on something in
> a branch, and I commit it to master using a rebase, then all you'll
> see in the master history is that rich0 committed 20k lines of code to
> master on May 31st, and that would be signed by me.
>
> I think that rebasing before merging is a pretty typical workflow
> anyway - when you submit a patch to Linus, he doesn't really care that
> you spent six months tweaking it - he just is getting a big blob of
> code that either works or doesn't. Does all that sub-history really
> matter? You could always push the branch to the repository if you
> wanted to keep it on the side.

That sounds like a pretty poor workflow to me. If I tweak something for
3 years, I'm likely to have a larger set of logically organized commits.
I'm not saying it's unlikely I'm going to rebase fixes for obvious
mistakes but putting everything onto one blob just makes the changes
harder to read and less maintainable.

For example, if I first create a basic function and then add additional
options to it, I'm likely to keep those as separate commits. If one of
the changes was a really bad idea, I'd like to revert the appropriate
commit rather than removing all traces of it by hand and mistakenly
introducing even worse breakage.

--
Best regards,
Michał Górny

Attachments:

signature.asc (0.31 KB)

1i5t5.duncan at cox

May 31, 2012, 1:26 PM

Post #113 of 163 (183 views)
Permalink

Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:

> On Thu, May 31, 2012 at 07:13:42PM +0000, Robin H. Johnson wrote:
>> - You have a commit, that you want to put into the Gentoo tree.
>> - You have already pushed it to your github, signed
>
> If I have a github tree, that would probably be because I didn't have
> push access to the official tree, so signing the commit probably
> isn'tgoing to matter; I would expect that a gentoo dev who has push
> access to the tree would sign the commit when it is put into the gentoo
> tree.

I don't know what's going to happen to all the overlays with the main
tree switch to git, but won't that break various "overlay first"
policies, say for the kde overlay?

Of course, if all the official overlays are converted to git branches of
the main tree... but won't they still require rebasing as they've already
been pushed? (This assumes your workaround idea doesn't work. If it
does, great!)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

michael at orlitzky

May 31, 2012, 1:27 PM

Post #114 of 163 (183 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On 05/31/12 16:09, Michał Górny wrote:
> On Thu, 31 May 2012 15:58:43 -0400
> Rich Freeman <rich0 [at] gentoo> wrote:
>
>> On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo>
>> wrote:
>>> What would git signing work with rebased commits? Would all of them
>>> have to be signed once again?
>>>
>>
>> The whole point of rebasing is to throw away history (which is either
>> good or bad based on your perspective).
>>
>> So, if 14 devs spend 3 years and 2000 commits working on something in
>> a branch, and I commit it to master using a rebase, then all you'll
>> see in the master history is that rich0 committed 20k lines of code to
>> master on May 31st, and that would be signed by me.
>>
>> I think that rebasing before merging is a pretty typical workflow
>> anyway - when you submit a patch to Linus, he doesn't really care that
>> you spent six months tweaking it - he just is getting a big blob of
>> code that either works or doesn't. Does all that sub-history really
>> matter? You could always push the branch to the repository if you
>> wanted to keep it on the side.
>
> That sounds like a pretty poor workflow to me. If I tweak something for
> 3 years, I'm likely to have a larger set of logically organized commits.
> I'm not saying it's unlikely I'm going to rebase fixes for obvious
> mistakes but putting everything onto one blob just makes the changes
> harder to read and less maintainable.
>
> For example, if I first create a basic function and then add additional
> options to it, I'm likely to keep those as separate commits. If one of
> the changes was a really bad idea, I'd like to revert the appropriate
> commit rather than removing all traces of it by hand and mistakenly
> introducing even worse breakage.
>

That isn't what rebasing does, unless you do an interactive rebase and
decide to squash the commits.

The usual use case for a rebase is just to avoid the ugly merge commit.
For example,

1. I clone your portage tree
2. I start making commits locally
3. You add a new package
4. I try to push my changes to you

Step 4 doesn't work because your repo has changed. I can either,

a) pull from you again (merge)
b) pull with a rebase

And then I should be able to push to you, assuming there were no conflicts.

The merge option works fine but generates an ugly merge commit. The
rebase takes our two histories and combines them into a nice linear
history. In this case, a rebase would (essentially) take all of my
commits and stick them on top of your "new package" commit. Afterwards,
it looks like there's a nice linear history: you added a package, and
then I did a bunch of other stuff. All of the commits are there.

Since that history is linear and it looks like just a bunch of stuff on
top of your existing tree, I can push it to you without problems.

The only downside to the rebase is that it modifies my local history.
So, if somebody cloned *my* repo in the middle of that, they could get
screwed up.

williamh at gentoo

May 31, 2012, 1:57 PM

Post #115 of 163 (185 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 08:26:58PM +0000, Duncan wrote:
> William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:
>
> I don't know what's going to happen to all the overlays with the main
> tree switch to git, but won't that break various "overlay first"
> policies, say for the kde overlay?
>
> Of course, if all the official overlays are converted to git branches of
> the main tree... but won't they still require rebasing as they've already
> been pushed? (This assumes your workaround idea doesn't work. If it
> does, great!)

Overlays aren't really part of this discussion; those are independent
trees which we have no control over, so commiting changes from overlays
to the main tree is the responsibility of the overlay maintainers.

William

williamh at gentoo

May 31, 2012, 2:06 PM

Post #116 of 163 (184 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 03:58:43PM -0400, Rich Freeman wrote:
> On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo> wrote:
> > What would git signing work with rebased commits? Would all of them
> > have to be signed once again?
> >
>
> The whole point of rebasing is to throw away history (which is either
> good or bad based on your perspective).
>
> So, if 14 devs spend 3 years and 2000 commits working on something in
> a branch, and I commit it to master using a rebase, then all you'll
> see in the master history is that rich0 committed 20k lines of code to
> master on May 31st, and that would be signed by me.

You don't commit to master with a rebase,; it is always a merge. The
type of merge is what controls what you see in the logs.

If you rebase your branch on master, merge to master then run "git pull
--rebase" then push, you will get a fast forward merge, which shows the
individual commits.

If you don't include the rebasing, you get another type of merge which
just shows up in the logs as one commit afaik.

William

kentfredric at gmail

May 31, 2012, 2:52 PM

Post #117 of 163 (182 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On 1 June 2012 07:52, Alexey Shvetsov <alexxy [at] gentoo> wrote:
>>
>> What would git signing work with rebased commits? Would all of them
>> have to be signed once again?
>
>
> Commits itsels still will be signed

Do you know how git does this? Do you have experience/information you
can cite as to that this works?

Commit signing seems poorly documented at present, and I've been
looking at the git internals, and it would *APPEAR* that the content
that is signed is the blob of text you normally get when you

git cat-file -p $SHA1

And indeed, if you git cat-file -p $SHA1 > file, extract the
SIGNATURE part into its own file (removing the leading spaces), and
remove the "gnupg" section from the commit headers, gpg --verify
$sigfile $file # tells me I have a good signature.

Just I haven't worked out what happens when the SHA1 of the 'parent'
header changes, which *will* change if the rebase is anything other
than a fast-forward.

If that SHA1 changes, the gpg signature will surely fail?

--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

kentfredric at gmail

May 31, 2012, 3:00 PM

Post #118 of 163 (182 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On 1 June 2012 07:58, Rich Freeman <rich0 [at] gentoo> wrote:
> On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo> wrote:
>> What would git signing work with rebased commits? Would all of them
>> have to be signed once again?
>>
>
> The whole point of rebasing is to throw away history (which is either
> good or bad based on your perspective).
>
> So, if 14 devs spend 3 years and 2000 commits working on something in
> a branch, and I commit it to master using a rebase, then all you'll
> see in the master history is that rich0 committed 20k lines of code to
> master on May 31st, and that would be signed by me.
>
> I think that rebasing before merging is a pretty typical workflow
> anyway - when you submit a patch to Linus, he doesn't really care that
> you spent six months tweaking it - he just is getting a big blob of
> code that either works or doesn't. Does all that sub-history really
> matter? You could always push the branch to the repository if you
> wanted to keep it on the side.
>
> Rich
>

I think you're conflating "rebasing" and "squashing commits".

You should rebase a long commit sequence and squash pointless fixup
commits, and to make the commit sequence logical and ordered, possibly
divided by logical changes that one may wish to later revert. ( That
way, backing out a problem is simply reversing that commit and
applying the reversal patch )

You should not rebase for the purpose of squashing an entire history
of changes into a single scattered commit.

Rebasing is more to make the history itself linear and non-complex,
as when walking backwards through history, there being 2 parallel
histories that generated a merged commit can be confusing for humans,
so eliminating the parallel histories is one of the primary purposes I
advocate use of rebase for.

Squashed commits are a handy feature of rebase, but I wouldn't want to
see an entire overlay squashed into the main tree as a single squashed
commit.

Another bad reason for squashed commits: if you're getting rid of the
Changes files, you'll have no history on anything if you just group
long histories into a single commit. None.

--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

williamh at gentoo

May 31, 2012, 3:04 PM

Post #119 of 163 (182 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote:
> On Thu, 31 May 2012 14:18:04 -0500
> William Hubbs <williamh [at] gentoo> wrote:
> > > Not sure I'm following, but I will be the first to admit that I'm a
> > > git novice. Would this be aided by a convention, like only
> > > committing to master on the gentoo official repository, and any
> > > on-the-side work on places like github/etc stays in branches?
> > > Those repositories would just keep getting fed commits on master
> > > from the official repository.
> >
> > Iagree with this; I think we should ban merge commits on master. That
> > would force everyone to rebase their work on current master before
> > they commit to master which would make the history clean.
>
> So what's the point of switching to git if you want to ban the main
> reason git exists?

To clarify: we should only allow fast-forward merges on master.

My big complaint about merge commits is if you do a git show <hash> on
a merge commit, you get nothing, so there is no way to see what actually
changed in that commit.

William

kentfredric at gmail

May 31, 2012, 3:04 PM

Post #120 of 163 (182 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On 1 June 2012 08:26, Duncan <1i5t5.duncan [at] cox> wrote:
> William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:
> Of course, if all the official overlays are converted to git branches of
> the main tree... but won't they still require rebasing as they've already
> been pushed? (This assumes your workaround idea doesn't work. If it
> does, great!)
>

End users will still want to work with overlays that are not merged
with the main tree, not merely git branches.

Its foreseeable that there will be git branches that /track/ overlays
and exist as an integration pipeline for content from the overlays
joining core gentoo, but end users will not want to use that.

For the simple reason of course, that as soon as you want >1 overlay,
portage's way of doing it with separate repositories is far more
effective.

You don't want each user to have to maintain an octopus merge between
all the branches they want to have commits from ;)

--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

peter at stuge

May 31, 2012, 3:28 PM

Post #121 of 163 (184 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

William Hubbs wrote:
> To clarify: we should only allow fast-forward merges on master.

Not a dev yet, but +1

rich0 at gentoo

May 31, 2012, 7:49 PM

Post #122 of 163 (181 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, May 31, 2012 at 5:52 PM, Kent Fredric <kentfredric [at] gmail> wrote:
> Just I haven't worked out what happens when the SHA1 of the 'parent'
> header changes, which *will* change if the rebase is anything other
> than a fast-forward.
>
> If that SHA1 changes, the gpg signature will surely fail?

Rebasing doesn't modify past commits - it creates new ones and the
past ones are no longer in the history of the current head. So, it
doesn't break the old signatures so much as discard them. You would
need to create new signatures in their place, presumably from whoever
performed the rebase.

I'm trying to glean what I can from the little info out there about
how the new commit signatures work, but I don't think that you can use
signatures to convey authorship if later authors are going to rebase
the branch. The situation is not unlike what we have now with
manifests.

As far as I can tell if you want to do work outside of master, and
then get those commits into master but preserve all the past
signatures in the history of master, then you'd need to do a merge
commit, so that all the deltas to do the merge are in a separate
commit which is then signed by the person doing the merge.

Rich

mgorny at gentoo

May 31, 2012, 8:59 PM

Post #123 of 163 (181 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, 31 May 2012 17:04:30 -0500
William Hubbs <williamh [at] gentoo> wrote:

> On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote:
> > On Thu, 31 May 2012 14:18:04 -0500
> > William Hubbs <williamh [at] gentoo> wrote:
> > > > Not sure I'm following, but I will be the first to admit that
> > > > I'm a git novice. Would this be aided by a convention, like
> > > > only committing to master on the gentoo official repository,
> > > > and any on-the-side work on places like github/etc stays in
> > > > branches? Those repositories would just keep getting fed
> > > > commits on master from the official repository.
> > >
> > > Iagree with this; I think we should ban merge commits on master.
> > > That would force everyone to rebase their work on current master
> > > before they commit to master which would make the history clean.
> >
> > So what's the point of switching to git if you want to ban the main
> > reason git exists?
>
> To clarify: we should only allow fast-forward merges on master.
>
> My big complaint about merge commits is if you do a git show <hash> on
> a merge commit, you get nothing, so there is no way to see what
> actually changed in that commit.

Or you use a graphical tool which shows the whole merge history and you
see the exact changes happening rather than some blob with 'do foo, do
bar, and some baz too'.

--
Best regards,
Michał Górny

Attachments:

signature.asc (0.31 KB)

mgorny at gentoo

May 31, 2012, 9:05 PM

Post #124 of 163 (182 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On Thu, 31 May 2012 16:27:48 -0400
Michael Orlitzky <michael [at] orlitzky> wrote:

> On 05/31/12 16:09, Michał Górny wrote:
> > On Thu, 31 May 2012 15:58:43 -0400
> > Rich Freeman <rich0 [at] gentoo> wrote:
> >
> >> On Thu, May 31, 2012 at 3:33 PM, Michał Górny <mgorny [at] gentoo>
> >> wrote:
> >>> What would git signing work with rebased commits? Would all of
> >>> them have to be signed once again?
> >>>
> >>
> >> The whole point of rebasing is to throw away history (which is
> >> either good or bad based on your perspective).
> >>
> >> So, if 14 devs spend 3 years and 2000 commits working on something
> >> in a branch, and I commit it to master using a rebase, then all
> >> you'll see in the master history is that rich0 committed 20k lines
> >> of code to master on May 31st, and that would be signed by me.
> >>
> >> I think that rebasing before merging is a pretty typical workflow
> >> anyway - when you submit a patch to Linus, he doesn't really care
> >> that you spent six months tweaking it - he just is getting a big
> >> blob of code that either works or doesn't. Does all that
> >> sub-history really matter? You could always push the branch to
> >> the repository if you wanted to keep it on the side.
> >
> > That sounds like a pretty poor workflow to me. If I tweak something
> > for 3 years, I'm likely to have a larger set of logically organized
> > commits. I'm not saying it's unlikely I'm going to rebase fixes for
> > obvious mistakes but putting everything onto one blob just makes
> > the changes harder to read and less maintainable.
> >
> > For example, if I first create a basic function and then add
> > additional options to it, I'm likely to keep those as separate
> > commits. If one of the changes was a really bad idea, I'd like to
> > revert the appropriate commit rather than removing all traces of it
> > by hand and mistakenly introducing even worse breakage.
> >
>
> That isn't what rebasing does, unless you do an interactive rebase and
> decide to squash the commits.

Yes, it isn't but such kind of work flow was presented in the message I
replied to.

> The usual use case for a rebase is just to avoid the ugly merge
> commit.

Which devs should simply do whenever they use the scenario you
mentioned. I agree we could block 'auto-merges' when pushing to
a modified repo but not disallow a valid merges from devs who know what
they're doing.

--
Best regards,
Michał Górny

Attachments:

signature.asc (0.31 KB)

kentfredric at gmail

May 31, 2012, 9:55 PM

Post #125 of 163 (181 views)
Permalink

Re: Re: Portage Git migration - clean cut or git-cvsserver [In reply to]

On 1 June 2012 14:49, Rich Freeman <rich0 [at] gentoo> wrote:
> On Thu, May 31, 2012 at 5:52 PM, Kent Fredric <kentfredric [at] gmail> wrote:
>> Just I haven't worked out what happens when the SHA1 of the 'parent'
>> header changes, which *will* change if the rebase is anything other
>> than a fast-forward.
>>
>> If that SHA1 changes, the gpg signature will surely fail?
>
> Rebasing doesn't modify past commits - it creates new ones and the
> past ones are no longer in the history of the current head. So, it
> doesn't break the old signatures so much as discard them. You would
> need to create new signatures in their place, presumably from whoever
> performed the rebase.

Hmm, thats annoying. Almost makes me wish it was the trees that were
signed, not the commits.

Although, I probably could brew up a prototype resigning tool ( based
on Git::PurePerl ,... when they accept and publish my changes ) , just
would be problematic because simply the act of signing a past commit
means the SHA1 of the commit itself is different, so all successive
commits after a re-signed commit will change and also need to be
rebased and re-signed.

--
Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

1 2 3 4 5 6 7

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads