Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Dev

Hg mirror available

 

 

Zope dev RSS feed   Index | Next | Previous | View Threaded


sdouche at gmail

Jun 17, 2009, 4:34 PM

Post #1 of 8 (841 views)
Permalink
Hg mirror available

This is a first attempt to build an Mercurial mirror :
http://hg.zope.mirrors.securactive.org/

- Size
SVN 2.13go
Hg 780mo
Hg (only trunk for each package) 440mo

- co (only last commit)
SVN
seb [at] wiene:~$ time svn co svn://svn.zope.org/repos/main/zope.component/trunk
...
real 0m5.158s
user 0m0.028s
sys 0m0.996s

Hg (all commits, it's a clone)
seb [at] wiene:~$ time hg clone
http://hg.zope.mirrors.securactive.org/zope.component/
...
real 0m3.394s
user 0m0.280s
sys 0m0.552s


Improvments
-------------------
- mapping of svn user / full name & email (hg convention)
- a description for each project


What do you think?

Hope this help everybody to understand the benefits of DVCS and
leverage facility to submit a patch.




--
Sebastien Douche <sdouche [at] gmail>
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

Jun 18, 2009, 3:31 AM

Post #2 of 8 (781 views)
Permalink
Re: Hg mirror available [In reply to]

* Sebastien Douche <sdouche [at] gmail> [2009-06-18 01:34]:
> This is a first attempt to build an Mercurial mirror :
> http://hg.zope.mirrors.securactive.org/

How did you convert the repository?

I'm asking because I noticed that basically all SVN->DVCS conversion
tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py,
svn-all-fast-export.cpp) do not convert the history properly, more
precisely, history that happened on branches is lost:

1. import /trunk/foo.txt
2. branch /trunk to /branches/mybranch
3. edit /mybranch/foo.txt
4. merge /mybranch to /trunk

If you now ask svn for the history of /trunk/foo.txt (say with 'svn
log'), you see both steps 3 and 4. After conversion to a DVCS with one
of the above mentioned tools, you only see step 4, while step 3 never
happened in the DVCS repository. I think that's unacceptable, most
importantly because all commit messages that happened on branches are
lost that way.

Does somebody here know something about this phenomenon, by any chance?
Am I missing something?

Wolfgang

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


sidnei.da.silva at gmail

Jun 18, 2009, 10:28 AM

Post #3 of 8 (775 views)
Permalink
Re: Hg mirror available [In reply to]

Hi Wolfgang,

On Thu, Jun 18, 2009 at 7:31 AM, Wolfgang Schnerring<ws [at] gocept> wrote:
> * Sebastien Douche <sdouche [at] gmail> [2009-06-18 01:34]:
>> This is a first attempt to build an Mercurial mirror :
>> http://hg.zope.mirrors.securactive.org/
>
> How did you convert the repository?
>
> I'm asking because I noticed that basically all SVN->DVCS conversion
> tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py,
> svn-all-fast-export.cpp) do not convert the history properly, more
> precisely, history that happened on branches is lost:
>
> 1. import /trunk/foo.txt
> 2. branch /trunk to /branches/mybranch
> 3. edit /mybranch/foo.txt
> 4. merge /mybranch to /trunk
>
> If you now ask svn for the history of /trunk/foo.txt (say with 'svn
> log'), you see both steps 3 and 4. After conversion to a DVCS with one
> of the above mentioned tools, you only see step 4, while step 3 never
> happened in the DVCS repository. I think that's unacceptable, most
> importantly because all commit messages that happened on branches are
> lost that way.
>
> Does somebody here know something about this phenomenon, by any chance?
> Am I missing something?

Here's some context about this from one of the Bazaar developers, John
Arbash Meinel. Hopefully that will solve some of your questions?

"""
In pretty much all dvcs merging a content exactly back to trunk does not
generate a change message when doing "bzr log foo.txt".

I'm not really sure what he means by "edit" and then "merge" without a
commit inbetween. So I'm assuming there is one.

Now, what really matters is whether or not *Subversion* recorded 4
correctly, such that it can actually see that it was a merge from 3.

My understanding is that before svn 1.5 that isn't possible. So you are
left with trying to infer that sort of thing from the history. Which
would be possible, but probably expensive.

I'm pretty sure SVN represents (4) as not a *merge* but as an indentical
commit.

I don't have a great answer there. Though the fact that Wolfgang says
svn shows both... I suppose because "svn log" shows everything across
all branches?
"""

--
Sidnei da Silva
Canonical Ltd.
 Landscape · Changing the way you manage your systems
http://landscape.canonical.com
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ree at ree

Jun 18, 2009, 9:28 PM

Post #4 of 8 (766 views)
Permalink
Re: Hg mirror available [In reply to]

On Thu, 18 Jun 2009 14:28:13 -0300, Sidnei da Silva wrote:

> Hi Wolfgang,
>
> On Thu, Jun 18, 2009 at 7:31 AM, Wolfgang Schnerring<ws [at] gocept>
> wrote:
>> * Sebastien Douche <sdouche [at] gmail> [2009-06-18 01:34]:
>>> This is a first attempt to build an Mercurial mirror :
>>> http://hg.zope.mirrors.securactive.org/
>>
>> How did you convert the repository?
>>
>> I'm asking because I noticed that basically all SVN->DVCS conversion
>> tools (hg convert, git-svn, bzr svn-import, svn2bzr,
>> svn-fast-export.py, svn-all-fast-export.cpp) do not convert the history
>> properly, more precisely, history that happened on branches is lost:
>>
>> 1. import /trunk/foo.txt
>> 2. branch /trunk to /branches/mybranch 3. edit /mybranch/foo.txt
>> 4. merge /mybranch to /trunk
>>
>> If you now ask svn for the history of /trunk/foo.txt (say with 'svn
>> log'), you see both steps 3 and 4. After conversion to a DVCS with one
>> of the above mentioned tools, you only see step 4, while step 3 never
>> happened in the DVCS repository. I think that's unacceptable, most
>> importantly because all commit messages that happened on branches are
>> lost that way.
>>
>> Does somebody here know something about this phenomenon, by any chance?
>> Am I missing something?
>
> Here's some context about this from one of the Bazaar developers, John
> Arbash Meinel. Hopefully that will solve some of your questions?
>
> """
> In pretty much all dvcs merging a content exactly back to trunk does not
> generate a change message when doing "bzr log foo.txt".
>
> I'm not really sure what he means by "edit" and then "merge" without a
> commit inbetween. So I'm assuming there is one.
>
> Now, what really matters is whether or not *Subversion* recorded 4
> correctly, such that it can actually see that it was a merge from 3.
>
> My understanding is that before svn 1.5 that isn't possible. So you are
> left with trying to infer that sort of thing from the history. Which
> would be possible, but probably expensive.
>
> I'm pretty sure SVN represents (4) as not a *merge* but as an indentical
> commit.
>
> I don't have a great answer there. Though the fact that Wolfgang says
> svn shows both... I suppose because "svn log" shows everything across
> all branches?
> """

I'm somewhat confused here. According to my understanding:

- all DVCS shows, if a merge is done, the changesets that origin from the
merged branch. (this is the normal operation)

- afaik svn does _not_ show this, what's more, it does not store any
metadata about the merges or changesets involved. When doing the merge
you really select a diff of the branch by specifying which changesets you
want to include back in trunk. This is why it's so important with svn to
note in the commit message, which revisions from which branch you merged.
Otherwise you would not know at all what has been merged.

So although, DVCS could represent the information about the merged
changesets, this information will not be imported from SVN, simply,
because the information is not represented in SVN.

I'd like to add that I'm not using Hg, I am only using bazaar and svn,
and I'm talking from what I experienced in practice with working on
various svn repos and bzr. It's possible that newer svn does try to
attack this problem by storing more metadata with the merges, which then
would make sense to be considered at a DVCS import, but I believe that in
the vast majority of svn repositories that you would consider importing,
this information would not be there anyway, due to the fact that they are
product of the older svn version.

--
Balazs Ree

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

Jun 18, 2009, 10:43 PM

Post #5 of 8 (760 views)
Permalink
Re: Hg mirror available [In reply to]

Hi there,

* Sidnei da Silva <sidnei.da.silva [at] gmail> [2009-06-18 14:28]:
> > I'm asking because I noticed that basically all SVN->DVCS conversion
> > tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py,
> > svn-all-fast-export.cpp) do not convert the history properly, more
> > precisely, history that happened on branches is lost:
>
> Here's some context about this from one of the Bazaar developers, John
> Arbash Meinel. Hopefully that will solve some of your questions?

Thanks for relaying!

> I'm not really sure what he means by "edit" and then "merge" without a
> commit inbetween. So I'm assuming there is one.

Sorry for being overly brief in my description; please find below a
sample shell script to set up a Subversion repository with a project
containing a trunk and a branch. To reproduce the problem I'm
concerned about, do this:

$ create-repos.sh repos
$ svn log repos/project/trunk/feature.txt
------------------------------------------------------------------------
r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line

merged feature1
------------------------------------------------------------------------
r8 | wosc | 2009-06-18 17:11:43 +0200 (Thu, 18 Jun 2009) | 1 line

implemented feature1
------------------------------------------------------------------------

As you can see, Subversion reports the history of the file that
happenened on the branch (in r8).

After converting the repository, however...

$ bzr init-repo repos-bzr
$ cd repos-bzr
$ bzr svn-import file://$PWD/../repos
$ bzr log repos/project/trunk/feature.txt
------------------------------------------------------------
revno: 3
svn revno: 9 (on /project/trunk)
committer: wosc
timestamp: Thu 2009-06-18 14:11:46 +0000
message:
merged feature1

... this history is lost. Not only does it not appear in the log
output, also when looking at bzr visualize it seems like that branch
never existed at all in bzr. (I'm new to bzr, am I missing something
here?)

> Now, what really matters is whether or not *Subversion* recorded 4
> correctly, such that it can actually see that it was a merge from 3.
> My understanding is that before svn 1.5 that isn't possible.

My understanding is quite different. ;-)
As far as I know, SVN "always" has recorded the history of a file
across copies and merges, and so I'm quite sure this has nothing to do
with 1.5-style merge tracking. I won't try to track down a reference
on that right now, but I'm currently reading up on SVN's APIs so I
expect to find the answer there, anyway.

> I'm pretty sure SVN represents (4) as not a *merge* but as an indentical
> commit.

As I said, I'm still digging through the SVN APIs, so unfortunately I
don't have a concrete answer, but I know positively that this history
information is readily available, which you can see by passing
--verbose to 'svn log':

$ svn log -v -c 9 repos
------------------------------------------------------------------------
r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line
Changed paths:
M /project/trunk
A /project/trunk/feature.txt (from /project/branches/feature1/feature.txt:8)
>>>>>>>>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
M /project/trunk/foo.txt

merged feature1
------------------------------------------------------------------------

Wolfgang


**

#!/bin/sh

if [ $# -lt 1 ]; then
echo 1>&2 "Usage: $0 repos-path"
exit 1
fi

set -e

mkdir $1
repos=$(cd "$1" && pwd)
rmdir $1

working=$(tempfile)
rm $working
mkdir $working

svnadmin create $repos
svn mkdir file://$repos/project -m "creating project home"
svn mkdir file://$repos/project/trunk -m "creating trunk"
svn mkdir file://$repos/project/branches -m "creating branches"
svn mkdir file://$repos/project/tags -m "creating tags"
svn co file://$repos/project/trunk $working
cd $working

mkdir alpha
echo "foo" > foo.txt
echo "bar" > alpha/bar.txt
svn add foo.txt alpha
svn commit -m "initial version"

svn cp file://$repos/project/trunk file://$repos/project/tags/0.1 -m "tagging 0.1"

svn cp file://$repos/project/trunk file://$repos/project/branches/feature1 -m "creating branch 1"
svn switch file://$repos/project/branches/feature1
echo "qux" > foo.txt
echo "feature1" > feature.txt
svn add feature.txt
svn commit -m "implemented feature1"

svn switch file://$repos/project/trunk
svn merge file://$repos/project/branches/feature1
svn commit -m "merged feature1"
svn rm file://$repos/project/branches/feature1 -m "remove branch after merge"

svn cp file://$repos/project/trunk file://$repos/project/tags/0.2 -m "tagging 0.2"

svn cp file://$repos/project/trunk file://$repos/project/branches/feature2 -m "creating branch 2"
svn switch file://$repos/project/branches/feature2
echo "feature2" >> feature.txt
svn commit -m "implemented feature 2"

rm -rf $working
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


wichert at wiggy

Jun 19, 2009, 12:18 AM

Post #6 of 8 (766 views)
Permalink
Re: Hg mirror available [In reply to]

On 6/19/09 6:28 AM, Balazs Ree wrote:
> - afaik svn does _not_ show this, what's more, it does not store any
> metadata about the merges or changesets involved. When doing the merge
> you really select a diff of the branch by specifying which changesets you
> want to include back in trunk. This is why it's so important with svn to
> note in the commit message, which revisions from which branch you merged.
> Otherwise you would not know at all what has been merged.

Subversion 1.5 and later do store that data in a svn:mergeinfo property.

Wichert.


--
Wichert Akkerman <wichert [at] wiggy> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


sidnei.da.silva at gmail

Jun 19, 2009, 5:26 AM

Post #7 of 8 (762 views)
Permalink
Re: Hg mirror available [In reply to]

Hi Wolfgang,

On Fri, Jun 19, 2009 at 2:43 AM, Wolfgang Schnerring<ws [at] gocept> wrote:
> Hi there,
>
> * Sidnei da Silva <sidnei.da.silva [at] gmail> [2009-06-18 14:28]:
>> > I'm asking because I noticed that basically all SVN->DVCS conversion
>> > tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py,
>> > svn-all-fast-export.cpp) do not convert the history properly, more
>> > precisely, history that happened on branches is lost:
>>
>> Here's some context about this from one of the Bazaar developers, John
>> Arbash Meinel. Hopefully that will solve some of your questions?
>
> Thanks for relaying!
>
>> I'm not really sure what he means by "edit" and then "merge" without a
>> commit inbetween. So I'm assuming there is one.
>
> Sorry for being overly brief in my description; please find below a
> sample shell script to set up a Subversion repository with a project
> containing a trunk and a branch. To reproduce the problem I'm
> concerned about, do this:
>
> $ create-repos.sh repos
> $ svn log repos/project/trunk/feature.txt
> ------------------------------------------------------------------------
> r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line
>
> merged feature1
> ------------------------------------------------------------------------
> r8 | wosc | 2009-06-18 17:11:43 +0200 (Thu, 18 Jun 2009) | 1 line
>
> implemented feature1
> ------------------------------------------------------------------------
>
> As you can see, Subversion reports the history of the file that
> happenened on the branch (in r8).

Can you show us your 'svn --version'? I suspect what you're seeing is
a result of svn 1.5 merge tracking.

> After converting the repository, however...
>
> $ bzr init-repo repos-bzr
> $ cd repos-bzr
> $ bzr svn-import file://$PWD/../repos
> $ bzr log repos/project/trunk/feature.txt
> ------------------------------------------------------------
> revno: 3
> svn revno: 9 (on /project/trunk)
> committer: wosc
> timestamp: Thu 2009-06-18 14:11:46 +0000
> message:
>  merged feature1
>
> ... this history is lost. Not only does it not appear in the log
> output, also when looking at bzr visualize it seems like that branch
> never existed at all in bzr. (I'm new to bzr, am I missing something
> here?)

Like John said, it *should* be possible to bring that information over
if the svn repository is in 1.5 format and contains informations about
merge, but at the moment that (apparently) hasn't been implemented.

--
Sidnei da Silva
Canonical Ltd.
 Landscape · Changing the way you manage your systems
http://landscape.canonical.com
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

Jun 21, 2009, 11:02 PM

Post #8 of 8 (718 views)
Permalink
Re: Hg mirror available [In reply to]

Hello,

* Sidnei da Silva <sidnei.da.silva [at] gmail> [2009-06-19 14:26]:
> On Fri, Jun 19, 2009 at 2:43 AM, Wolfgang Schnerring<ws [at] gocept> wrote:
>> $ svn log repos/project/trunk/feature.txt
>> ------------------------------------------------------------------------
>> r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line
>>
>> merged feature1
>> ------------------------------------------------------------------------
>> r8 | wosc | 2009-06-18 17:11:43 +0200 (Thu, 18 Jun 2009) | 1 line
>>
>> implemented feature1
>> ------------------------------------------------------------------------
>>
>> As you can see, Subversion reports the history of the file that
>> happenened on the branch (in r8).
>
> Can you show us your 'svn --version'? I suspect what you're seeing is
> a result of svn 1.5 merge tracking.

I've dug a little deeper, and found that the example I presented is a
special case, since 'feature.txt' has been *added* on the branch, and
Subversion has indeed always treated the history of copied/added/removed
files differently.

You're right that in the general case, Subversion is only able to show
the whole history (including branches) using the 1.5-mergeinfo support.
One can invoke this via 'svn log --use-merge-history': on my example
repository, compare the output for project/trunk/foo.txt with and
without that switch; without it only shows the merge commit.

Hmm. This means that even if conversion tools did something useful with
the mergeinfo properties (which right now they don't), it would only
solve my specific problem of preserving history for the last 6-12 months
or so, since the repository I'm dealing with stems from 2003 or
something, meaning Subversion 1.2ish, which did not even *store* the
information necessary to recover the history.

I guess I'll need to think about this some more (and then move to a
different mailing list, as we're getting wayyyyy off-topic for zope-dev ;-)

Thanks for your insights,
Wolfgang

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Zope dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.