Mailing List Archive: Python: Dev

Building a Windows MSI for Python /trunk

Index | Next | Previous | View Threaded

ctb at msu

Nov 25, 2009, 7:39 PM

Post #1 of 5 (966 views)
Permalink

Building a Windows MSI for Python /trunk

Hi all,

I got an MSI build working on my WinXP VM just now, and I wanted to
touch base with whomever it is that is maintaining this (wonderful!)
set of scripts...

I ran into three problems, and I managed to figure out two of them; the third
wasn't fatal. Note, the diff of my fixed checkout is attached.

First, the script that finds & builds the external dependencies has two
minor problems.

* it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
tcl8.* and tk8.* to grab the license text. I changed the glob strings
appropriately and that seemed to work.

* Tix isn't downloaded/installed/built automatically like everything else,
and msi.py looks for its license file, too. I just removed the
Tix reference. I can't figure out how to build Tix appropriately; any
tips?

Second, the buildmsi.bat file refers to python26a3.hhp instead of
python27a0.hhp.

Third, I could not get _tkinter to build properly, although it wasn't fatal
to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although
tcl85g.lib existed.

Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
had to figure out with google -- things like nasm/nasmw, for example. Are
these documented somewhere, or would it be helpful to document them? I think I
had to install:

- Microsoft HTML Help Compiler
- cygwin with make and python2.5 to build the docs
- nasm (and copy nasm.exe to nasmw.exe)
- cabarc

Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

I'd love to get this build process working completely automatically and
100% correctly, too.

Hat tip to Trent Nelson, who helped me figure out where the scripts are
and what other things I needed...

cheers,
--titus
--
C. Titus Brown, ctb [at] msu

Attachments:

msi-build.diff (1.31 KB)

martin at v

Nov 26, 2009, 12:54 AM

Post #2 of 5 (909 views)
Permalink

Re: Building a Windows MSI for Python /trunk [In reply to]

> First, the script that finds & builds the external dependencies has two
> minor problems.
>
> * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
> tcl8.* and tk8.* to grab the license text. I changed the glob strings
> appropriately and that seemed to work.

Not sure what "it" is that put it there; perhaps you are referring to
the Tools/buildbot scripts?

Unfortunately, the naming convention that these scripts establish
doesn't quite work, as Tix would fail to build under these conventions.
So in my own checkout, I manually renamed the trees, and that's what
msi.py refers to.

> * Tix isn't downloaded/installed/built automatically like everything else,
> and msi.py looks for its license file, too. I just removed the
> Tix reference. I can't figure out how to build Tix appropriately; any
> tips?

See above. I keep forgetting how to build Tix; one set of command lines
is in PCbuild/build_tkinter.py.

> Second, the buildmsi.bat file refers to python26a3.hhp instead of
> python27a0.hhp.

Yes, that script hasn't been updated for a long time, ever since we
stopped having automated builds.

Ideally, the script would find the Python version on its own.

> Third, I could not get _tkinter to build properly, although it wasn't fatal
> to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> tcl85g.lib existed.

You'll need to build release versions of Tcl/Tk/Tix.

> Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
> had to figure out with google -- things like nasm/nasmw, for example. Are
> these documented somewhere, or would it be helpful to document them?

See PCbuild/readme.txt.

> Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

Thanks, fixed.

> I'd love to get this build process working completely automatically and
> 100% correctly, too.
>
> Hat tip to Trent Nelson, who helped me figure out where the scripts are
> and what other things I needed...

Feel free to submit patches. There may be a misunderstanding, though, in
that buildmsi.bat isn't actually used for anything, and isn't "meant" to
be run by users, but instead by build slaves.

One consequence is that these scripts build in debug mode (perhaps
except for buildmsi), whereas end users would typically want to build in
release mode.

Another consequence is that different committers actually use different
procedures. Trent created much of the Tools/buildbot setup, so he is
obviously in favor of that strategy. Christian Heimes created
PCbuild/build_tkinter, so he would probably prefer to use that instead.
When I took over Windows builds from Tim Peters, PCbuild/readme.txt
was the official reference, and I try to stick to that.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ctb at msu

Nov 26, 2009, 6:03 AM

Post #3 of 5 (907 views)
Permalink

Re: Building a Windows MSI for Python /trunk [In reply to]

On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote:
> > First, the script that finds & builds the external dependencies has two
> > minor problems.
> >
> > * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
> > tcl8.* and tk8.* to grab the license text. I changed the glob strings
> > appropriately and that seemed to work.
>
> Not sure what "it" is that put it there; perhaps you are referring to
> the Tools/buildbot scripts?

Yes, sorry, Tools/buildbot/external*.bat, as in the diff.

> Unfortunately, the naming convention that these scripts establish
> doesn't quite work, as Tix would fail to build under these conventions.
> So in my own checkout, I manually renamed the trees, and that's what
> msi.py refers to.

OK, thanks!

> > * Tix isn't downloaded/installed/built automatically like everything else,
> > and msi.py looks for its license file, too. I just removed the
> > Tix reference. I can't figure out how to build Tix appropriately; any
> > tips?
>
> See above. I keep forgetting how to build Tix; one set of command lines
> is in PCbuild/build_tkinter.py.

Ok, thanks!

> > Second, the buildmsi.bat file refers to python26a3.hhp instead of
> > python27a0.hhp.
>
> Yes, that script hasn't been updated for a long time, ever since we
> stopped having automated builds.
>
> Ideally, the script would find the Python version on its own.

Right...

> > Third, I could not get _tkinter to build properly, although it wasn't fatal
> > to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> > tcl85g.lib existed.
>
> You'll need to build release versions of Tcl/Tk/Tix.

Yes, I saw a reference to that online, but I wasn't sure if that was the
problem -- is the naming convention really that 'tcl85g.lib' is the debug
lib!?

> > Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
> > had to figure out with google -- things like nasm/nasmw, for example. Are
> > these documented somewhere, or would it be helpful to document them?
>
> See PCbuild/readme.txt.

OK, thanks.

> > I'd love to get this build process working completely automatically and
> > 100% correctly, too.
> >
> > Hat tip to Trent Nelson, who helped me figure out where the scripts are
> > and what other things I needed...
>
> Feel free to submit patches. There may be a misunderstanding, though, in
> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
> be run by users, but instead by build slaves.

Well, yes, although wouldn't we want the scripts to run without any user
editing, for the build slaves?

> One consequence is that these scripts build in debug mode (perhaps
> except for buildmsi), whereas end users would typically want to build in
> release mode.

Right.

Is there a reason the buildbot builds were stopped? I've added binary
file uploads to pony-build, so that platform-specific files can be
uploaded to build results -- see, e.g.

http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/

or

http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/

and I thought daily -latest builds for Python latest on Windows and Mac would
be a good demo...

Incidentally, the pony-build results & file upload scripts can be used from
within buildbot, if people are interested.

> Another consequence is that different committers actually use different
> procedures. Trent created much of the Tools/buildbot setup, so he is
> obviously in favor of that strategy. Christian Heimes created
> PCbuild/build_tkinter, so he would probably prefer to use that instead.
> When I took over Windows builds from Tim Peters, PCbuild/readme.txt
> was the official reference, and I try to stick to that.

OK, I see. Thanks! So if I can bring the scripts into concordance with
that it might be valuable?

--titus
--
C. Titus Brown, ctb [at] msu
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

martin at v

Nov 26, 2009, 7:07 AM

Post #4 of 5 (906 views)
Permalink

Re: Building a Windows MSI for Python /trunk [In reply to]

>> You'll need to build release versions of Tcl/Tk/Tix.
>
> Yes, I saw a reference to that online, but I wasn't sure if that was the
> problem -- is the naming convention really that 'tcl85g.lib' is the debug
> lib!?

Yes; Tcl apparently follows Unix conventions here rather than Windows
conventions (where the debug build would end with "d").

>> Feel free to submit patches. There may be a misunderstanding, though, in
>> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
>> be run by users, but instead by build slaves.
>
> Well, yes, although wouldn't we want the scripts to run without any user
> editing, for the build slaves?

Sure. For the scripts that are in actual use, that's also the case. I
stopped working on buildmsi because it was just too tedious to get
right.

> Is there a reason the buildbot builds were stopped?

It was too difficult to get right, plus nobody was interested in
using the daily MSI files.

At some point, it was broken for several months (IIRC), with nobody
reporting that breakage. So when we noticed, we just turned the service
off.

> and I thought daily -latest builds for Python latest on Windows and Mac would
> be a good demo...

I'll guess that you will find the same: nobody is really interested
in *using* daily builds of Python.

> OK, I see. Thanks! So if I can bring the scripts into concordance with
> that it might be valuable?

Most certainly, yes.

Regards,
Martin

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

db3l.net at gmail

Nov 26, 2009, 3:14 PM

Post #5 of 5 (895 views)
Permalink

Re: Building a Windows MSI for Python /trunk [In reply to]

"Martin v. Löwis" <martin [at] v> writes:

> It was too difficult to get right, plus nobody was interested in
> using the daily MSI files.
>
> At some point, it was broken for several months (IIRC), with nobody
> reporting that breakage. So when we noticed, we just turned the service
> off.

Sounds right - the daily MSIs were built on my slave from Sep 2007
into 2008 somewhere, but it had been failing for 6 months or more when
the task was turned off last December. I'm not sure precisely when
the failures started (the buildbot status page was also down for a
while starting in March), but it had been failing for a long time.

I believe the failures at the time were tcl related - maybe some of
the MSI buildbot-related scripts getting out of date with ongoing
development or something. Sounds like it might be the same stuff
Titus is running into. It didn't seem worth the effort to track down
at the time, given that nobody had seemed to notice the lack of
builds.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads