Mailing List Archive: Maemo: Developers

Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle))

Index | Next | Previous | View Threaded

andrew at bleb

Nov 10, 2009, 8:11 AM

Post #1 of 7 (586 views)
Permalink

Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle))

On Tue, Nov 10, 2009 at 16:00, Anderson Lizardo
<anderson.lizardo [at] openbossa> wrote:
>
> The PyMaemo team is pleased to announce the final release of PyMaemo
> for Maemo 5!

BTW, I've tested with bind mounts and /opt.

I've done the following and it seems to work well:

* Created /opt/python2.5/lib
* Moved the contents of /usr/lib/python2.5 to /opt/python2.5/lib
* Created an init script, symlinked to S20python-optify, which does
(on start):

mount --bind /opt/python2.5/lib /usr/lib/python2.5

This freed up lots of space on the rootfs and does not seem to have
impacted start-up time of Python apps particularly noticably.

I can share the startup/shutdown script and maybe even package this as
a "Python Optifier" if you want.

I think this is one of the best ways of optifying Python and without
any patches. I'd suggest this gets included in the next release of
PyMaemo.

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Nov 10, 2009, 8:33 AM

Post #2 of 7 (561 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

On Tue, Nov 10, 2009 at 12:11 PM, Andrew Flegg <andrew [at] bleb> wrote:
> On Tue, Nov 10, 2009 at 16:00, Anderson Lizardo
> <anderson.lizardo [at] openbossa> wrote:
>>
>> The PyMaemo team is pleased to announce the final release of PyMaemo
>> for Maemo 5!
>
> BTW, I've tested with bind mounts and /opt.
> [...]

Nice to hear that! We decided to leave out the optification for the
final release, just not to delay it even more. But now I believe we
can work on it as an update through extras-devel (I just hope that
that QA process will take any possible regressions with the new
packages we upload).

> I've done the following and it seems to work well:
>
> * Created /opt/python2.5/lib
> * Moved the contents of /usr/lib/python2.5 to /opt/python2.5/lib
> * Created an init script, symlinked to S20python-optify, which does
> (on start):
>
> mount --bind /opt/python2.5/lib /usr/lib/python2.5
>
> This freed up lots of space on the rootfs and does not seem to have
> impacted start-up time of Python apps particularly noticably.
>
> I can share the startup/shutdown script and maybe even package this as
> a "Python Optifier" if you want.

Do you think it can be made a generic dh_* like tool that handles this
automatically? This way it could be called from debian/rules as e.g.:

maemo-python-optify /usr/lib/python2.5

and the init scripts be generated automatically. What do you think?

> I think this is one of the best ways of optifying Python and without
> any patches. I'd suggest this gets included in the next release of
> PyMaemo.

Did you do any kind of upgrade tests? I'm worried how that would work
if you are upgrading from an older non-optitified python installation.

I also suppose that this implicitly moves files from other packages to
/opt/python2.5 ? E.g. from python-hildon, python-gtk2 etc. ?

What about the /usr/bin/python2.5 binary (which takes some MB) did you
move it to /opt too?

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Nov 10, 2009, 8:45 AM

Post #3 of 7 (558 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

On Tue, Nov 10, 2009 at 16:33, Anderson Lizardo
<anderson.lizardo [at] openbossa> wrote:
>
> Nice to hear that! We decided to leave out the optification for the
> final release, just not to delay it even more. But now I believe we
> can work on it as an update through extras-devel (I just hope that
> that QA process will take any possible regressions with the new
> packages we upload).

Agreed.

> Do you think it can be made a generic dh_* like tool that handles this
> automatically? This way it could be called from debian/rules as e.g.:
>
> maemo-python-optify /usr/lib/python2.5
>
> and the init scripts be generated automatically. What do you think?

Could do. After a discussion on #maemo, it seems something might have
to go in /etc/event.d and I tricked myself into thinking the mount
happened at boot.

> Did you do any kind of upgrade tests? I'm worried how that would work
> if you are upgrading from an older non-optitified python installation.

This should work as we're not replacing anything with a symlink (the
big problem with upgrading an existing install).

> I also suppose that this implicitly moves files from other packages to
> /opt/python2.5 ? E.g. from python-hildon, python-gtk2 etc. ?

There are two things we probably have to do:

1) Install /opt/python2.5/lib/... using dpkg.
2) Move anything left in /usr/lib/python2.5/ manually after install.

Future installs of something like python-hildon will be into the bind
mount, and so go to the right place automatically. maemo-optify will
be updated to not touch anything which depends on Python if we do
this, so Python libraries will continue to install into
/usr/lib/python2.5/site-packages. It's just that this will now be on
the eMMC.

> What about the /usr/bin/python2.5 binary (which takes some MB) did you
> move it to /opt too?

No, I didn't try that. Do you think a symlink should work for that?

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

eero.tamminen at nokia

Nov 12, 2009, 7:30 AM

Post #4 of 7 (538 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

Hi,

ext Anderson Lizardo wrote:
> On Tue, Nov 10, 2009 at 12:11 PM, Andrew Flegg <andrew [at] bleb> wrote:
>> On Tue, Nov 10, 2009 at 16:00, Anderson Lizardo
>> <anderson.lizardo [at] openbossa> wrote:
>>> The PyMaemo team is pleased to announce the final release of PyMaemo
>>> for Maemo 5!
>> BTW, I've tested with bind mounts and /opt.
>> [...]
>
> Nice to hear that! We decided to leave out the optification for the
> final release, just not to delay it even more. But now I believe we
> can work on it as an update through extras-devel (I just hope that
> that QA process will take any possible regressions with the new
> packages we upload).
>
>> I've done the following and it seems to work well:
>>
>> * Created /opt/python2.5/lib
>> * Moved the contents of /usr/lib/python2.5 to /opt/python2.5/lib
>> * Created an init script, symlinked to S20python-optify, which does
>> (on start):
>>
>> mount --bind /opt/python2.5/lib /usr/lib/python2.5
>>
>> This freed up lots of space on the rootfs and does not seem to have
>> impacted start-up time of Python apps particularly noticably.
>>
>> I can share the startup/shutdown script and maybe even package this as
>> a "Python Optifier" if you want.
>
> Do you think it can be made a generic dh_* like tool that handles this
> automatically? This way it could be called from debian/rules as e.g.:
>
> maemo-python-optify /usr/lib/python2.5
>
> and the init scripts be generated automatically. What do you think?

Are you suggesting that each python package would themselves do
the bind mount? And hide anything that was before in that directory?

Saner solution is that the bind mount is done by something from which
the package depends from (be it python package itself or something
else).

>> I think this is one of the best ways of optifying Python and without
>> any patches. I'd suggest this gets included in the next release of
>> PyMaemo.
>
> Did you do any kind of upgrade tests? I'm worried how that would work
> if you are upgrading from an older non-optitified python installation.
>
> I also suppose that this implicitly moves files from other packages to
> /opt/python2.5 ? E.g. from python-hildon, python-gtk2 etc. ?
>
> What about the /usr/bin/python2.5 binary (which takes some MB) did you
> move it to /opt too?

- Eero
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Nov 12, 2009, 8:00 AM

Post #5 of 7 (539 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

On Thu, Nov 12, 2009 at 11:30 AM, Eero Tamminen <eero.tamminen [at] nokia> wrote:
> ext Anderson Lizardo wrote:
>> Do you think it can be made a generic dh_* like tool that handles this
>> automatically? This way it could be called from debian/rules as e.g.:
>>
>> maemo-python-optify /usr/lib/python2.5
>>
>> and the init scripts be generated automatically. What do you think?
>
> Are you suggesting that each python package would themselves do
> the bind mount? And hide anything that was before in that directory?
>
> Saner solution is that the bind mount is done by something from which
> the package depends from (be it python package itself or something
> else).

No no, I'm just suggesting make it a generic, more automatic tool
(like maemo-optifier itself) that can be used by other bigger packages
that maemo-optify cannot handle generically (although I'm not aware of
any other cases). This is optional though, and for Python we will
handle it now instead of waiting for this tool to be written.

For Python, we will bind mount just the core python2.5 package. This
way, all packages that depend on python2.5 and install
modules/extensions to /usr/lib/python2.5 will benefit from it
transparently.

I should also remember that we have to handle packages that use
python-central/python-support too, as they move the extensions to
other places (directories under /usr/share/... and /var/lib/... IIRC).
They will be handled similarly as well.

We will also make sure all possible installation paths (fresh install,
upgrade from optified package, downgrade to non-optified package,
removal) are properly handled so that we can cleanly move in/out from
the "bind mount" solution and avoid upgrade breakages.

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

corsac at debian

Nov 12, 2009, 11:18 PM

Post #6 of 7 (522 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

On mar., 2009-11-10 at 12:33 -0400, Anderson Lizardo wrote:
> Nice to hear that! We decided to leave out the optification for the
> final release, just not to delay it even more. But now I believe we
> can work on it as an update through extras-devel (I just hope that
> that QA process will take any possible regressions with the new
> packages we upload).

Hmhm so everything will install on OneNAND ? Or everything will install
on eMMC ?

--
Yves-Alexis

Attachments:

signature.asc (0.19 KB)

anderson.lizardo at openbossa

Nov 13, 2009, 3:31 AM

Post #7 of 7 (527 views)
Permalink

Re: Optification (was Re: PyMaemo (Python for Maemo) final release for Maemo 5 (Fremantle)) [In reply to]

On Fri, Nov 13, 2009 at 3:18 AM, Yves-Alexis Perez <corsac [at] debian> wrote:
> On mar., 2009-11-10 at 12:33 -0400, Anderson Lizardo wrote:
>> Nice to hear that! We decided to leave out the optification for the
>> final release, just not to delay it even more. But now I believe we
>> can work on it as an update through extras-devel (I just hope that
>> that QA process will take any possible regressions with the new
>> packages we upload).
>
> Hmhm so everything will install on OneNAND ? Or everything will install
> on eMMC ?

The current release installs on the flash root device, but we are
already working on a new version that will install into the eMMC.

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads