Mailing List Archive: Maemo: Developers

Ovi Store disk space requirements miscalculations

Index | Next | Previous | View Threaded

martin at martin

Jul 21, 2010, 5:31 AM

Post #1 of 9 (433 views)
Permalink

Ovi Store disk space requirements miscalculations

Hi,

I just got my initial Maemo Ovi Store application rejected in QA, and I'd
like discuss the measurements used in the disk space usage requirement.

The part of the report is this:

ISSUE #1 SUMMARY:
OPT folder is not 80% more than that of USR folder.

PRECONDITION:
N/A.

STEPS:
1. Open Xterminal, cd MyDocs.
2. cp <packagename> /home/user/.
3. cd /home/user/.
4. mkdir <packagename>-install.
5. dpkg -x ./<packagename> <packagename>-install.
6. cd <packagename>-install.
7. ls.

EXPECTED OUTCOME:
OPT folder value should be 80% of USR folder.

ACTUAL OUTCOME:
Opt folder value is not 80% of usr folder ,usr=84,opt=388k.
FREQUENCY OF OCCURRENCE:.
Always.

If I do those steps (but do 'du' instead of 'ls' in step 7), I get the
following:

376 ./opt/maemo/usr/bin
380 ./opt/maemo/usr
384 ./opt/maemo
388 ./opt
4 ./usr/sbin
4 ./usr/bin
8 ./usr/share/icons/hicolor/64x64/apps
12 ./usr/share/icons/hicolor/64x64
8 ./usr/share/icons/hicolor/26x26/apps
12 ./usr/share/icons/hicolor/26x26
8 ./usr/share/icons/hicolor/40x40/apps
12 ./usr/share/icons/hicolor/40x40
40 ./usr/share/icons/hicolor
44 ./usr/share/icons
8 ./usr/share/doc/<appname>
12 ./usr/share/doc
8 ./usr/share/applications/hildon
12 ./usr/share/applications
72 ./usr/share
84 ./usr
476 .

According to this, my app is wasting 84 KB of space on /usr. Let's see
what it _really_ does. In practice, it installs three icons in
/usr/share/icons, 751, 1075 and 1570 bytes in size, one 24 byte copyright
file in /usr/share/doc/<appname>, and a 181 .desktop file in
/usr/share/applications.

Since this is done on /home which is ext3, every directory consumes 4 KB,
too. In practice, if I'd install the package, all those directories
(except the /usr/share/doc/<appname> dir) would exist already, so no disk
space would be consumed for them.

Optifying them would only help a little - symlinks seem to count as 0
bytes in this listing, but the number of directories would still add up to
way too much.

If I try extracting it to e.g. /root instead of /home/user, I get these
numbers instead:

368 ./opt/maemo/usr/bin
368 ./opt/maemo/usr
368 ./opt/maemo
368 ./opt
0 ./usr/bin
0 ./usr/sbin
4 ./usr/share/doc/<appname>
4 ./usr/share/doc
4 ./usr/share/applications/hildon
4 ./usr/share/applications
4 ./usr/share/icons/hicolor/26x26/apps
4 ./usr/share/icons/hicolor/26x26
4 ./usr/share/icons/hicolor/40x40/apps
4 ./usr/share/icons/hicolor/40x40
4 ./usr/share/icons/hicolor/64x64/apps
4 ./usr/share/icons/hicolor/64x64
12 ./usr/share/icons/hicolor
12 ./usr/share/icons
20 ./usr/share
20 ./usr
388 .

On this file system, directories themselves don't seem to add up to the
total, and my application would pass the test.

So, what's the proper way of dealing with all this nonsense? Should I stop
shipping properly sized icons with my app? Or just stuff in a 1 MB "data"
file (read from /dev/urandom) under /opt to make the percentages right?

Also, is there any idea in trying to discuss these issues in Ovi Publish
by commenting back? On Ovi/Symbian things, I've never ever seen a reviewer
actually take my comments in account, so I don't think anybody ever reads
the comments the submitter writes.

// Martin
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

sascha.makela at gmail

Jul 21, 2010, 5:59 AM

Post #2 of 9 (426 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

Hi
Martin,

I've had my fair struggle with the QA (and still am), but I found the best
way is to write an email to Ovi Publisher Support and refer to the app and
file revision in question. I would basically copy and paste what you wrote
here to them.

Cheers,

Sascha

On Wed, Jul 21, 2010 at 15:31, Martin Storsjö <martin [at] martin> wrote:

> Hi,
>
> I just got my initial Maemo Ovi Store application rejected in QA, and I'd
> like discuss the measurements used in the disk space usage requirement.
>
> The part of the report is this:
>
> ISSUE #1 SUMMARY:
> OPT folder is not 80% more than that of USR folder.
>
> PRECONDITION:
> N/A.
>
> STEPS:
> 1. Open Xterminal, cd MyDocs.
> 2. cp <packagename> /home/user/.
> 3. cd /home/user/.
> 4. mkdir <packagename>-install.
> 5. dpkg -x ./<packagename> <packagename>-install.
> 6. cd <packagename>-install.
> 7. ls.
>
> EXPECTED OUTCOME:
> OPT folder value should be 80% of USR folder.
>
> ACTUAL OUTCOME:
> Opt folder value is not 80% of usr folder ,usr=84,opt=388k.
> FREQUENCY OF OCCURRENCE:.
> Always.
>
>
> If I do those steps (but do 'du' instead of 'ls' in step 7), I get the
> following:
>
> 376 ./opt/maemo/usr/bin
> 380 ./opt/maemo/usr
> 384 ./opt/maemo
> 388 ./opt
> 4 ./usr/sbin
> 4 ./usr/bin
> 8 ./usr/share/icons/hicolor/64x64/apps
> 12 ./usr/share/icons/hicolor/64x64
> 8 ./usr/share/icons/hicolor/26x26/apps
> 12 ./usr/share/icons/hicolor/26x26
> 8 ./usr/share/icons/hicolor/40x40/apps
> 12 ./usr/share/icons/hicolor/40x40
> 40 ./usr/share/icons/hicolor
> 44 ./usr/share/icons
> 8 ./usr/share/doc/<appname>
> 12 ./usr/share/doc
> 8 ./usr/share/applications/hildon
> 12 ./usr/share/applications
> 72 ./usr/share
> 84 ./usr
> 476 .
>
> According to this, my app is wasting 84 KB of space on /usr. Let's see
> what it _really_ does. In practice, it installs three icons in
> /usr/share/icons, 751, 1075 and 1570 bytes in size, one 24 byte copyright
> file in /usr/share/doc/<appname>, and a 181 .desktop file in
> /usr/share/applications.
>
> Since this is done on /home which is ext3, every directory consumes 4 KB,
> too. In practice, if I'd install the package, all those directories
> (except the /usr/share/doc/<appname> dir) would exist already, so no disk
> space would be consumed for them.
>
> Optifying them would only help a little - symlinks seem to count as 0
> bytes in this listing, but the number of directories would still add up to
> way too much.
>
> If I try extracting it to e.g. /root instead of /home/user, I get these
> numbers instead:
>
> 368 ./opt/maemo/usr/bin
> 368 ./opt/maemo/usr
> 368 ./opt/maemo
> 368 ./opt
> 0 ./usr/bin
> 0 ./usr/sbin
> 4 ./usr/share/doc/<appname>
> 4 ./usr/share/doc
> 4 ./usr/share/applications/hildon
> 4 ./usr/share/applications
> 4 ./usr/share/icons/hicolor/26x26/apps
> 4 ./usr/share/icons/hicolor/26x26
> 4 ./usr/share/icons/hicolor/40x40/apps
> 4 ./usr/share/icons/hicolor/40x40
> 4 ./usr/share/icons/hicolor/64x64/apps
> 4 ./usr/share/icons/hicolor/64x64
> 12 ./usr/share/icons/hicolor
> 12 ./usr/share/icons
> 20 ./usr/share
> 20 ./usr
> 388 .
>
> On this file system, directories themselves don't seem to add up to the
> total, and my application would pass the test.
>
> So, what's the proper way of dealing with all this nonsense? Should I stop
> shipping properly sized icons with my app? Or just stuff in a 1 MB "data"
> file (read from /dev/urandom) under /opt to make the percentages right?
>
> Also, is there any idea in trying to discuss these issues in Ovi Publish
> by commenting back? On Ovi/Symbian things, I've never ever seen a reviewer
> actually take my comments in account, so I don't think anybody ever reads
> the comments the submitter writes.
>
> // Martin
> _______________________________________________
> maemo-developers mailing list
> maemo-developers [at] maemo
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>

martin at martin

Jul 21, 2010, 6:07 AM

Post #3 of 9 (425 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

Hi Sascha,

On Wed, 21 Jul 2010, Sascha Mäkelä wrote:

> I've had my fair struggle with the QA (and still am), but I found the best
> way is to write an email to Ovi Publisher Support and refer to the app and
> file revision in question. I would basically copy and paste what you wrote
> here to them.

Thanks, I guess I'll do that when resubmitting a version where I've fixed
the slightly more valid issues.

// Martin

marius.vollmer at nokia

Jul 21, 2010, 6:50 AM

Post #4 of 9 (425 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

ext Martin Storsjö <martin [at] martin> writes:

> OPT folder value should be 80% of USR folder.

Taken literally, this means that the OPT folder is actually supposed to
be smaller than the USR folder... :-)

> ACTUAL OUTCOME:
> Opt folder value is not 80% of usr folder ,usr=84,opt=388k.

This actually checks out for me:

388 / (388 + 84) = 0.82 > 0.8

Maybe they want USR to be at most 20% of OPT? That would be a bit strange, but
would explain the verdict:

84 / 388 = 0.21 > 0.20

If I remember right, Urho and me invented these numbers hand wavingly
some time ago. The idea back then was that if the OPT/USR ratio of a
package is always larger than the OPT/USR ration of the actual
partitions, then you can install as many packages as will fit into OPT
without running out of USR.

The partitions are 2G / 256M = 8.

With the "du" numbers on ext3, your package has a ratio of

388 / 84 = 4.6 < 8

which would indeed fail the requirements.

With your numbers on ubifs, we have

368 / 20 = 18.4 > 8

and you win by a huge margin.

However, I agree that directories should not be counted at all. Then
you will probably win by a even larger margin.

To summarize: I don't know what they have calculated, but I think they
should calculate the OPT/USR ratio and require it to be larger than 8.
Furthermore, only actual files should be counted. Filesystem specific
effects like directories should be exluded. This is a heuristic anyway
and making it simpler and easier to reproduce in different environments
is therefore more important than accurate simulation of what would
happen on the device.

(If we want to simulate the device, we need to take UBIFS compression
into account.)

> Also, is there any idea in trying to discuss these issues in Ovi Publish
> by commenting back? On Ovi/Symbian things, I've never ever seen a reviewer
> actually take my comments in account, so I don't think anybody ever reads
> the comments the submitter writes.

I'll try to ping the right people here.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

daniil.ivanov at gmail

Jul 21, 2010, 7:19 AM

Post #5 of 9 (422 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

Hi Martin!

I will tell you a dirty secret:

Nokia-N900:~# ls -l /usr/share/ | grep icons
lrwxrwxrwx 1 root root 20 Jul 13 14:27 icons ->
/opt/usr/share/icons

Thanks, Daniil.

On Wed, Jul 21, 2010 at 4:50 PM, Marius Vollmer
<marius.vollmer [at] nokia> wrote:
> ext Martin Storsj� <martin [at] martin> writes:
>
>> OPT folder value should be 80% of USR folder.
>
> Taken literally, this means that the OPT folder is actually supposed to
> be smaller than the USR folder... :-)
>
>> ACTUAL OUTCOME:
>> Opt folder value is not 80% of usr folder ,usr=84,opt=388k.
>
> This actually checks out for me:
>
> � �388 / (388 + 84) = 0.82 > 0.8
>
> Maybe they want USR to be at most 20% of OPT? �That would be a bit strange, but
> would explain the verdict:
>
> � �84 / 388 = 0.21 > 0.20
>
> If I remember right, Urho and me invented these numbers hand wavingly
> some time ago. �The idea back then was that if the OPT/USR ratio of a
> package is always larger than the OPT/USR ration of the actual
> partitions, then you can install as many packages as will fit into OPT
> without running out of USR.
>
> The partitions are 2G / 256M = 8.
>
> With the "du" numbers on ext3, your package has a ratio of
>
> � � 388 / 84 = 4.6 < 8
>
> which would indeed fail the requirements.
>
>
> With your numbers on ubifs, we have
>
> � �368 / 20 = 18.4 > 8
>
> and you win by a huge margin.
>
> However, I agree that directories should not be counted at all. �Then
> you will probably win by a even larger margin.
>
>
> To summarize: I don't know what they have calculated, but I think they
> should calculate the OPT/USR ratio and require it to be larger than 8.
> Furthermore, only actual files should be counted. �Filesystem specific
> effects like directories should be exluded. �This is a heuristic anyway
> and making it simpler and easier to reproduce in different environments
> is therefore more important than accurate simulation of what would
> happen on the device.
>
> (If we want to simulate the device, we need to take UBIFS compression
> into account.)
>
>> Also, is there any idea in trying to discuss these issues in Ovi Publish
>> by commenting back? On Ovi/Symbian things, I've never ever seen a reviewer
>> actually take my comments in account, so I don't think anybody ever reads
>> the comments the submitter writes.
>
> I'll try to ping the right people here.
> _______________________________________________
> maemo-developers mailing list
> maemo-developers [at] maemo
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

marius.vollmer at nokia

Jul 21, 2010, 7:21 AM

Post #6 of 9 (419 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

"Vollmer Marius (Nokia-MS/Helsinki)" <marius.vollmer [at] nokia> writes:

> The partitions are 2G / 256M = 8.

Of course, there is less than 256M free for apps in the rootfs, so
scratch the 8. It's probably more like 2G / 100M = 20ish. A bit
tougher, but you are almost there... :)
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

martin at martin

Jul 21, 2010, 8:38 AM

Post #7 of 9 (424 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

Hi Daniil,

On Wed, 21 Jul 2010, Daniil Ivanov wrote:

> I will tell you a dirty secret:
>
> Nokia-N900:~# ls -l /usr/share/ | grep icons
> lrwxrwxrwx 1 root root 20 Jul 13 14:27 icons ->
> /opt/usr/share/icons

Oh, good point. Even less reason for the current behaviour.

// Martin
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

martin at martin

Jul 21, 2010, 9:07 AM

Post #8 of 9 (423 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

Hi Marius,

Thanks for the explanations!

On Wed, 21 Jul 2010, Marius Vollmer wrote:

> ext Martin Storsjö <martin [at] martin> writes:
>
> > OPT folder value should be 80% of USR folder.
>
> Taken literally, this means that the OPT folder is actually supposed to
> be smaller than the USR folder... :-)

Yes, this one felt dubious to say the least, but I was able to guess what
the intent of the test was.

> If I remember right, Urho and me invented these numbers hand wavingly
> some time ago. The idea back then was that if the OPT/USR ratio of a
> package is always larger than the OPT/USR ration of the actual
> partitions, then you can install as many packages as will fit into OPT
> without running out of USR.

While I agree with the general idea, this simple heuristic doesn't work
well for small applications. Say a small application with a binary of 50
KB in /opt - this one wouldn't be allowed to put the mandatory icons and
.desktop launcher items in /usr as any other aplication, even if it is
much smaller in total than many other apps.

So given that, I'd say a proper limit would be e.g. 10-20% of what goes
into /opt, but at least 30 KB for all those mandatory things (icons,
.desktop files). Or perhaps only a fixed limit, max 30-50 KB in /usr for
all the mandatory system things that expect to find things there, and as
much as is needed in /opt.

> However, I agree that directories should not be counted at all. Then
> you will probably win by a even larger margin.
>
>
> To summarize: I don't know what they have calculated, but I think they
> should calculate the OPT/USR ratio and require it to be larger than 8.
> Furthermore, only actual files should be counted. Filesystem specific
> effects like directories should be exluded. This is a heuristic anyway
> and making it simpler and easier to reproduce in different environments
> is therefore more important than accurate simulation of what would
> happen on the device.
>
> (If we want to simulate the device, we need to take UBIFS compression
> into account.)

Also, as Daniil pointed out, /usr/share/icons actually resides in /opt, so
that should be taken into account, too.

And things under /usr/share/doc gets wiped regularly, so those files
perhaps should be skipped from the calculation, too.

> > Also, is there any idea in trying to discuss these issues in Ovi Publish
> > by commenting back? On Ovi/Symbian things, I've never ever seen a reviewer
> > actually take my comments in account, so I don't think anybody ever reads
> > the comments the submitter writes.
>
> I'll try to ping the right people here.

Thanks, that's appreciated!

// Martin

martin at martin

Jul 30, 2010, 2:50 AM

Post #9 of 9 (345 views)
Permalink

Re: Ovi Store disk space requirements miscalculations [In reply to]

On Wed, 21 Jul 2010, Sascha Mäkelä wrote:

> I've had my fair struggle with the QA (and still am), but I found the best
> way is to write an email to Ovi Publisher Support and refer to the app and
> file revision in question. I would basically copy and paste what you wrote
> here to them.

Btw, on which email address did you contact them, and where did you find
it?

I sent them a message via the Publisher Portal, Support, Contact Us form a
week ago, asking for clarification on one of the issues that they pointed
out, that was so poorly written that I really have to guess what they're
trying to say, but haven't yet received any answer.

// Martin

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads