Mailing List Archive: Maemo: Developers

Installing packages in extra storage?

Index | Next | Previous | View Threaded

barbieri at gmail

Dec 11, 2005, 9:43 AM

Post #1 of 11 (3632 views)
Permalink

Installing packages in extra storage?

Hello,

Is it possible to install software on extra storage, like MMC or some
other kind of share (smb, nfs, ...)?

Nokia 770 with just 64Mb may raise some problems and we must have
platform support for this feature. If this feature is not available I
will write a enhancement proposal and try to implement it.

I think of something like a special directory that, if available,
would be included as overlay to currently installed packages. Add and
Remove scripts would be required to initialize/finish support when the
extra storage with extra packages is detected. And I see 4 ways to
make these aplications available:
- export PATH and LD_LIBRARY_PATH: not really useful since entire
environment must be restarted to use these variables
- symbolic links: not a clean solution, but works... I dislike it too.
- remount directory as overlay to system: may have problems when
installing new software on device, it would go in extra storage
instead
- maemo support: application launcher/menu would detect new software
and launch it directly from extra storage. Problems may arise if
package was dependent on absolute paths. Clear solution is to provide
scripts to setup environment and fix package to use this new
environment instead of hardcoded paths. The best solution IMHO.

any ideas?

--
Gustavo Sverzut Barbieri
--------------------------------------
Computer Engineer 2001 - UNICAMP
Mobile: +55 (19) 9165 8010
Phone: +1 (347) 624 6296; 08122692 [at] sip
Jabber: gsbarbieri [at] jabber
ICQ#: 17249123
MSN: barbieri [at] gmail
GPG: 0xB640E1A2 @ wwwkeys.pgp.net
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

dieter.govaerts at pandora

Dec 12, 2005, 2:33 PM

Post #2 of 11 (3498 views)
Permalink

Re: Installing packages in extra storage? [In reply to]

Hi Gustavo,

Gustavo Sverzut Barbieri schreef:
> Hello,
>
> Is it possible to install software on extra storage, like MMC or some
> other kind of share (smb, nfs, ...)?
>
> Nokia 770 with just 64Mb may raise some problems and we must have
> platform support for this feature. If this feature is not available I
> will write a enhancement proposal and try to implement it.
>

I'm currently working on my master thesis which is a research project to
examine the possibilities of a solution to this very same problem. The
idea is to use some sort of streaming technique for applications on a
remote server and do some caching of the remote applications on the
local device.

A quick overview of the system:
1) Upon first activation of an application by the user, a small
installation package is transferred to the device form the remote
server, containing installation scripts and a list of files used by the
application. The package contains no files from the application itself,
only setup-files.
2) The application is installed on the local device and the
application’s files are added to the caching file system but the files
remain empty for now.
3) All the application’s files are stored on the remote server and are
split-up and made available as individual memory pages.
4) When the application starts running, the operating system begins to
receive page faults. For each page fault, the caching file system is
asked to provide the application code to be loaded into memory. The
caching file system checks if it has a local copy available and if not,
requests it from the remote server.

Pros to this approach:
* Only the used parts of an application are downloaded, a user almost
never uses the complete functionality of an application. This saves
initial bandwidth usage and local storage.
* Since only the used parts of an application are cached locally, more
space remains to cache the used parts of other applications.
* Since the installation of an application only involves adding empty
files to the caching file system, and starting that the application for
the first time involves downloading a few application code pages, a user
is able to use his new application much faster.

Cons to this approach:
* That’s what my research project is all about :-). But some may include:
* High bandwidth usage when too many applications are cached due to
eviction of used pages from the cache.
* Reduced user experience due to increased latency when using uncached
features of an application.

I’m currently still in the research phase for this system. Design and
implementation of a prototype system starts in February. Any ideas,
suggestions or hints regarding architecture, design or implementation
(mainly about the local caching part and ‘fooling’ the operating system
with those cached files), or information about existing projects that
can be used in my project, are greatly appreciated.

Best regards,

Dieter Govaerts

--
Student Computer Engineering
at the University of Ghent (Belgium)
dieter.govaerts(at)ugent.be

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

stefano.delliponti at gmail

Dec 14, 2005, 8:49 AM

Post #3 of 11 (3475 views)
Permalink

Re: Installing packages in extra storage? [In reply to]

Gustavo Sverzut Barbieri wrote:
> Hello,
>
> Is it possible to install software on extra storage, like MMC or some
> other kind of share (smb, nfs, ...)?
>
> Nokia 770 with just 64Mb may raise some problems and we must have
> platform support for this feature. If this feature is not available I
> will write a enhancement proposal and try to implement it.
>

Hi Gustavo,

I do think that support for this is needed.
Together with some control panel option to set a swap file on MMC (plus
some machinery to swapon/swappoff on insertion/extraction of MMC).

I have done this the dirty way (deleted the noexec in /etc/fstab and
swap(ed)on /media/mmc1/swapfile) but we need a comfortable way to set
these configurations and an installer that gives you the possibility to
install on MMC.

Sted

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

barbieri at gmail

Dec 14, 2005, 11:21 AM

Post #4 of 11 (3467 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

On 12/14/05, Stefano Delli Ponti <stefano.delliponti [at] gmail> wrote:
>
> Hi Gustavo,
>
> I do think that support for this is needed.
> Together with some control panel option to set a swap file on MMC (plus
> some machinery to swapon/swappoff on insertion/extraction of MMC).
>
> I have done this the dirty way (deleted the noexec in /etc/fstab and
> swap(ed)on /media/mmc1/swapfile) but we need a comfortable way to set
> these configurations and an installer that gives you the possibility to
> install on MMC.

I don't get it, how swap is related to installing software on extra storage?

Also, I want to install software not only on MMC, but on network
shares and have them visible automatically. Imagine that I want my
"program-that-I-just-use-at-home" to be on a NFS/SMB share at home, so
when I connect it will be available. When I'm at work, it'll just
vanish and no space is wasted.

--
Gustavo Sverzut Barbieri
--------------------------------------
Computer Engineer 2001 - UNICAMP
Mobile: +55 (19) 9165 8010
Phone: +1 (347) 624 6296; 08122692 [at] sip
Jabber: gsbarbieri [at] jabber
ICQ#: 17249123
MSN: barbieri [at] gmail
GPG: 0xB640E1A2 @ wwwkeys.pgp.net
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

bradb667 at gmail

Dec 14, 2005, 12:25 PM

Post #5 of 11 (3484 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

Hey Gustavo-

On 12/14/05, Gustavo Sverzut Barbieri <barbieri [at] gmail> wrote:
>
> I don't get it, how swap is related to installing software on extra
> storage?

I think the connection is simply to help manage resources - by moving
installed programs to the MMC, as well as enabling swap, will give the
machine more memory to work with.

What I did to solve this problem was partition the MMC into both swap (24MB)
and an ext2fs file system (40MB).
Then I used tar to backup the contents of /var/lib/install, then copied
those files to the MMC. I did an "rm -rf /var/lib/install", then during
boot set the system up to mount my ext2fs file system over /var/lib/install.

By doing this all programs installed - by default - will now go to the MMC.
I think this shoud be the default configuration, actually (well, maybe
replace the swap partition with a VFAT partition - as soon as I buy a 1GB
card I'm not going to risk damaging it). I also replaced ~/MyDocs with a
symbolic link to /var/lib/install/MyDocs to free up even more space.

The only problem: both 'free', '/proc/meminfo' and the little status bar
plug-in all report far more memory in use than I'd expect to see. I _know_
I deleted the contents of /var/lib/install (as well as ~/MyDocs), and when I
install new programs I see my memory used go up.

I assume this is simply a result of how memory is managed, or have I done
something stupid?

Also, I want to install software not only on MMC, but on network
> shares and have them visible automatically. Imagine that I want my
> "program-that-I-just-use-at-home" to be on a NFS/SMB share at home, so
> when I connect it will be available. When I'm at work, it'll just
> vanish and no space is wasted.

That's a great idea. Though in my (limited) experience, NFS and Samba don't
handle being disconnected and reconnected at will.

Brad.

mpl22 at cornell

Dec 14, 2005, 12:52 PM

Post #6 of 11 (3477 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

> The only problem: both 'free', '/proc/meminfo' and the little
> status bar plug-in all report far more memory in use than I'd
> expect to see. I _know_ I deleted the contents of
> /var/lib/install (as well as ~/MyDocs), and when I install new
> programs I see my memory used go up.

free and meminfo measure RAM usage. Try df or the memory control panel to
measure flash usage.

Is there a compelling reason to reformat the card? I haven't yet, but am
planning to try mounting the MMC without NOEXEC and just symlinking
/var/lib/install to a directory on the VFAT partition. Am I doomed to
failure?

Mike

PS It might be polite to move this conversation to the users list at some
point.

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

bradb667 at gmail

Dec 14, 2005, 1:04 PM

Post #7 of 11 (3471 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

On 12/14/05, Michael P. Lococo <mpl22 [at] cornell> wrote:
>
> > The only problem: both 'free', '/proc/meminfo' and the little
> > status bar plug-in all report far more memory in use than I'd
> > expect to see. I _know_ I deleted the contents of
> > /var/lib/install (as well as ~/MyDocs), and when I install new
> > programs I see my memory used go up.
>
> free and meminfo measure RAM usage. Try df or the memory control panel to
> measure flash usage.

Of course. But the memory control panel is the easiest to use, and it
reports the same thing. Installing programs - like bash, for example - to
/var/lib/install causes memory usage to increase. I can see some use
occuring, but I guess I need to run more tests.

Is there a compelling reason to reformat the card? I haven't yet, but am
> planning to try mounting the MMC without NOEXEC and just symlinking
> /var/lib/install to a directory on the VFAT partition. Am I doomed to
> failure?

Well, it's my understanding that VFAT is a problem - it doesn't support long
file names and I think there are other issues (I hope someone else who knows
this better chimes in).

Mike
>
> PS It might be polite to move this conversation to the users list at some
> point.

Done. I'm not sure on proper netiquette though, so be gentle ;-)

Brad.

barbieri at gmail

Dec 14, 2005, 2:22 PM

Post #8 of 11 (3490 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

On 12/14/05, Brad Burleson <bradb667 [at] gmail> wrote:
> That's a great idea. Though in my (limited) experience, NFS and Samba don't
> handle being disconnected and reconnected at will.

You can manually mount/umount, as you do in your desktop.

My idea is to help maemo platform in the package management area. This
is one idea: enable users to use MMC, SMB, NFS and others as media to
store applications. The other is to enable users to beam their
applications to other users. Both are kind of connected.

I'll start a proposal of enhancement in wiki next week.

--
Gustavo Sverzut Barbieri
--------------------------------------
Computer Engineer 2001 - UNICAMP
Mobile: +55 (19) 9165 8010
Phone: +1 (347) 624 6296; 08122692 [at] sip
Jabber: gsbarbieri [at] jabber
ICQ#: 17249123
MSN: barbieri [at] gmail
GPG: 0xB640E1A2 @ wwwkeys.pgp.net
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

bradb667 at gmail

Dec 14, 2005, 2:29 PM

Post #9 of 11 (3470 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

On 12/14/05, Gustavo Sverzut Barbieri <barbieri [at] gmail> wrote:
>
> On 12/14/05, Brad Burleson <bradb667 [at] gmail> wrote:
> > That's a great idea. Though in my (limited) experience, NFS and Samba
> don't
> > handle being disconnected and reconnected at will.
>
> You can manually mount/umount, as you do in your desktop.
>
> My idea is to help maemo platform in the package management area. This
> is one idea: enable users to use MMC, SMB, NFS and others as media to
> store applications. The other is to enable users to beam their
> applications to other users. Both are kind of connected.
>
> I'll start a proposal of enhancement in wiki next week.

This is an excellent idea, but the "manually mount/umount" is where I see
the real problem. It's not really practical to expect the user to remember
to manually handle the mount/umount process. At least for me, I guess.

Plus, what happens when the wifi connect drops and comes back? I've seen
all sorts of problems on *nix where recovering from temporary network
outages doesn't work totally correctly (and you end up with hung/zombied
processes, etc.).

But I'm looking forward to seeing how this goes.

Brad.

stefano.delliponti at gmail

Dec 15, 2005, 5:42 AM

Post #10 of 11 (3457 views)
Permalink

Re: Installing packages in extra storage? [In reply to]

[re-posted]
Gustavo Sverzut Barbieri wrote:
> On 12/14/05, Stefano Delli Ponti <stefano.delliponti [at] gmail> wrote:
>> Hi Gustavo,
>>
>> I do think that support for this is needed.
>> Together with some control panel option to set a swap file on MMC (plus
>> some machinery to swapon/swappoff on insertion/extraction of MMC).
>>
>> I have done this the dirty way (deleted the noexec in /etc/fstab and
>> swap(ed)on /media/mmc1/swapfile) but we need a comfortable way to set
>> these configurations and an installer that gives you the possibility to
>> install on MMC.
>
> I don't get it, how swap is related to installing software on extra storage?
>

Sorry, I have not been clear enough.
My concerns are mostly related to lack of system resources: flash memory
for the filesystem and ram.
I was pointing to the two problems together and to some hypothesis for
the solutions.
I know you meant to talk in a specific way (and in a broader sense)
about software installation on different storage media.

Anyway, It is possible to install software on the MMC.
A clean and easy way to do this is IMHO necessary.

Regards,
Sted

P.S.
Try the swapfile, it makes this tiny wonderful device a lot more powerful.

> Also, I want to install software not only on MMC, but on network
> shares and have them visible automatically. Imagine that I want my
> "program-that-I-just-use-at-home" to be on a NFS/SMB share at home, so
> when I connect it will be available. When I'm at work, it'll just
> vanish and no space is wasted.
>
> --
> Gustavo Sverzut Barbieri
> --------------------------------------
> Computer Engineer 2001 - UNICAMP
> Mobile: +55 (19) 9165 8010
> Phone: +1 (347) 624 6296; 08122692 [at] sip
> Jabber: gsbarbieri [at] jabber
> ICQ#: 17249123
> MSN: barbieri [at] gmail
> GPG: 0xB640E1A2 @ wwwkeys.pgp.net

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

kuisma.salonen at maemo

Dec 20, 2005, 12:24 AM

Post #11 of 11 (3466 views)
Permalink

Re: Re: Installing packages in extra storage? [In reply to]

> PS It might be polite to move this conversation to the users list at some
> point.
>

Just giving a hint (maybe too obvious) to application developers first
;) If your application takes more than 10 megs you could do with default
software image (the one averate end users use) this:

Application installer installs just few files which include wrapper
executable and main program is extracted on a MMC (of course, you could
install it also on some other type of mount). Then the wrapper would
just check if media is in and launch the application itself, if it's not
it it would bitch something like "Correct media not inserted" to user
(does this remind you of some proprietary CD games which say something
like "Please insert CD this and that"?).

I guess this was all Captain Obvious had to say this time...

-- Kuisma
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://maemo.org/mailman/listinfo/maemo-developers

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads