Mailing List Archive: Zope: Users

DirectoryStorage and UFS2

Index | Next | Previous | View Threaded

kgunders at teamcool

May 19, 2004, 9:07 AM

Post #1 of 3 (351 views)
Permalink

DirectoryStorage and UFS2

Howdy:

I'd have a Zope/Zeo site that I'd like to set up to use
DirectoryStorage. Platform is FreeBSD5.2.1 using UFS2 w/softupdates.
Probably will also mirror with vinum. DirectoryStorage docs state:

bushy:

If you are using a filesystem that is inefficient with directories
containing larger than a few hundred items. This is optimal for most
conventional filesystems, such as ext2 on linux. In this format each
object is given its own directory nested inside 8 levels of
subdirectory. Each revision of an object has its own file in that
directory.

chunky:

If you are using a filesystem that is most efficient with directories
containing tens of thousands of subdirectories, or hundreds of
thousands of files. This is optimal for filesystems such as reiser3 or
JFS on linux. In this format, one subdirectory is shared by up to 65000
objects. That subdirectory is nested inside 4 levels of subdirectory.

I am unsure whether UFS2 would best utilize the "chunky" or "busy"
options. The 64 bit pointers allow for up to 65K subdirs w/in a dir,
correct? So the large number of subdirs under chunky format could be
handled. Dirhash and dirpref can deal can cope with large numbers of
files per directory, correct. But then doesn't UFS2 still use more
linear model?

Any insights from the FBSD/filesystem gurus would be appreciated.

--
Best regards,

Ken Gunderson
GPG Key-- 9F5179FD

"Freedom begins between the ears." -- Edward Abbey

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

matth at netsight

May 19, 2004, 10:18 AM

Post #2 of 3 (340 views)
Permalink

Re: DirectoryStorage and UFS2 [In reply to]

Ken Gunderson wrote:

> I am unsure whether UFS2 would best utilize the "chunky" or "busy"
> options. The 64 bit pointers allow for up to 65K subdirs w/in a dir,
> correct? So the large number of subdirs under chunky format could be
> handled. Dirhash and dirpref can deal can cope with large numbers of
> files per directory, correct. But then doesn't UFS2 still use more
> linear model?
>
> Any insights from the FBSD/filesystem gurus would be appreciated.

We use DirectoryStorage on UFS+softupdates on FreeBSD 4.x and use bushy.
I did a few simple tests of bushy vs. chunky and didn't get any
significant difference. Whether this holds true for UFS2 or not I don't
know.

-Matt

--
Matt Hamilton matth [at] netsight
Netsight Internet Solutions, Ltd. Business Vision on the Internet
http://www.netsight.co.uk +44 (0)117 9090901
Web Design | Zope/Plone Development & Consulting | Co-location | Hosting

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

kgunders at teamcool

May 19, 2004, 10:32 AM

Post #3 of 3 (345 views)
Permalink

Re: DirectoryStorage and UFS2 [In reply to]

On Wednesday 19 May 2004 11:18 am, Matt Hamilton wrote:
> Ken Gunderson wrote:
> > I am unsure whether UFS2 would best utilize the "chunky" or "busy"
> > options. The 64 bit pointers allow for up to 65K subdirs w/in a
> > dir, correct? So the large number of subdirs under chunky format
> > could be handled. Dirhash and dirpref can deal can cope with large
> > numbers of files per directory, correct. But then doesn't UFS2
> > still use more linear model?
> >
> > Any insights from the FBSD/filesystem gurus would be appreciated.
>
> We use DirectoryStorage on UFS+softupdates on FreeBSD 4.x and use
> bushy. I did a few simple tests of bushy vs. chunky and didn't get
> any significant difference. Whether this holds true for UFS2 or not
> I don't know.

FWIW-- I did a few informal tests as well. Chunky "feels" a bit faster,
but I didn't test with huge dataset, so I'm not sure about scalability.
Chunky also seems to require a bit less disk space, e.g. virgin
dirstorage is 26MB vs 21MB. Guess I'll play it safe and stick with
bushy.

Thanks for the input.

--
Best regards,

Ken Gunderson
GPG Key-- 9F5179FD

"Freedom begins between the ears." -- Edward Abbey

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com