Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux: Kernel

document ext3 requirements

 

 

First page Previous page 1 2 3 4 Next page Last page  View All Linux kernel RSS feed   Index | Next | Previous | View Threaded


pavel at suse

Jan 3, 2009, 4:38 AM

Post #1 of 88 (4222 views)
Permalink
document ext3 requirements

Using ext3 is only safe if storage subsystem meets certain
criteria. Document those.

Errors=remount-ro is documented as default, but superblock setting
overrides that and mkfs defaults to errors=continue... so the default
is errors=continue in practice.

readonly mount does actually write to the media in some cases. Document that.

Signed-off-by: Pavel Machek <pavel [at] suse>

diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
index 9dd2a3b..74a73b0 100644
--- a/Documentation/filesystems/ext3.txt
+++ b/Documentation/filesystems/ext3.txt
@@ -14,6 +14,9 @@ Options
When mounting an ext3 filesystem, the following option are accepted:
(*) == default

+ro Note that ext3 will replay the journal (and thus write
+ to the partition) even when mounted "read only".
+
journal=update Update the ext3 file system's journal to the current
format.

@@ -95,6 +98,8 @@ debug Extra debugging information is sent to syslog.
errors=remount-ro(*) Remount the filesystem read-only on an error.
errors=continue Keep going on a filesystem error.
errors=panic Panic and halt the machine if an error occurs.
+ (Note that default is overriden by superblock
+ setting on most systems).

data_err=ignore(*) Just print an error message if an error occurs
in a file data buffer in ordered mode.
@@ -188,6 +193,34 @@ mke2fs: create a ext3 partition with the -j flag.
debugfs: ext2 and ext3 file system debugger.
ext2online: online (mounted) ext2 and ext3 filesystem resizer

+Requirements
+============
+
+Ext3 expects disk/storage subsystem to behave sanely. On sanely
+behaving disk subsystem, data that have been successfully synced will
+stay on the disk. Sane means:
+
+* writes to media never fail. Even if disk returns error condition during
+ write, ext3 can't handle that correctly, because success on fsync was already
+ returned when data hit the journal.
+
+ (Fortunately writes failing are very uncommon on disks, as they
+ have spare sectors they use when write fails.)
+
+* either whole sector is correctly written or nothing is written during
+ powerfail.
+
+ (Unfortuantely, none of the cheap USB/SD flash cards I seen do behave
+ like this, and are unsuitable for ext3. Because RAM tends to fail
+ faster than rest of system during powerfail, special hw killing
+ DMA transfers may be neccessary. Not sure how common that problem
+ is on generic PC machines).
+
+* either write caching is disabled, or hw can do barriers and they are enabled.
+
+ (Note that barriers are disabled by default, use "barrier=1"
+ mount option after making sure hw can support them).
+

References
==========

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mmokrejs at ribosome

Jan 3, 2009, 1:17 PM

Post #2 of 88 (4150 views)
Permalink
Re: document ext3 requirements [In reply to]

Can one avoid replay of the journal then if it would be unclean?
Just curious.
M.

Pavel Machek wrote:
> Using ext3 is only safe if storage subsystem meets certain
> criteria. Document those.
>
> Errors=remount-ro is documented as default, but superblock setting
> overrides that and mkfs defaults to errors=continue... so the default
> is errors=continue in practice.
>
> readonly mount does actually write to the media in some cases. Document that.
>
> Signed-off-by: Pavel Machek <pavel [at] suse>
>
> diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


pavel at suse

Jan 3, 2009, 2:06 PM

Post #3 of 88 (4163 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sat 2009-01-03 22:17:11, Martin MOKREJÅ  wrote:
> Can one avoid replay of the journal then if it would be unclean?
> Just curious.

Well, mounting unclean filesystem is dangerous but depending on
circumstances, it may be better than writing to the filesystems.

(You may not be able to read some data and may provoke kernel bugs,
but at least you don't damage what is on disk. If you are collecting
evidence -- not writing is very important. If you suspect something is
very wrong with the drive, not writing is good idea).

Pavel
>
> Pavel Machek wrote:
> > Using ext3 is only safe if storage subsystem meets certain
> > criteria. Document those.
> >
> > Errors=remount-ro is documented as default, but superblock setting
> > overrides that and mkfs defaults to errors=continue... so the default
> > is errors=continue in practice.
> >
> > readonly mount does actually write to the media in some cases. Document that.
> >
> > Signed-off-by: Pavel Machek <pavel [at] suse>
> >
> > diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


duaneg at dghda

Jan 3, 2009, 2:17 PM

Post #4 of 88 (4147 views)
Permalink
Re: document ext3 requirements [In reply to]

[Fixed top-posting]

2009/1/3 Martin MOKREJŠ <mmokrejs [at] ribosome>:
> Pavel Machek wrote:
>> readonly mount does actually write to the media in some cases. Document that.
>>
> Can one avoid replay of the journal then if it would be unclean?
> Just curious.

Nope. If the underlying block device is read-only then mounting the
filesystem will fail. I tried to fix this some time ago, and have a
set of patches that almost always work, but "almost always" isn't good
enough. Unfortunately I never managed to figure out a way to finish it
off without disgusting hacks or major surgery.

> M.

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


pavel at suse

Jan 3, 2009, 2:29 PM

Post #5 of 88 (4151 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sat 2009-01-03 22:17:15, Duane Griffin wrote:
> [Fixed top-posting]
>
> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
> > Pavel Machek wrote:
> >> readonly mount does actually write to the media in some cases. Document that.
> >>
> > Can one avoid replay of the journal then if it would be unclean?
> > Just curious.
>
> Nope. If the underlying block device is read-only then mounting the
> filesystem will fail. I tried to fix this some time ago, and have a
> set of patches that almost always work, but "almost always" isn't good
> enough. Unfortunately I never managed to figure out a way to finish it
> off without disgusting hacks or major surgery.

Uhuh, can you just ignore the journal and mount it anyway?
...basically treating it like an ext2?

...ok, that will present "old" version of the filesystem to the
user... violating fsync() semantics.

Still handy for recovering badly broken filesystems, I'd say.

Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mmokrejs at ribosome

Jan 3, 2009, 3:01 PM

Post #6 of 88 (4151 views)
Permalink
Re: document ext3 requirements [In reply to]

Pavel Machek wrote:
> On Sat 2009-01-03 22:17:15, Duane Griffin wrote:
>> [Fixed top-posting]
>>
>> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>>> Pavel Machek wrote:
>>>> readonly mount does actually write to the media in some cases. Document that.
>>>>
>>> Can one avoid replay of the journal then if it would be unclean?
>>> Just curious.
>> Nope. If the underlying block device is read-only then mounting the
>> filesystem will fail. I tried to fix this some time ago, and have a
>> set of patches that almost always work, but "almost always" isn't good
>> enough. Unfortunately I never managed to figure out a way to finish it
>> off without disgusting hacks or major surgery.
>
> Uhuh, can you just ignore the journal and mount it anyway?
> ...basically treating it like an ext2?
>
> ...ok, that will present "old" version of the filesystem to the
> user... violating fsync() semantics.

Hmm, so if my dual-boot machine does not shutdown correctly and I boot
accidentally in M$ Win where I use ext2 IFS driver and modify some
stuff on the ext3 drive, after a while reboot to linux and the journal
get re-played ... Mmm ...

>
> Still handy for recovering badly broken filesystems, I'd say.

Me as well. How about improving you doc patch with some summary of
this thread (although it is probably not over yet)? ;-) Definitely,
a note that one can mount it as ext2 while read-only would be helpful
when doing some forensics on the disk.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


duaneg at dghda

Jan 3, 2009, 3:12 PM

Post #7 of 88 (4142 views)
Permalink
Re: document ext3 requirements [In reply to]

2009/1/3 Pavel Machek <pavel [at] suse>:
> On Sat 2009-01-03 22:17:15, Duane Griffin wrote:
>> [Fixed top-posting]
>>
>> 2009/1/3 Martin MOKREJŠ <mmokrejs [at] ribosome>:
>> > Pavel Machek wrote:
>> >> readonly mount does actually write to the media in some cases. Document that.
>> >>
>> > Can one avoid replay of the journal then if it would be unclean?
>> > Just curious.
>>
>> Nope. If the underlying block device is read-only then mounting the
>> filesystem will fail. I tried to fix this some time ago, and have a
>> set of patches that almost always work, but "almost always" isn't good
>> enough. Unfortunately I never managed to figure out a way to finish it
>> off without disgusting hacks or major surgery.
>
> Uhuh, can you just ignore the journal and mount it anyway?
> ...basically treating it like an ext2?

I'm afraid not, ext2 won't mount an FS with EXT3_FEATURE_INCOMPAT_RECOVER set.

> ...ok, that will present "old" version of the filesystem to the
> user... violating fsync() semantics.
>
> Still handy for recovering badly broken filesystems, I'd say.
>
> Pavel

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


duaneg at dghda

Jan 3, 2009, 3:38 PM

Post #8 of 88 (4148 views)
Permalink
Re: document ext3 requirements [In reply to]

2009/1/3 Martin MOKREJŠ <mmokrejs [at] ribosome>:
> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
> accidentally in M$ Win where I use ext2 IFS driver and modify some
> stuff on the ext3 drive, after a while reboot to linux and the journal
> get re-played ... Mmm ...

You *really* wouldn't want to be doing that.

The other scenario that people have reported trouble with is
suspending the system, booting a live CD which "read-only" mounts the
filesystem (and replays the journal), then resuming.

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mmokrejs at ribosome

Jan 3, 2009, 3:50 PM

Post #9 of 88 (4148 views)
Permalink
Re: document ext3 requirements [In reply to]

Duane Griffin wrote:
> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
>> accidentally in M$ Win where I use ext2 IFS driver and modify some
>> stuff on the ext3 drive, after a while reboot to linux and the journal
>> get re-played ... Mmm ...
>
> You *really* wouldn't want to be doing that.
>
> The other scenario that people have reported trouble with is
> suspending the system, booting a live CD which "read-only" mounts the
> filesystem (and replays the journal), then resuming.

Why does not "mount -ro" die when it would have to replay the journal
with a message that user must run fsck.ext3 in order to be able to mount
it albeit read-only? Still I would prefer having an extra switch to
force mount RO while not touching the journal for disk forensics.
I think that would also prevent the cases when a LiveCD/rescue distribution
would not mount+replay it automagically but user would really have to
provide the switch to the command. I am really not using the recovery
boot cd to touch my partitions in some cases unwillingly.

Sure that does not prevent my case when I let ext2 IFS writing onto
my ext3 partition. Actually, couldn't the driver at least warn me
the journal log is non-empty (am just a user, sorry, cannot check
myself the code at www.fs-driver.org if it could do at least this
although it does not understand ext3). ;-)

Martin

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


hancockr at shaw

Jan 3, 2009, 3:58 PM

Post #10 of 88 (4147 views)
Permalink
Re: document ext3 requirements [In reply to]

Martin MOKREJÅ  wrote:
> Duane Griffin wrote:
>> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>>> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
>>> accidentally in M$ Win where I use ext2 IFS driver and modify some
>>> stuff on the ext3 drive, after a while reboot to linux and the journal
>>> get re-played ... Mmm ...
>> You *really* wouldn't want to be doing that.
>>
>> The other scenario that people have reported trouble with is
>> suspending the system, booting a live CD which "read-only" mounts the
>> filesystem (and replays the journal), then resuming.
>
> Why does not "mount -ro" die when it would have to replay the journal
> with a message that user must run fsck.ext3 in order to be able to mount
> it albeit read-only? Still I would prefer having an extra switch to

That would break typical system bootup in the unclean journal case,
normally the root FS is mounted read-only to start with (which replays
the journal) and remounted read-write later on - and usually the fsck
utilities are located on the root filesystem..

> force mount RO while not touching the journal for disk forensics.
> I think that would also prevent the cases when a LiveCD/rescue distribution
> would not mount+replay it automagically but user would really have to
> provide the switch to the command. I am really not using the recovery
> boot cd to touch my partitions in some cases unwillingly.

I agree, there should be a way to force it to mount "really read only"
so it doesn't try to replay the journal. That might require just
ignoring the journal content, which may result in the FS appearing
corrupt, but for recovery/forensics purposes that seems better than
nothing..
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


hancockr at shaw

Jan 3, 2009, 3:58 PM

Post #11 of 88 (4143 views)
Permalink
Re: document ext3 requirements [In reply to]

Martin MOKREJÅ  wrote:
> Duane Griffin wrote:
>> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>>> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
>>> accidentally in M$ Win where I use ext2 IFS driver and modify some
>>> stuff on the ext3 drive, after a while reboot to linux and the journal
>>> get re-played ... Mmm ...
>> You *really* wouldn't want to be doing that.
>>
>> The other scenario that people have reported trouble with is
>> suspending the system, booting a live CD which "read-only" mounts the
>> filesystem (and replays the journal), then resuming.
>
> Why does not "mount -ro" die when it would have to replay the journal
> with a message that user must run fsck.ext3 in order to be able to mount
> it albeit read-only? Still I would prefer having an extra switch to

That would break typical system bootup in the unclean journal case,
normally the root FS is mounted read-only to start with (which replays
the journal) and remounted read-write later on - and usually the fsck
utilities are located on the root filesystem..

> force mount RO while not touching the journal for disk forensics.
> I think that would also prevent the cases when a LiveCD/rescue distribution
> would not mount+replay it automagically but user would really have to
> provide the switch to the command. I am really not using the recovery
> boot cd to touch my partitions in some cases unwillingly.

I agree, there should be a way to force it to mount "really read only"
so it doesn't try to replay the journal. That might require just
ignoring the journal content, which may result in the FS appearing
corrupt, but for recovery/forensics purposes that seems better than
nothing..

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


duaneg at dghda

Jan 3, 2009, 4:00 PM

Post #12 of 88 (4139 views)
Permalink
Re: document ext3 requirements [In reply to]

2009/1/3 Martin MOKREJŠ <mmokrejs [at] ribosome>:
> Why does not "mount -ro" die when it would have to replay the journal
> with a message that user must run fsck.ext3 in order to be able to mount
> it albeit read-only? Still I would prefer having an extra switch to
> force mount RO while not touching the journal for disk forensics.
> I think that would also prevent the cases when a LiveCD/rescue distribution
> would not mount+replay it automagically but user would really have to
> provide the switch to the command. I am really not using the recovery
> boot cd to touch my partitions in some cases unwillingly.

Well, that would make things rather tricky. As in, shutting down
uncleanly would render your system unbootable.

> Sure that does not prevent my case when I let ext2 IFS writing onto
> my ext3 partition. Actually, couldn't the driver at least warn me
> the journal log is non-empty (am just a user, sorry, cannot check
> myself the code at www.fs-driver.org if it could do at least this
> although it does not understand ext3). ;-)

The driver certainly should warn you in that case. I have no idea
whether it does, as I don't use it, sorry.

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mmokrejs at ribosome

Jan 3, 2009, 4:08 PM

Post #13 of 88 (4152 views)
Permalink
Re: document ext3 requirements [In reply to]

Robert Hancock wrote:
> Martin MOKREJÅ  wrote:
>> Duane Griffin wrote:
>>> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>>>> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
>>>> accidentally in M$ Win where I use ext2 IFS driver and modify some
>>>> stuff on the ext3 drive, after a while reboot to linux and the journal
>>>> get re-played ... Mmm ...
>>> You *really* wouldn't want to be doing that.
>>>
>>> The other scenario that people have reported trouble with is
>>> suspending the system, booting a live CD which "read-only" mounts the
>>> filesystem (and replays the journal), then resuming.
>>
>> Why does not "mount -ro" die when it would have to replay the journal
>> with a message that user must run fsck.ext3 in order to be able to mount
>> it albeit read-only? Still I would prefer having an extra switch to
>
> That would break typical system bootup in the unclean journal case,
> normally the root FS is mounted read-only to start with (which replays
> the journal) and remounted read-write later on - and usually the fsck
> utilities are located on the root filesystem..

Couldn't that be handled by e.g. openRC during boot, to provide the
say to be provided --force-journal-replay during "normal" boot?
Yes, that would mean e2fsprogs would become incompatible with older
versions but why not "fix" the logic?

>
>> force mount RO while not touching the journal for disk forensics. I
>> think that would also prevent the cases when a LiveCD/rescue
>> distribution would not mount+replay it automagically but user would
>> really have to provide the switch to the command. I am really not
>> using the recovery boot cd to touch my partitions in some cases
>> unwillingly.
>
> I agree, there should be a way to force it to mount "really read only"
> so it doesn't try to replay the journal. That might require just
> ignoring the journal content, which may result in the FS appearing
> corrupt, but for recovery/forensics purposes that seems better than
> nothing..

Fully agree.

M.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mmokrejs at ribosome

Jan 3, 2009, 4:11 PM

Post #14 of 88 (4147 views)
Permalink
Re: document ext3 requirements [In reply to]

Duane Griffin wrote:
> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
>> Why does not "mount -ro" die when it would have to replay the journal
>> with a message that user must run fsck.ext3 in order to be able to mount
>> it albeit read-only? Still I would prefer having an extra switch to
>> force mount RO while not touching the journal for disk forensics.
>> I think that would also prevent the cases when a LiveCD/rescue distribution
>> would not mount+replay it automagically but user would really have to
>> provide the switch to the command. I am really not using the recovery
>> boot cd to touch my partitions in some cases unwillingly.
>
> Well, that would make things rather tricky. As in, shutting down
> uncleanly would render your system unbootable.

??? If I am booted off a CD/DVD drive I just do not want my system
to be touched. I am fine if the dist mounts my drives automagically
in read-only mode but if that currently forces journal replay then no,
thanks. ;)

M.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


pavel at suse

Jan 3, 2009, 4:19 PM

Post #15 of 88 (4145 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sun 2009-01-04 00:01:58, Martin MOKREJÅ  wrote:
> Pavel Machek wrote:
> > On Sat 2009-01-03 22:17:15, Duane Griffin wrote:
> >> [Fixed top-posting]
> >>
> >> 2009/1/3 Martin MOKREJÅ  <mmokrejs [at] ribosome>:
> >>> Pavel Machek wrote:
> >>>> readonly mount does actually write to the media in some cases. Document that.
> >>>>
> >>> Can one avoid replay of the journal then if it would be unclean?
> >>> Just curious.
> >> Nope. If the underlying block device is read-only then mounting the
> >> filesystem will fail. I tried to fix this some time ago, and have a
> >> set of patches that almost always work, but "almost always" isn't good
> >> enough. Unfortunately I never managed to figure out a way to finish it
> >> off without disgusting hacks or major surgery.
> >
> > Uhuh, can you just ignore the journal and mount it anyway?
> > ...basically treating it like an ext2?
> >
> > ...ok, that will present "old" version of the filesystem to the
> > user... violating fsync() semantics.
>
> Hmm, so if my dual-boot machine does not shutdown correctly and I boot
> accidentally in M$ Win where I use ext2 IFS driver and modify some
> stuff on the ext3 drive, after a while reboot to linux and the journal
> get re-played ... Mmm ...

ext2 driver should refuse to mount dirty ext3 filesystem. (Linux ext2
driver does that).

> > Still handy for recovering badly broken filesystems, I'd say.
>
> Me as well. How about improving you doc patch with some summary of
> this thread (although it is probably not over yet)? ;-) Definitely,
> a note that one can mount it as ext2 while read-only would be helpful
> when doing some forensics on the disk.

No, you can't mount unclean ext3 as an ext2; patch to do that would be
possible but...

I believe the patch is correct & useful.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


duaneg at dghda

Jan 3, 2009, 4:41 PM

Post #16 of 88 (4126 views)
Permalink
Re: document ext3 requirements [In reply to]

2009/1/4 Martin MOKREJŠ <mmokrejs [at] ribosome>:
> Duane Griffin wrote:
>> 2009/1/3 Martin MOKREJŠ <mmokrejs [at] ribosome>:
>>> Why does not "mount -ro" die when it would have to replay the journal
>>> with a message that user must run fsck.ext3 in order to be able to mount
>>> it albeit read-only? Still I would prefer having an extra switch to
>>> force mount RO while not touching the journal for disk forensics.
>>> I think that would also prevent the cases when a LiveCD/rescue distribution
>>> would not mount+replay it automagically but user would really have to
>>> provide the switch to the command. I am really not using the recovery
>>> boot cd to touch my partitions in some cases unwillingly.
>>
>> Well, that would make things rather tricky. As in, shutting down
>> uncleanly would render your system unbootable.
>
> ??? If I am booted off a CD/DVD drive I just do not want my system
> to be touched. I am fine if the dist mounts my drives automagically
> in read-only mode but if that currently forces journal replay then no,
> thanks. ;)

I agree, it isn't a great situation. Nonetheless, it has always been
thus for ext3, and so far we've muddled along. Unless and until we can
replay the journal in-memory without touching the on-disk data, we are
stuck with it.

We can't refuse to mount an unclean FS, as that would break booting.
We also can't ignore the journal by default, if/when we get a patch to
do so at all, as that effectively corrupts random chunks of the FS.
Fine for forensics and recovery; not so much for booting from.

> M.

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


tytso at mit

Jan 3, 2009, 6:32 PM

Post #17 of 88 (4158 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sat, Jan 03, 2009 at 01:38:15PM +0100, Pavel Machek wrote:
> +Requirements
> +============
> +
> +Ext3 expects disk/storage subsystem to behave sanely. On sanely
> +behaving disk subsystem, data that have been successfully synced will
> +stay on the disk. Sane means:
> +
> +* writes to media never fail. Even if disk returns error condition during
> + write, ext3 can't handle that correctly, because success on fsync was already
> + returned when data hit the journal.
> +
> + (Fortunately writes failing are very uncommon on disks, as they
> + have spare sectors they use when write fails.)

This is not unique to ext3; per the discussion two weeks ago, this is
largely because of the fsync() interface not possibly being able to
return errors caused by failures when creating or modifying parent
directories. Given this, it's a bit misleading to place this in the
Documentation/filesystems/ext3.txt. At the minimum it should include
a discussion about what the issues might be, and given that pretty
much any Unix/Linux filesystem doesn't have a way of reflecting these
errors to application programs, it probably should be in a
filesystem-independent documentation file.

> +* either whole sector is correctly written or nothing is written during
> + powerfail.
> +
> + (Unfortuantely, none of the cheap USB/SD flash cards I seen do behave
> + like this, and are unsuitable for ext3. Because RAM tends to fail
> + faster than rest of system during powerfail, special hw killing
> + DMA transfers may be neccessary. Not sure how common that problem
> + is on generic PC machines).

Again, this is true for other filesystems (it was first discovered on
SGI "pizza boxes" machines running XFS, and special hardware changes
added to allow DMA aborts) --- in fact, because of ext3's use of
physical block journaling, it's much more likely that it will recover
from these sorts of errors. So it's very misleading to have this sort
of discussion in Documentation/filesystems/ext3.txt.

> +* either write caching is disabled, or hw can do barriers and they are enabled.
> +
> + (Note that barriers are disabled by default, use "barrier=1"
> + mount option after making sure hw can support them).

We really should get akpm to agree to accept the patch to default
barriers by default instead. :-)

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


Valdis.Kletnieks at vt

Jan 3, 2009, 7:52 PM

Post #18 of 88 (4114 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sun, 04 Jan 2009 00:41:51 GMT, Duane Griffin said:

> I agree, it isn't a great situation. Nonetheless, it has always been
> thus for ext3, and so far we've muddled along. Unless and until we can
> replay the journal in-memory without touching the on-disk data, we are
> stuck with it.

Is there a way using md/dm/lvm etc to make the source partition R/O and
replay the journal onto a CoW snapshop? Admittedly, not easy to do inside
the 'mount' command itself, but at least it might be workable for LiveCD R/O
mounts and forensics work, where you can *tell* beforehand that's what you
want and can jump through setup games before doing the mount...


patrakov at gmail

Jan 4, 2009, 5:35 AM

Post #19 of 88 (4097 views)
Permalink
Re: document ext3 requirements [In reply to]

Pavel Machek wrote:
[.CC: Alan Cox because of his reply in the "XFS internal error" thread]

> Using ext3 is only safe if storage subsystem meets certain
> criteria. Document those.

Thanks for this patch. However, after reading this, I have a stupid
question: which file system should I use if I had to reinstall my
computers from scratch now?

Ext3 means either hardware that supports barriers (not sure how to
check, and anyway I have to use encryption on the work laptop due to the
corporate policy) or disabling write cache (but, as Alan Cox said, this
shortens the lifespan of the disk). Does this requirement apply to other
journaling filesystems? Do I need journaling at all, given that I have
an UPS on my desktop and a battery in the laptop?

--
Alexander E. Patrakov
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


Valdis.Kletnieks at vt

Jan 4, 2009, 5:53 AM

Post #20 of 88 (4098 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sun, 04 Jan 2009 18:35:41 +0500, "Alexander E. Patrakov" said:

> Ext3 means either hardware that supports barriers (not sure how to
> check, and anyway I have to use encryption on the work laptop due to the
> corporate policy) or disabling write cache (but, as Alan Cox said, this
> shortens the lifespan of the disk).

False dichotomy. This isn't an "either/or", as there's a *third* case:

"understand the issues and risks involved if you have a write cache and
no barrier support, and learn to deal with it".

As you point out, if it's a laptop with a battery, the risk may be *very* low.
Let's say there's a 1 in 10,000 chance that you'll trash a file system and
need to restore from backups.

That may be totally acceptable if you've already estimated a 1 in 500 chance
of the whole damned laptop going walkies while you're not looking, and then
you *still* need to be able to restore from backups onto a replacement machine.

Yes, for some systems, the whole barriers/write cache thing is in fact very
important. But for others, data loss due to spilled coffee is a bigger worry...


duaneg at dghda

Jan 4, 2009, 6:24 AM

Post #21 of 88 (4105 views)
Permalink
Re: document ext3 requirements [In reply to]

2009/1/4 <Valdis.Kletnieks [at] vt>:
> On Sun, 04 Jan 2009 00:41:51 GMT, Duane Griffin said:
>
>> I agree, it isn't a great situation. Nonetheless, it has always been
>> thus for ext3, and so far we've muddled along. Unless and until we can
>> replay the journal in-memory without touching the on-disk data, we are
>> stuck with it.
>
> Is there a way using md/dm/lvm etc to make the source partition R/O and
> replay the journal onto a CoW snapshop? Admittedly, not easy to do inside
> the 'mount' command itself, but at least it might be workable for LiveCD R/O
> mounts and forensics work, where you can *tell* beforehand that's what you
> want and can jump through setup games before doing the mount...

Yes, something like that is best practice, as I understand it. The
LiveCD init scripts could check whether they are about to R/O mount an
ext[34] filesystem needing recovery and either refuse with a useful
message to the user, or even automatically create and mount a COW
snapshot, as you described. They'd still need to warn the user though,
since things like remounting R/W wouldn't work as expected.

Cheers,
Duane.

--
"I never could learn to drink that blood and call it wine" - Bob Dylan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


mjt at tls

Jan 4, 2009, 10:21 AM

Post #22 of 88 (4088 views)
Permalink
Re: document ext3 requirements [In reply to]

Alexander E. Patrakov wrote:
[]
> Ext3 means either hardware that supports barriers (not sure how to
> check, and anyway I have to use encryption on the work laptop due to the
> corporate policy) or disabling write cache (but, as Alan Cox said, this
> shortens the lifespan of the disk). Does this requirement apply to other
> journaling filesystems? Do I need journaling at all, given that I have
> an UPS on my desktop and a battery in the laptop?

There's another possibility too, somewhat more risky. Namely, run with
write cache ON by default, and switch it off when running off battery
(either UPS or notebook). Should save both worlds, PROVIDED the battery
actually/UPS works :)

/mjt


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


tytso at mit

Jan 4, 2009, 10:38 AM

Post #23 of 88 (4104 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sun, Jan 04, 2009 at 06:35:41PM +0500, Alexander E. Patrakov wrote:
>
> Ext3 means either hardware that supports barriers (not sure how to
> check

Pretty much all modern disk drives supports barriers. And note that
w/o barriers ext3 has worked pretty well. *If* you have a workload
pushes your system into a mode which where it is very low on memory,
so it is constantly paging/thrashing and you have a workload which is
metadata intensive, and you crash the machine while it is thrashing,
it is possible to end up in a situation where your filesystem is
corrupted and you have to use e2fsck to correct the filesystem. In
practice this is often not the case, which is why the default for ext3
has been with barriers disabled, and most people have not noted major
problems. This is why Andrew Morton has refused accept the patch for
ext3 which disables barriers by default; he's not convinced the
performance hit is worth the improvement in reliability.

Ext4 does enable barriers by defaults, mainly because filesystem
developers tend to be believe the reliability is more important than
performance. (On the other hand, Google runs with ext2 w/o
journalling, because everything is replicated three times and it's
easier to just blow away the filesystem and resync from one of the
duplicate copies; so in the right circumstances, maybe worrying only
about performance and ignoring reliability makes perfect sense.)

> and anyway I have to use encryption on the work laptop due to the
> corporate policy

If dm supported barriers, this wouldn't be an issue. Personally, I
find the convenience of LVM is so useful that I use ext4 with LVM,
even though the barrier requests get dropped on the ground. And I'm a
kernel developer, and I use a laptop with suspend/resume, which means
I often crash uncleanly --- and I've not lost data yet, despite the
lack of barriers. (On the other hand, my laptop has 4 gigs of memory,
so I'm rarely thrashing due memory pressure.)

> or disabling write cache (but, as Alan Cox said, this
> shortens the lifespan of the disk).

Huh? I've never heard an assertion that disabling the write cache (I
assume you mean using write-through caching as opposed to write-back
caching), shortens the lifespan of disk drives. Aggressive battery
saving mode is far more likely to shorten disk drive life, due to
spinning the platters up and down a lot.

> Does this requirement apply to other
> journaling filesystems? Do I need journaling at all, given that I have
> an UPS on my desktop and a battery in the laptop?

Which requirement? Barriers? Most journaling filesystems simply
enable barriers by default.

And journalling is useful so that if your system crashes, say due to
suspend and resume not working out, or the battery runs dry without
your noticing it, you can avoid running fsck at boot time. It's
really more about shorting the boot time after a crash more than
anything else.

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


tytso at mit

Jan 4, 2009, 10:40 AM

Post #24 of 88 (4095 views)
Permalink
Re: document ext3 requirements [In reply to]

On Sun, Jan 04, 2009 at 02:24:43PM +0000, Duane Griffin wrote:
> > Is there a way using md/dm/lvm etc to make the source partition R/O and
> > replay the journal onto a CoW snapshop? Admittedly, not easy to do inside
> > the 'mount' command itself, but at least it might be workable for LiveCD R/O
> > mounts and forensics work, where you can *tell* beforehand that's what you
> > want and can jump through setup games before doing the mount...
>
> Yes, something like that is best practice, as I understand it. The
> LiveCD init scripts could check whether they are about to R/O mount an
> ext[34] filesystem needing recovery and either refuse with a useful
> message to the user, or even automatically create and mount a COW
> snapshot, as you described. They'd still need to warn the user though,
> since things like remounting R/W wouldn't work as expected.

So what's the use case where people want to be able to mount a
filesystem needing recovery read/only without running the journal?

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/


sitsofe at yahoo

Jan 4, 2009, 11:08 AM

Post #25 of 88 (4101 views)
Permalink
Re: document ext3 requirements [In reply to]

Theodore Tso wrote:
> So what's the use case where people want to be able to mount a
> filesystem needing recovery read/only without running the journal?

Corrupted SD card[1] that's been locked to read only for recovery
purposes without having the FS tear itself apart further?

Others seem to be saying that it is useful for forensics...

[1] http://pavelmachek.livejournal.com/68701.html

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

First page Previous page 1 2 3 4 Next page Last page  View All Linux kernel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.