Mailing List Archive: Linux: Kernel

Automating basic patch tests...

Index | Next | Previous | View Threaded

rhw at bigfoot

Sep 30, 1998, 1:59 PM

Post #1 of 6 (40 views)
Permalink

Automating basic patch tests...

Hi there.
===8<=== CUT ===>8=== (Subject was "Re: Jitterbug")
> JitterBug could also test new patches in some way or another (at
> the least it could see if with the new patch the kernel would
> compile), and then send a reply to the submitter if it fails with
> an explanation as to why. Admittedly this will cause the machine
> hosting this to be rather busy if alot of patches come in, but it
> would be a usefull test..
As a very basic minimum, I'd like to suggest that it perform the
following tests:
1. The submitted patch applies cleanly to the current kernel,
else it is automatically rejected. As I see it, there's little
point putting patches into the system if one has to use an old
kernel to apply them to, or if they can only be applied against
some semi-mangled kernel in the first place.
Yes, I have seen patches posted that just don't apply cleanly
to the current stable kernel when they claim to...and both of
the above reasons have occurred...
2. Does the patch only modify documentation files? If so, then
it probably doesn't need to go to Linus at all, but to somebody
who has taken responsibility for the documentation and thus
relieved Linus of the headache of worrying about that.
Incidentally, if this suggestion is taken up, the person made
responsible for the documentation needs to be agreed by ALL
of the current admin, otherwise it just won't work...and since
I'm basically an unknown to the Linux admin, there's no point
in my volunteering for the job, much as I'd like to.
3. How many different files does the patch modify? Should there
be a limit on this number, to discourage submission of patches
that change too much at once? In my opinion, it's easier to
evaluate a patch that only patches one file than one that
modifies a dozen unrelated files.
4. (As an alternative to 3) How many different directories does
the patch modify files in? If a patch is limited to files in
a single directory, or in two directories that share a
parent-child relationship, it probably only patches a single
subsystem of the Linux kernel.
As I see things, any tests for compilability should only be applied
AFTER all of the above tests have been passed.
Comments, anybody?
Best wishes from Riley.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

helge.hafting at daldata

Oct 1, 1998, 1:30 AM

Post #2 of 6 (36 views)
Permalink

Re: Automating basic patch tests... [In reply to]

In <Pine.LNX.3.96.980930214011.31107D-100000 [at] ps>, on
09/30/98
at 09:59 PM, Riley Williams <rhw [at] bigfoot> said:
[...]
>As a very basic minimum, I'd like to suggest that it perform the
>following tests:
[good tests snipped]
> 3. How many different files does the patch modify? Should there
> be a limit on this number, to discourage submission of patches
> that change too much at once?
> 4. (As an alternative to 3) How many different directories does
> the patch modify files in?
I believe there will be infrequent occations where a patch
has to modify files "all over the place" If those are auto-
rejected then some alternate mechanism must be used for such
occations. (Such as emailing Linus directly)
Not good, because people getting rejects for other reasons may decide to
use the alternate approach too.
Instead of auto-rejecting just send the author a warning that he ought to
be careful with this type of patch, and mark it as "dubious". Only real
people can make a good decision.
If such patches become a workload problem anyway, forward them to someone
else who can see if they make sense before
possibly passing them on to Linus.
Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

rhw at bigfoot

Oct 1, 1998, 6:01 AM

Post #3 of 6 (35 views)
Permalink

Re: Automating basic patch tests... [In reply to]

Hi Helge.
On Thu, 1 Oct 1998, Helge Hafting wrote:
>> As a very basic minimum, I'd like to suggest that it perform the
>> following tests:
> [good tests snipped]
>> 3. How many different files does the patch modify? Should there
>> be a limit on this number, to discourage submission of patches
>> that change too much at once?
>> 4. (As an alternative to 3) How many different directories does
>> the patch modify files in?
> I believe there will be infrequent occations where a patch
> has to modify files "all over the place". If those are auto-
> rejected then some alternate mechanism must be used for such
> occations. (Such as emailing Linus directly)
I agree there, but would suggest that a patch that failed either of
the above tests was in need of serious scrutiny BEFORE any sort of
consideration is given as to its compilability - see below for how I
see the patch submittal process going...
> Not good, because people getting rejects for other reasons may
> decide to use the alternate approach too.
Agreed.
> Instead of auto-rejecting just send the author a warning that he
> ought to be careful with this type of patch, and mark it as
> "dubious". Only real people can make a good decision.
Like I said, the purpose of those two tests isn't to reject the
patches, but to determine whether they are good candidates for
auto-verification or should be passed directly to a human verifier.
> If such patches become a workload problem anyway, forward them to
> someone else who can see if they make sense before possibly passing
> them on to Linus.
I would see the patch submittal procedure as going something like the
following sketch diagram...
---------------------------
[ Patch submitted by author ]
---------------------------
|
------------------------------
[ Auto-verifier checks patches ]
------------------------------
| | |
Query Reject Accept
| | |
| | ----------------
| | [ Auto-compiler ]
| | [ checks patches ]
| | ----------------
| | | |
| | Reject Accept
| | | |
| |-<------+ |
| | |
| ------------------ |
| [ Email rejection ] |
| [ to patch author ] |
| [ with explanation ] |
| ------------------ |
| | |
| --------------- |
| [ Discard patch ] |
| --------------- |
| |
+------------>-+-<-------------+
|
-------------------------
[ Forward patch to most ]
[ relevant human verifier ]
-------------------------
How does that look to you?
I would also add that the explanation should include suggestions as to
measures the author could take to make the patch more acceptable to
the system. Items in this list could include...
1. If the patch fails to apply cleanly against the current kernel,
suggest that the author upgrade to the current sources, then
apply the patch to them and submit the result.
2. If the patch fails to compile against the current kernel, supply
the compiler's error messages and ask the author to investigate
and submit a corrected patch.
3. Some patches will duplicate the results of other patches. If this
is detected, it might be an idea to suggest that the patch authors
concerned contact each other to discuss their ideas.
I'm sure other suggestions will soon come to light as well...
Best wishes from Riley.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

jrv at vanzandt

Oct 4, 1998, 2:57 PM

Post #4 of 6 (35 views)
Permalink

Re: Automating basic patch tests... [In reply to]

Helge Hafting writes:
> at 09:59 PM, Riley Williams <rhw [at] bigfoot> said:
>> 3. How many different files does the patch modify? Should there
>> be a limit on this number, to discourage submission of patches
>> that change too much at once?
...
>I believe there will be infrequent occations where a patch
>has to modify files "all over the place"
For example: spelling corrections
- Jim Van Zandt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

linker at z

Oct 5, 1998, 10:56 PM

Post #5 of 6 (35 views)
Permalink

Re: Automating basic patch tests... [In reply to]

On Sun, 4 Oct 1998, James R. Van Zandt wrote:
>
>
> Helge Hafting writes:
> > at 09:59 PM, Riley Williams <rhw [at] bigfoot> said:
> >> 3. How many different files does the patch modify? Should there
> >> be a limit on this number, to discourage submission of patches
> >> that change too much at once?
> ...
> >I believe there will be infrequent occations where a patch
> >has to modify files "all over the place"
>
> For example: spelling corrections
>
> - Jim Van Zandt
Or better, struct changes in central .h files.. Or say we added a second
param to udelay (why? idunno? RT support?)..
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

rhw at bigfoot

Oct 6, 1998, 1:02 PM

Post #6 of 6 (34 views)
Permalink

Re: Automating basic patch tests... [In reply to]

Hi Gregory, James...
>>>> 3. How many different files does the patch modify? Should there
>>>> be a limit on this number, to discourage submission of patches
>>>> that change too much at once?
>>> I believe there will be infrequent occations where a patch
>>> has to modify files "all over the place"
>> For example: spelling corrections
> Or better, struct changes in central .h files.. Or say we added a
> second param to udelay (why? idunno? RT support?)..
Try reading my original post, or my previous reply clarifying the
misunderstanding both of you appear to have made...
1. I stated that if there were more than a few files modified by a
given patch, there was very little point in having any form of
auto-compiler check the patch out, so it should go direct to a
human verifier.
2. In general, spelling corrections will normally refer not to the
actual source files, but to documentation files, and a previous
test has already excluded them from consideration for this test,
so that example is in general a non-starter...
Personally, I'm beginning to wonder whether anybody really cares what
happens to Linux, since the response to my post has all been negative
responses to the same suggestion as you replied to, and all have made
the same misunderstanding of what I said as you appear to have...
Best wishes from Riley.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo [at] vger
Please read the FAQ at http://www.tux.org/lkml/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads