Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-User

SIGSEGV when indexing documents.

 

 

Lucene java-user RSS feed   Index | Next | Previous | View Threaded


frankmoss1950 at gmail

Jan 11, 2012, 12:28 AM

Post #1 of 8 (439 views)
Permalink
SIGSEGV when indexing documents.

I have a collection of 50 million documents and I hit the SIGSEGV error.
For every 10000 documents I perform commit.
The logs and the question has been posted to SO here: http://bit.ly/xyZUEG
where I was encouraged to post this question to this mailing list.

I have tested with different versions of java both in 6.0 and 7.0 release
lines (including older versions). I get the same error when running with
Jrockit jvm.
Memory test was ok.

Anybody had similar issue? Any tips?

Thanks.
Regards,
FM


dawid.weiss at gmail

Jan 11, 2012, 12:33 AM

Post #2 of 8 (434 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

This is a fairly old VM you're running with, but if you get the same
error with jrockit then I would assume it's a hardware problem. Any
other software crashing? Do you have ECC-memory? Have you tried IBM's
J9 VM as well maybe?

Dawid


On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail> wrote:
> I have a collection of 50 million documents and I hit the SIGSEGV error.
> For every 10000 documents I perform commit.
> The logs and the question has been posted to SO here: http://bit.ly/xyZUEG
> where I was encouraged to post this question to this mailing list.
>
> I have tested with different versions of java both in 6.0 and 7.0 release
> lines (including older versions). I get the same error when running with
> Jrockit jvm.
> Memory test was ok.
>
> Anybody had similar issue? Any tips?
>
> Thanks.
> Regards,
> FM

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


frankmoss1950 at gmail

Jan 11, 2012, 12:50 AM

Post #3 of 8 (435 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

Thanks for your quick reply.

As I said, it happens with the latest version of VM, i.e, both
"jdk1.6.0_30" and "jdk1.7.0_02"

The memory is not an ECC one it seems.

I have a kit of this (3*4GB * 2 = 24GB):

http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf

On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:

> This is a fairly old VM you're running with, but if you get the same
> error with jrockit then I would assume it's a hardware problem. Any
> other software crashing? Do you have ECC-memory? Have you tried IBM's
> J9 VM as well maybe?
>
> Dawid
>
>
> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail>
> wrote:
> > I have a collection of 50 million documents and I hit the SIGSEGV error.
> > For every 10000 documents I perform commit.
> > The logs and the question has been posted to SO here:
> http://bit.ly/xyZUEG
> > where I was encouraged to post this question to this mailing list.
> >
> > I have tested with different versions of java both in 6.0 and 7.0 release
> > lines (including older versions). I get the same error when running with
> > Jrockit jvm.
> > Memory test was ok.
> >
> > Anybody had similar issue? Any tips?
> >
> > Thanks.
> > Regards,
> > FM
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>
>


dawid.weiss at gmail

Jan 11, 2012, 12:56 AM

Post #4 of 8 (437 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

The dump you're getting indicates a sigserv in a garbage collection.
This isn't unlikely (there are bugs in there as well), but less likely
than a hardware error on your side... at least in my opinion. I would
experiment with the following:

1) do you really need a 4gb max stack? Seems weird to me.
2) collect the dump from jrockit; is it something related to memory
collection? If so this would suggest memory issues again
3) try ibm j9 and see if you can reproduce the problem. If so, again
-- I'd double check the hardware.
4) collect more dumps from hotspot (same version) and compare them
against each other; is the problem recurring in the same place
(gc-related)? If so, then run with the -fastdebug VM (unfortunately
you'll probably need to build this one yourself these days) and make
the core dump available to gc mailing list folks, they'll be able to
investigate.

Dawid

On Wed, Jan 11, 2012 at 9:50 AM, Frank Moss <frankmoss1950 [at] gmail> wrote:
> Thanks for your quick reply.
>
> As I said, it happens with the latest version of VM, i.e, both
> "jdk1.6.0_30" and "jdk1.7.0_02"
>
> The memory is not an ECC one it seems.
>
> I have a kit of this (3*4GB * 2 = 24GB):
>
> http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf
>
> On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:
>
>> This is a fairly old VM you're running with, but if you get the same
>> error with jrockit then I would assume it's a hardware problem. Any
>> other software crashing? Do you have ECC-memory? Have you tried IBM's
>> J9 VM as well maybe?
>>
>> Dawid
>>
>>
>> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail>
>> wrote:
>> > I have a collection of 50 million documents and I hit the SIGSEGV error.
>> > For every 10000 documents I perform commit.
>> > The logs and the question has been posted to SO here:
>> http://bit.ly/xyZUEG
>> > where I was encouraged to post this question to this mailing list.
>> >
>> > I have tested with different versions of java both in 6.0 and 7.0 release
>> > lines (including older versions). I get the same error when running with
>> > Jrockit jvm.
>> > Memory test was ok.
>> >
>> > Anybody had similar issue? Any tips?
>> >
>> > Thanks.
>> > Regards,
>> > FM
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> For additional commands, e-mail: java-user-help [at] lucene
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


frankmoss1950 at gmail

Jan 11, 2012, 1:02 AM

Post #5 of 8 (438 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

4gb is the initial heap size. Are you thinking about Xss? I will try it
as well as the rest of your suggestions and post back the results.

Thanks.


On Wed, Jan 11, 2012 at 9:56 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:

> The dump you're getting indicates a sigserv in a garbage collection.
> This isn't unlikely (there are bugs in there as well), but less likely
> than a hardware error on your side... at least in my opinion. I would
> experiment with the following:
>
> 1) do you really need a 4gb max stack? Seems weird to me.
> 2) collect the dump from jrockit; is it something related to memory
> collection? If so this would suggest memory issues again
> 3) try ibm j9 and see if you can reproduce the problem. If so, again
> -- I'd double check the hardware.
> 4) collect more dumps from hotspot (same version) and compare them
> against each other; is the problem recurring in the same place
> (gc-related)? If so, then run with the -fastdebug VM (unfortunately
> you'll probably need to build this one yourself these days) and make
> the core dump available to gc mailing list folks, they'll be able to
> investigate.
>
> Dawid
>
> On Wed, Jan 11, 2012 at 9:50 AM, Frank Moss <frankmoss1950 [at] gmail>
> wrote:
> > Thanks for your quick reply.
> >
> > As I said, it happens with the latest version of VM, i.e, both
> > "jdk1.6.0_30" and "jdk1.7.0_02"
> >
> > The memory is not an ECC one it seems.
> >
> > I have a kit of this (3*4GB * 2 = 24GB):
> >
> > http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf
> >
> > On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail>
> wrote:
> >
> >> This is a fairly old VM you're running with, but if you get the same
> >> error with jrockit then I would assume it's a hardware problem. Any
> >> other software crashing? Do you have ECC-memory? Have you tried IBM's
> >> J9 VM as well maybe?
> >>
> >> Dawid
> >>
> >>
> >> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail>
> >> wrote:
> >> > I have a collection of 50 million documents and I hit the SIGSEGV
> error.
> >> > For every 10000 documents I perform commit.
> >> > The logs and the question has been posted to SO here:
> >> http://bit.ly/xyZUEG
> >> > where I was encouraged to post this question to this mailing list.
> >> >
> >> > I have tested with different versions of java both in 6.0 and 7.0
> release
> >> > lines (including older versions). I get the same error when running
> with
> >> > Jrockit jvm.
> >> > Memory test was ok.
> >> >
> >> > Anybody had similar issue? Any tips?
> >> >
> >> > Thanks.
> >> > Regards,
> >> > FM
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> >> For additional commands, e-mail: java-user-help [at] lucene
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>
>


dawid.weiss at gmail

Jan 11, 2012, 1:06 AM

Post #6 of 8 (428 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

Opps, yes, sorry -- I only quickly looked at the invocation line on
stack overflow and overlooked it. -Xms4g shouldn't make any
difference.

Dawid

On Wed, Jan 11, 2012 at 10:02 AM, Frank Moss <frankmoss1950 [at] gmail> wrote:
> 4gb is the initial heap size. Are you  thinking about Xss?  I will try it
> as well as the rest of your suggestions and post back the results.
>
> Thanks.
>
>
> On Wed, Jan 11, 2012 at 9:56 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:
>
>> The dump you're getting indicates a sigserv in a garbage collection.
>> This isn't unlikely (there are bugs in there as well), but less likely
>> than a hardware error on your side... at least in my opinion. I would
>> experiment with the following:
>>
>> 1) do you really need a 4gb max stack? Seems weird to me.
>> 2) collect the dump from jrockit; is it something related to memory
>> collection? If so this would suggest memory issues again
>> 3) try ibm j9 and see if you can reproduce the problem. If so, again
>> -- I'd double check the hardware.
>> 4) collect more dumps from hotspot (same version) and compare them
>> against each other; is the problem recurring in the same place
>> (gc-related)? If so, then run with the -fastdebug VM (unfortunately
>> you'll probably need to build this one yourself these days) and make
>> the core dump available to gc mailing list folks, they'll be able to
>> investigate.
>>
>> Dawid
>>
>> On Wed, Jan 11, 2012 at 9:50 AM, Frank Moss <frankmoss1950 [at] gmail>
>> wrote:
>> > Thanks for your quick reply.
>> >
>> > As I said, it happens with the latest version of VM, i.e, both
>> > "jdk1.6.0_30" and "jdk1.7.0_02"
>> >
>> > The memory is not an ECC one it seems.
>> >
>> > I have a kit of this (3*4GB * 2 = 24GB):
>> >
>> > http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf
>> >
>> > On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail>
>> wrote:
>> >
>> >> This is a fairly old VM you're running with, but if you get the same
>> >> error with jrockit then I would assume it's a hardware problem. Any
>> >> other software crashing? Do you have ECC-memory? Have you tried IBM's
>> >> J9 VM as well maybe?
>> >>
>> >> Dawid
>> >>
>> >>
>> >> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail>
>> >> wrote:
>> >> > I have a collection of 50 million documents and I hit the SIGSEGV
>> error.
>> >> > For every 10000 documents I perform commit.
>> >> > The logs and the question has been posted to SO here:
>> >> http://bit.ly/xyZUEG
>> >> > where I was encouraged to post this question to this mailing list.
>> >> >
>> >> > I have tested with different versions of java both in 6.0 and 7.0
>> release
>> >> > lines (including older versions). I get the same error when running
>> with
>> >> > Jrockit jvm.
>> >> > Memory test was ok.
>> >> >
>> >> > Anybody had similar issue? Any tips?
>> >> >
>> >> > Thanks.
>> >> > Regards,
>> >> > FM
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> >> For additional commands, e-mail: java-user-help [at] lucene
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> For additional commands, e-mail: java-user-help [at] lucene
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


frankmoss1950 at gmail

Jan 11, 2012, 1:47 AM

Post #7 of 8 (428 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

The same with IBM J9. The dump file is attached.

It seems to be HW related. Recently, we have added more RAM. We actually
removed old memory bricks and put a new set. Before increasing the RAM, I
kept getting OutOfMemoryError.

How do I check what exactly the problem is and with what hardware? I run
the memory check and it did not detect any errors on OpenSuse 11.3, 64bit.



On Wed, Jan 11, 2012 at 10:06 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:

> Opps, yes, sorry -- I only quickly looked at the invocation line on
> stack overflow and overlooked it. -Xms4g shouldn't make any
> difference.
>
> Dawid
>
> On Wed, Jan 11, 2012 at 10:02 AM, Frank Moss <frankmoss1950 [at] gmail>
> wrote:
> > 4gb is the initial heap size. Are you thinking about Xss? I will try it
> > as well as the rest of your suggestions and post back the results.
> >
> > Thanks.
> >
> >
> > On Wed, Jan 11, 2012 at 9:56 AM, Dawid Weiss <dawid.weiss [at] gmail>
> wrote:
> >
> >> The dump you're getting indicates a sigserv in a garbage collection.
> >> This isn't unlikely (there are bugs in there as well), but less likely
> >> than a hardware error on your side... at least in my opinion. I would
> >> experiment with the following:
> >>
> >> 1) do you really need a 4gb max stack? Seems weird to me.
> >> 2) collect the dump from jrockit; is it something related to memory
> >> collection? If so this would suggest memory issues again
> >> 3) try ibm j9 and see if you can reproduce the problem. If so, again
> >> -- I'd double check the hardware.
> >> 4) collect more dumps from hotspot (same version) and compare them
> >> against each other; is the problem recurring in the same place
> >> (gc-related)? If so, then run with the -fastdebug VM (unfortunately
> >> you'll probably need to build this one yourself these days) and make
> >> the core dump available to gc mailing list folks, they'll be able to
> >> investigate.
> >>
> >> Dawid
> >>
> >> On Wed, Jan 11, 2012 at 9:50 AM, Frank Moss <frankmoss1950 [at] gmail>
> >> wrote:
> >> > Thanks for your quick reply.
> >> >
> >> > As I said, it happens with the latest version of VM, i.e, both
> >> > "jdk1.6.0_30" and "jdk1.7.0_02"
> >> >
> >> > The memory is not an ECC one it seems.
> >> >
> >> > I have a kit of this (3*4GB * 2 = 24GB):
> >> >
> >> > http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf
> >> >
> >> > On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail>
> >> wrote:
> >> >
> >> >> This is a fairly old VM you're running with, but if you get the same
> >> >> error with jrockit then I would assume it's a hardware problem. Any
> >> >> other software crashing? Do you have ECC-memory? Have you tried IBM's
> >> >> J9 VM as well maybe?
> >> >>
> >> >> Dawid
> >> >>
> >> >>
> >> >> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss <frankmoss1950 [at] gmail
> >
> >> >> wrote:
> >> >> > I have a collection of 50 million documents and I hit the SIGSEGV
> >> error.
> >> >> > For every 10000 documents I perform commit.
> >> >> > The logs and the question has been posted to SO here:
> >> >> http://bit.ly/xyZUEG
> >> >> > where I was encouraged to post this question to this mailing list.
> >> >> >
> >> >> > I have tested with different versions of java both in 6.0 and 7.0
> >> release
> >> >> > lines (including older versions). I get the same error when running
> >> with
> >> >> > Jrockit jvm.
> >> >> > Memory test was ok.
> >> >> >
> >> >> > Anybody had similar issue? Any tips?
> >> >> >
> >> >> > Thanks.
> >> >> > Regards,
> >> >> > FM
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> >> >> For additional commands, e-mail: java-user-help [at] lucene
> >> >>
> >> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> >> For additional commands, e-mail: java-user-help [at] lucene
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>
>
Attachments: javacore.20120111.103257.31238.0002.txt (208 KB)


dawid.weiss at gmail

Jan 11, 2012, 1:51 AM

Post #8 of 8 (430 views)
Permalink
Re: SIGSEGV when indexing documents. [In reply to]

No clue, I am not a hardware expert. Removing memory extensions one by
one (or binary-searching for the faulty one)?

Dawid

On Wed, Jan 11, 2012 at 10:47 AM, Frank Moss <frankmoss1950 [at] gmail> wrote:
> The same with IBM J9. The dump file is attached.
>
> It seems to be HW related. Recently, we have added more RAM. We actually
> removed old memory bricks and put a new set. Before increasing the RAM, I
> kept getting OutOfMemoryError.
>
> How do I check what exactly the problem is and with what hardware? I run the
> memory check and it did not detect any errors on OpenSuse 11.3,  64bit.
>
>
>
> On Wed, Jan 11, 2012 at 10:06 AM, Dawid Weiss <dawid.weiss [at] gmail> wrote:
>>
>> Opps, yes, sorry -- I only quickly looked at the invocation line on
>> stack overflow and overlooked it. -Xms4g shouldn't make any
>> difference.
>>
>> Dawid
>>
>> On Wed, Jan 11, 2012 at 10:02 AM, Frank Moss <frankmoss1950 [at] gmail>
>> wrote:
>> > 4gb is the initial heap size. Are you  thinking about Xss?  I will try
>> > it
>> > as well as the rest of your suggestions and post back the results.
>> >
>> > Thanks.
>> >
>> >
>> > On Wed, Jan 11, 2012 at 9:56 AM, Dawid Weiss <dawid.weiss [at] gmail>
>> > wrote:
>> >
>> >> The dump you're getting indicates a sigserv in a garbage collection.
>> >> This isn't unlikely (there are bugs in there as well), but less likely
>> >> than a hardware error on your side... at least in my opinion. I would
>> >> experiment with the following:
>> >>
>> >> 1) do you really need a 4gb max stack? Seems weird to me.
>> >> 2) collect the dump from jrockit; is it something related to memory
>> >> collection? If so this would suggest memory issues again
>> >> 3) try ibm j9 and see if you can reproduce the problem. If so, again
>> >> -- I'd double check the hardware.
>> >> 4) collect more dumps from hotspot (same version) and compare them
>> >> against each other; is the problem recurring in the same place
>> >> (gc-related)? If so, then run with the -fastdebug VM (unfortunately
>> >> you'll probably need to build this one yourself these days) and make
>> >> the core dump available to gc mailing list folks, they'll be able to
>> >> investigate.
>> >>
>> >> Dawid
>> >>
>> >> On Wed, Jan 11, 2012 at 9:50 AM, Frank Moss <frankmoss1950 [at] gmail>
>> >> wrote:
>> >> > Thanks for your quick reply.
>> >> >
>> >> > As I said, it happens with the latest version of VM, i.e, both
>> >> > "jdk1.6.0_30" and "jdk1.7.0_02"
>> >> >
>> >> > The memory is not an ECC one it seems.
>> >> >
>> >> > I have a kit of this (3*4GB * 2 = 24GB):
>> >> >
>> >> > http://www.kingston.com/datasheets/KHX1600C9S3K2_8GX.pdf
>> >> >
>> >> > On Wed, Jan 11, 2012 at 9:33 AM, Dawid Weiss <dawid.weiss [at] gmail>
>> >> wrote:
>> >> >
>> >> >> This is a fairly old VM you're running with, but if you get the same
>> >> >> error with jrockit then I would assume it's a hardware problem. Any
>> >> >> other software crashing? Do you have ECC-memory? Have you tried
>> >> >> IBM's
>> >> >> J9 VM as well maybe?
>> >> >>
>> >> >> Dawid
>> >> >>
>> >> >>
>> >> >> On Wed, Jan 11, 2012 at 9:28 AM, Frank Moss
>> >> >> <frankmoss1950 [at] gmail>
>> >> >> wrote:
>> >> >> > I have a collection of 50 million documents and I hit the SIGSEGV
>> >> error.
>> >> >> > For every 10000 documents I perform commit.
>> >> >> > The logs and the question has been posted to SO here:
>> >> >> http://bit.ly/xyZUEG
>> >> >> > where I was encouraged to post this question to this mailing list.
>> >> >> >
>> >> >> > I have tested with different versions of java both in 6.0 and 7.0
>> >> release
>> >> >> > lines (including older versions). I get the same error when
>> >> >> > running
>> >> with
>> >> >> > Jrockit jvm.
>> >> >> > Memory test was ok.
>> >> >> >
>> >> >> > Anybody had similar issue? Any tips?
>> >> >> >
>> >> >> > Thanks.
>> >> >> > Regards,
>> >> >> > FM
>> >> >>
>> >> >>
>> >> >> ---------------------------------------------------------------------
>> >> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> >> >> For additional commands, e-mail: java-user-help [at] lucene
>> >> >>
>> >> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> >> For additional commands, e-mail: java-user-help [at] lucene
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
>> For additional commands, e-mail: java-user-help [at] lucene
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Lucene java-user RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.