Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: kinosearch: discuss

Error in function kino_FSFileDes_fdseek

 

 

kinosearch discuss RSS feed   Index | Next | Previous | View Threaded


edwardbetts at gmail

Mar 4, 2007, 12:21 AM

Post #1 of 3 (261 views)
Permalink
Error in function kino_FSFileDes_fdseek

I'm seeing this error when I try the new KinoSearch:

Error in function kino_FSFileDes_fdseek at
c_src/KinoSearch/Store/FSFileDes.c:64: File operation failed: Invalid
argument
at /usr/lib/perl5/KinoSearch/Index/SegReader.pm line 120
KinoSearch::Index::SegReader::fetch_term_info('KinoSearch::Index::SegReader=HASH(0x8485600)',
'KinoSearch::Index::Term=SCALAR(0x84a9f70)') called at
/usr/lib/perl5/KinoSearch/Index/SegReader.pm line 125
KinoSearch::Index::SegReader::doc_freq('KinoSearch::Index::SegReader=HASH(0x8485600)',
'KinoSearch::Index::Term=SCALAR(0x84a9f70)') called at
/usr/lib/perl5/KinoSearch/Searcher.pm line 163
KinoSearch::Searcher::doc_freq('KinoSearch::Searcher=HASH(0x8485210)',
'KinoSearch::Index::Term=SCALAR(0x84a9f70)') called at
/usr/lib/perl5/KinoSearch/Search/Similarity.pm line 31
KinoSearch::Search::Similarity::idf('KinoSearch::Search::Similarity=SCALAR(0x84aa210)',
'KinoSearch::Index::Term=SCALAR(0x84a9f70)',
'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Search/TermQuery.pm line 65
KinoSearch::Search::TermWeight::init_instance('KinoSearch::Search::TermWeight=HASH(0x84aa1c8)')
called at /usr/lib/perl5/KinoSearch/Util/Class.pm line 40
KinoSearch::Util::Class::new('KinoSearch::Search::TermWeight',
'parent', 'KinoSearch::Search::TermQuery=HASH(0x84a9fa0)', 'searcher',
'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Search/TermQuery.pm line 26
KinoSearch::Search::TermQuery::create_weight('KinoSearch::Search::TermQuery=HASH(0x84a9fa0)',
'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Search/BooleanQuery.pm line 97
KinoSearch::Search::BooleanWeight::init_instance('KinoSearch::Search::BooleanWeight=HASH(0x84ae9d0)')
called at /usr/lib/perl5/KinoSearch/Util/Class.pm line 40
KinoSearch::Util::Class::new('KinoSearch::Search::BooleanWeight',
'parent', 'KinoSearch::Search::BooleanQuery=HASH(0x84a9fe8)',
'searcher', 'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Search/BooleanQuery.pm line 54
KinoSearch::Search::BooleanQuery::create_weight('KinoSearch::Search::BooleanQuery=HASH(0x84a9fe8)',
'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Search/Query.pm line 33
KinoSearch::Search::Query::to_weight('KinoSearch::Search::BooleanQuery=HASH(0x84a9fe8)',
'KinoSearch::Searcher=HASH(0x8485210)') called at
/usr/lib/perl5/KinoSearch/Searcher.pm line 168
KinoSearch::Searcher::create_weight('KinoSearch::Searcher=HASH(0x8485210)',
'KinoSearch::Search::BooleanQuery=HASH(0x84a9fe8)') called at
/usr/lib/perl5/KinoSearch/Searcher.pm line 86
KinoSearch::Searcher::search_top_docs('KinoSearch::Searcher=HASH(0x8485210)',
'num_wanted', 10, 'query',
'KinoSearch::Search::BooleanQuery=HASH(0x84a9fe8)', 'filter', 'undef',
'sort_spec', 'undef', ...) called at
/usr/lib/perl5/KinoSearch/Search/Hits.pm line 47
KinoSearch::Search::Hits::seek(3) called at
/usr/lib/perl5/KinoSearch/Searcher.pm line 70
KinoSearch::Searcher::search('KinoSearch::Searcher=HASH(0x8485210)',
'query', 'the') called at ./demo.pl line 42

Any ideas what's going wrong? I'm running on Debian unstable, Perl
version v5.8.8. The code that is giving this error looks like this:

#!/usr/bin/perl

use strict; use warnings;

use KinoSearch::Analysis::PolyAnalyzer;
use KinoSearch::InvIndexer;
use KinoSearch::Searcher;

package bigmunch::ingredients;
use base qw( KinoSearch::Schema::FieldSpec );
package bigmunch::intro;
use base qw( KinoSearch::Schema::FieldSpec );
package bigmunch;
use base qw( KinoSearch::Schema );

__PACKAGE__->init_fields(qw(ingredients intro));

sub analyzer {
KinoSearch::Analysis::PolyAnalyzer->new( language => 'en' );
}

package main;

my $invindexer = KinoSearch::InvIndexer->new( invindex =>
bigmunch->clobber('index') );

my @items = (
{
ingredients => "For an 8 inch (20 cm) cake:\n6 oz (175 g)
self-raising flour\n1 rounded tsp baking powder\n3 large eggs at room
temperature\n6 oz (175 g) very soft butter\n6 oz (175 g) golden caster
sugar\n\xBD tsp vanilla extract\na little sifted icing sugar, for
dusting\nFor the filling:\n6 passionfruit\n9 oz (250 g) mascarpone\n1
level dessertspoon golden caster sugar\n1 tsp vanilla extract\n7 fl oz
(200 ml) fromage frais\nNote: You will also need two 8 inch (20 cm)
1\xBD inch (4 cm) deep sponge tins, lightly greased and the bases
lined with silicone paper (parchment).",
intro => "",
},
{
ingredients => "For the spice paste\n2 plump stalks of smaller
ones)\nwalnut-sized lump of ginger, peeled and shredded\n2 small red
shallots, roughly chopped\n4 chopped\n\xBDtsp blachan (dried shrimp
paste)\nFor the curry\n3 tbsp groundnut oil\n400g/14oz mushrooms,
sliced in half\n325ml/11fl oz vegetable stock\n8 baby or egg
aubergines, quartered\n6 large tomatoes (not beefsteak),
chopped\n200ml/7fl oz coconut milk\nsmall handful of coriander
leaves\nsteamed basmati rice, to serve",
intro => " As always with curries, lots of ingredients but very
little hands-on cooking time. ",
},
);

$invindexer->add_doc($_) for @items;
$invindexer->finish;

my $searcher = KinoSearch::Searcher->new( invindex => bigmunch->open('index') );
$searcher->search( query => 'the' );

--
Edward Betts


marvin at rectangular

Mar 5, 2007, 9:35 AM

Post #2 of 3 (248 views)
Permalink
Error in function kino_FSFileDes_fdseek [In reply to]

On Mar 3, 2007, at 11:44 PM, Edward Betts wrote:

> The code that is giving this error looks like this:
>
> #!/usr/bin/perl

Fabulous, Edward. Thanks for the failing test case!!

I'm able to reproduce the problem 100% on my box and will look into it.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/


marvin at rectangular

Mar 6, 2007, 2:11 PM

Post #3 of 3 (246 views)
Permalink
Error in function kino_FSFileDes_fdseek [In reply to]

On Mar 3, 2007, at 11:44 PM, Edward Betts wrote:

> Error in function kino_FSFileDes_fdseek at
> c_src/KinoSearch/Store/FSFileDes.c:64: File operation failed: Invalid

Found the bug. It was one missing line in c_src/KinoSearch/Index/
TermListWriter.c.

Index: c_src/KinoSearch/Index/TermListWriter.c
===================================================================
--- ../c_src/KinoSearch/Index/TermListWriter.c (revision 2112)
+++ ../c_src/KinoSearch/Index/TermListWriter.c (revision 2113)
@@ -157,6 +157,7 @@
TInfo_Reset(self->last_tinfo);
self->last_tinfo->field_num = field_num;
self->last_text->len = 0;
+ self->last_tl_ptr = 0;
}
/* Close files, store metadata for a field.

Previously, all terms in all fields were held in a single pair of
files per segment (_XXX.tis and _XXX.tii). In KS 0.20, each field now
gets its own dedicated pair of term list files (_XXX.tlYYY and
_XXX.tlxYYY) -- a change that has been long planned, but was finally
implemented with a commit on Febrary 23.

The problem occurred when switching over to a new file. Instead of
resetting the last_tl_fileptr (last term list file pointer) variable
to 0, it was held over. This incorrect value could lead to garbage
file pointers when indexing multiple fields, for fields other than
the first -- provided that the first field had sufficient content.

All the multi-field search tests in the KS test suite used minimal
content for simplicity, and the stress test is single-field -- thus
the buggy code was not exercised.

Squashing this bug makes the failing test app run cleanly and solves
the problem I identified with large indexes returning incorrect
results. I strongly suspect it solves Karel K's crashing problem as
well, looking at his stack trace.

A new dev release will be uploaded to CPAN later today if prep goes
cleanly (Valgrind reveals no glitches, etc).

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

kinosearch discuss RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.