jira at apache
Nov 26, 2009, 5:03 PM
Post #7 of 8
(323 views)
Permalink
|
|
[jira] Updated: (LUCENE-1844) Speed up junit tests
[In reply to]
|
|
[ https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1844:
---------------------------------------
Attachment: LUCENE-1844.patch
OK I started from Erick's last patch, and trimmed back some more tests -- patch attached.
Also I temporarily instrumented the tests to print msec per case -- with the attached patch, here are the test times that are > 1 sec:
{code}
Test times:
17.489 sec TestIndexWriter [test-tag]
17.315 sec TestIndexWriter
9.952 sec TestQualityRun
9.163 sec TestIndexWriterReader [test-tag]
9.008 sec TestIndexWriterReader
8.508 sec TestTimeLimitingCollector
8.410 sec TestTimeLimitingCollector [test-tag]
6.982 sec TestIndexReader
6.930 sec TestIndexReader [test-tag]
6.756 sec TestAtomicUpdate
6.740 sec TestAtomicUpdate [test-tag]
6.596 sec TestAtomicUpdate:testAtomicUpdates [test-tag]
6.303 sec TestTransactions
6.290 sec TestTransactions [test-tag]
6.287 sec TestIndexWriterExceptions
6.277 sec TestIndexWriterExceptions [test-tag]
6.140 sec TestTransactions:testTransactions [test-tag]
5.759 sec TestNRTReaderWithThreads
5.694 sec TestNRTReaderWithThreads [test-tag]
5.630 sec TestBooleanMinShouldMatch
5.622 sec TestBooleanMinShouldMatch [test-tag]
5.599 sec TestIndexReaderReopen [test-tag]
5.552 sec TestNRTReaderWithThreads:testIndexing [test-tag]
5.399 sec TestIndexReaderReopen
5.254 sec TestNumericRangeQuery64
5.099 sec TestSnapshotDeletionPolicy
5.081 sec TestIndexReaderClone
5.056 sec TestIndexReaderClone [test-tag]
4.962 sec TestSnapshotDeletionPolicy [test-tag]
4.852 sec TestBasics
4.836 sec TestBasics [test-tag]
4.813 sec TestNumericRangeQuery64 [test-tag]
4.525 sec TestPerfTasksLogic
4.494 sec TestStressIndexing
4.468 sec TestBoolean2
4.465 sec TestStressIndexing [test-tag]
4.358 sec TestSnapshotDeletionPolicy:testSnapshotDeletionPolicy [test-tag]
4.323 sec TestStressIndexing:testStressIndexAndSearching [test-tag]
4.297 sec TestBoolean2 [test-tag]
4.233 sec TestCustomScoreQuery
4.172 sec TestBooleanMinShouldMatch:testRandomQueries [test-tag]
4.115 sec TestCustomScoreQuery [test-tag]
3.981 sec TestNumericRangeQuery32
3.847 sec TestNumericRangeQuery32 [test-tag]
3.810 sec TestTimeLimitingCollector:testModifyResolution [test-tag]
3.708 sec TestDeletionPolicy
3.701 sec TestDeletionPolicy [test-tag]
3.613 sec TestIndexWriterReader:testDuringAddIndexes [test-tag]
3.605 sec TestSimpleExplanations [test-tag]
3.556 sec TestIndexWriter:testAddIndexOnDiskFull [test-tag]
3.491 sec TestLockFactory
3.478 sec TestLockFactory [test-tag]
3.381 sec TestSimpleExplanations
3.297 sec TestIndexWriterReader:testDuringAddDelete [test-tag]
3.181 sec TestComplexExplanations [test-tag]
3.125 sec TestIndexWriterExceptions:testRandomExceptions [test-tag]
3.028 sec TestIndexReader:testLockObtainFailed [test-tag]
3.020 sec TestDeletionPolicy:testExpirationTimeDeletionPolicy [test-tag]
3.008 sec TestIndexWriterExceptions:testRandomExceptionsThreads [test-tag]
2.938 sec TestIndexableBinaryStringTools [test-tag]
2.922 sec TestComplexExplanations
2.815 sec TestIndexableBinaryStringTools
2.686 sec TestQueryParserWrapper
2.626 sec TestQPHelper
2.547 sec TestMultiTermConstantScore [test-tag]
2.543 sec TestMultiTermConstantScore
2.527 sec TestMultiValuedNumericRangeQuery [test-tag]
2.510 sec TestIndicesEquals
2.489 sec TestHugeRamFile [test-tag]
2.444 sec TestHugeRamFile
2.435 sec MemoryIndexTest
2.383 sec TestTermRangeFilter [test-tag]
2.379 sec TestMultiValuedNumericRangeQuery:testMultiValuedNRQ [test-tag]
2.372 sec TestBoolean2:testRandomQueries [test-tag]
2.355 sec TestSpanExplanations [test-tag]
2.332 sec TestHugeRamFile:testHugeFile [test-tag]
2.328 sec TestBackwardsCompatibility [test-tag]
2.291 sec TestBackwardsCompatibility
2.277 sec JEStoreTest
2.245 sec TestSpanExplanations
2.239 sec TestMultiValuedNumericRangeQuery
2.207 sec TestSort [test-tag]
2.125 sec TestThreadedOptimize [test-tag]
2.122 sec TestIndexReaderReopen:testThreadSafety [test-tag]
2.107 sec TestSort
2.101 sec TestQueryParser
2.082 sec TestQueryParser [test-tag]
2.057 sec TestTermRangeFilter
2.055 sec TestThreadedOptimize
2.024 sec TestIndexReader:testLastModified [test-tag]
1.981 sec TestThreadedOptimize:testThreadedOptimize [test-tag]
1.937 sec TestStressIndexing2
1.932 sec TestStressIndexing2 [test-tag]
1.831 sec TestLazyBug [test-tag]
1.817 sec TestLazyBug
1.811 sec TestPhraseQuery
1.801 sec TestPhraseQuery [test-tag]
1.762 sec TestBufferedIndexInput
1.754 sec TestBufferedIndexInput [test-tag]
1.744 sec TestConcurrentMergeScheduler
1.729 sec TestCrash
1.716 sec TestCrash [test-tag]
1.702 sec TestFieldsReader [test-tag]
1.700 sec TestIndexReaderCloneNorms
1.698 sec TestFieldsReader
1.684 sec TestIndexReaderCloneNorms [test-tag]
1.678 sec TestStringIntern
1.644 sec TestIndexableBinaryStringTools:testEncodedSortability [test-tag]
1.615 sec TestConcurrentMergeScheduler [test-tag]
1.606 sec TestStringIntern [test-tag]
1.567 sec TestCustomScoreQuery:testCustomScoreByte [test-tag]
1.560 sec TestTermAttributeImpl
1.525 sec TestFieldMaskingSpanQuery
1.496 sec TestFieldMaskingSpanQuery [test-tag]
1.486 sec TestPayloadTermQuery
1.484 sec TestToken [test-tag]
1.484 sec TestToken
1.465 sec TestPayloadTermQuery [test-tag]
1.460 sec TestStringIntern:testStringIntern [test-tag]
1.456 sec TestTermAttributeImpl [test-tag]
1.424 sec TestBufferedIndexInput:testReadBytes [test-tag]
1.414 sec TestTimeLimitingCollector:testTimeoutGreedy [test-tag]
1.366 sec TestIndexWriterReader:testAddIndexesAndDoDeletesThreads [test-tag]
1.364 sec TestSpans [test-tag]
1.361 sec TestPayloadSpans [test-tag]
1.328 sec TestSort:testStringSort [test-tag]
1.327 sec TestSpans
1.323 sec TestDisjunctionMaxQuery [test-tag]
1.300 sec TestNorms [test-tag]
1.297 sec TestToken:testGrow [test-tag]
1.288 sec TestTimeLimitingCollector:testTimeoutMultiThreaded [test-tag]
1.281 sec TestTermAttributeImpl:testGrow [test-tag]
1.280 sec TestNorms
1.269 sec TestTimeLimitingCollector:testTimeoutNotGreedy [test-tag]
1.269 sec TestDisjunctionMaxQuery
1.265 sec TestPayloadSpans
1.244 sec TestPrefixInBooleanQuery [test-tag]
1.177 sec TestLockFactory:testStressLocks [test-tag]
1.154 sec TestNorms:testNorms [test-tag]
1.150 sec TestFieldsReader:testLazyPerformance [test-tag]
1.144 sec TestFieldCacheRangeFilter
1.141 sec TestPrefixInBooleanQuery
1.124 sec TestIndexableBinaryStringTools:testRandomBinaryRoundTrip [test-tag]
1.111 sec TestFieldCacheRangeFilter [test-tag]
1.105 sec TestIndexWriterMergePolicy [test-tag]
1.098 sec TestIndexReaderClone:testParallelReader [test-tag]
1.089 sec TestIndexWriter:testNoWaitClose [test-tag]
1.070 sec TestIndexWriterMergePolicy
1.064 sec TestIndexReaderClone:testCloneWriteToClone [test-tag]
1.060 sec TestCustomScoreQuery:testCustomScoreFloat [test-tag]
1.057 sec TestIndexWriter:testThreadInterruptDeadlock [test-tag]
1.049 sec TestIndexReaderClone:testMixedReaders [test-tag]
1.047 sec TestThreadSafe
1.039 sec TestIndexReaderCloneNorms:testNormsRefCounting [test-tag]
1.023 sec TestThreadSafe [test-tag]
1.012 sec TestIndexReader:testLock [test-tag]
1.012 sec TestFieldCacheSanityChecker [test-tag]
1.011 sec TestIndexReaderClone:testSegmentReaderDelDocsReferenceCounting [test-tag]
1.010 sec TestLockFactory:testDefaultRAMDirectory [test-tag]
1.009 sec TestLockFactory:testStressLocksNativeFSLockFactory [test-tag]
1.005 sec TestFieldCacheSanityChecker
1.003 sec TestStressIndexing2:testMultiConfig [test-tag]
{code}
> Speed up junit tests
> --------------------
>
> Key: LUCENE-1844
> URL: https://issues.apache.org/jira/browse/LUCENE-1844
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Mark Miller
> Attachments: FastCnstScoreQTest.patch, hi_junit_test_runtimes.png, LUCENE-1844-Junit3.patch, LUCENE-1844.patch, LUCENE-1844.patch
>
>
> As Lucene grows, so does the number of JUnit tests. This is obviously a good thing, but it comes with longer and longer test times. Now that we also run back compat tests in a standard test run, this problem is essentially doubled.
> There are some ways this may get better, including running parallel tests. You will need the hardware to fully take advantage, but it should be a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the beginnings of something we might be able to count on soon. 4.6 was buggy, and 4.7 still doesn't come with nice ant integration. Parallel tests will come though.
> Beyond parallel testing, I think we also need to concentrate on keeping our tests lean. We don't want to sacrifice coverage or quality, but I'm sure there is plenty of fat to skim.
> I've started making a list of some of the longer tests - I think with some work we can make our tests much faster - and then with parallelization, I think we could see some really great gains.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene
|
