ITADN

segfault crash in knn::BinaryQuantizer_c::Quantize4Bit when threads>1 used in query (not thread safe code)

#4521Openpopalot2 创建于 2026-04-26
bug
P
popalot2commented
### Bug Description: intermittent segfault crash in knn::BinaryQuantizer_c::Quantize4Bit when query uses [threads>1] attached reproducible case, if crash doesn't happen restart manticore server and try again - just running the queries (don't reload the table again) - sometimes a few restarts are required. Looks like bug is caused by SpanResizeable_T<float> m_dVecMinusCentroid; SpanResizeable_T<uint8_t> m_dQuantized; being defined as private members of BinaryQuantizer_c, which is reused between different threads. After I moved m_dVecMinusCentroid and m_dQuantized into the method Quantize4Bit - on the stack - the crash stopped. Not sure about performance repercussions of moving these to stack and if it's a good fix or not. ``` to reproduce, create and populate the table with python test_manticore_crash.py --populate --data-file vector_data.tsv then run python test_manticore_crash.py --query --query-file bench_static_mrl_v1_d256_q1bit_c224o80.vec to induce the crash, sometimes multiple runs are required. ``` seems the tickets https://github.com/manticoresoftware/manticoresearch/issues/3854 https://github.com/manticoresoftware/manticoresearch/issues/4212 are the same bug. (gdb) where #0 0x00007fbcfbb4e1fd in knn::BinaryQuantizer_c::Quantize4Bit (this=0x7fb8dd000760, dVector=..., dCentroid=std::vector of length 256, capacity 256 = {...}, dResult=std::vector of length 0, capacity 0) at /var/lib/manticore/src/columnar/knn/quantizer.cpp:603 #1 0x00007fbcfba526f5 in knn::HNSWIndex_c::Search (this=0x7fb8dd0007e0, dResults=std::vector of length 0, capacity 0, dData=..., iResults=60, iEf=0, dQuantized=std::vector of length 0, capacity 0, pDistanceComputations=0x0, pFilter=0x0, ePolicy=knn::HNSWTerminationPolicy_e::QUANTILE) at /var/lib/manticore/src/columnar/knn/knn.cpp:362 #2 0x00007fbcfbb4a0a3 in knn::RowidIteratorKNN_c::RowidIteratorKNN_c (this=0x7fbcf000f060, tIndex=..., dData=..., iResults=<optimized out>, iEf=<optimized out>, bCollectMetrics=<optimized out>, pFilter=0x0, ePolicy=knn::HNSWTerminationPolicy_e::QUANTILE) at /var/lib/manticore/src/columnar/knn/iterator.cpp:92 #3 0x00007fbcfbb4a174 in knn::CreateIterator (tIndex=..., dData=..., iResults=60, iEf=0, bCollectMetrics=<optimized out>, pFilter=0x0, ePolicy=knn::HNSWTerminationPolicy_e::QUANTILE) at /var/lib/manticore/src/columnar/knn/iterator.cpp:143 #4 0x0000000001070b82 in CreateKNNIterator (pKNN=pKNN@entry=0x7fb8dc53d070, tQuery=..., tIndexSchema=..., tSorterSchema=..., pFilter=pFilter@entry=0x0, ePolicy=ePolicy@entry=knn::HNSWTerminationPolicy_e::QUANTILE, pProfile=0x0, sError=...) at /var/lib/manticore/src/manticoresearch/src/knnmisc.cpp:794 #5 0x00000000010719fc in CreateKNNIterators (pKNN=0x7fb8dc53d070, tQuery=..., tIndexSchema=..., tSorterSchema=..., pFilter=pFilter@entry=0x0, ePolicy=knn::HNSWTerminationPolicy_e::QUANTILE, pProfile=0x0, bError=@0x7fb9640f8f0f: false, sError=...) at /var/lib/manticore/src/manticoresearch/src/knnmisc.cpp:827 #6 0x0000000000d5a441 in CSphIndex_VLN::SpawnIterators (this=this@entry=0x7fb8dc53c7a0, tQuery=..., dFilters=..., dJsonSITransforms=..., tCtx=..., tFlx=..., tMaxSorterSchema=..., dSorterSchemas=..., pModifiedMatchSchema=std::unique_ptr<ISphSchema> = {...}, tMeta=..., iCutoff=-1, iThreads=6, dModifiedFilters=..., bUseSICache=false, pRanker=0x0) at /usr/include/c++/11/bits/unique_ptr.h:173 #7 0x0000000000d5f396 in CSphIndex_VLN::MultiScan (this=0x7fb8dc53c7a0, tResult=..., tQuery=..., dSorters=..., tArgs=..., tmMaxTimer=0) at /var/lib/manticore/src/manticoresearch/src/sphinx.cpp:8878 #8 0x0000000000d70d6c in operator() (tMultiArgs=..., dLocalSorters=..., tQuery=..., tChunkResult=..., __closure=<optimized out>) at /var/lib/manticore/src/manticoresearch/src/sphinx.cpp:11462 #9 operator() (__closure=0x7fb9640da670) at /var/lib/manticore/src/manticoresearch/src/sphinx.cpp:11322 #10 0x0000000001002426 in std::function<void()>::operator() (this=0x7fb964018f98) at /usr/include/c++/11/bits/std_function.h:590 #11 operator() (__closure=0x7fb964018f90) at /var/lib/manticore/src/manticoresearch/src/task_info.cpp:158 #12 std::__invoke_impl<void, myinfo::OwnMini(Threads::Handler)::<lambda()>&> (__f=...) at /usr/include/c++/11/bits/invoke.h:61 #13 std::__invoke_r<void, myinfo::OwnMini(Threads::Handler)::<lambda()>&> (__fn=...) at /usr/include/c++/11/bits/invoke.h:111 #14 std::_Function_handler<void(), myinfo::OwnMini(Threads::Handler)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/11/bits/std_function.h:290 #15 0x0000000001ce513c in std::function<void()>::operator() (this=0x7fb9640da760) at /usr/include/c++/11/bits/std_function.h:590 #16 Threads::CoRoutine_c::WorkerLowest (tCtx=<optimized out>, this=0x7fb9640da758) at /var/lib/manticore/src/manticoresearch/src/coroutine.cpp:64 #17 Threads::CoRoutine_c::CreateContext(std::function<void ()>, std::pair<boost::context::stack_context, Threads::StackFlavour_E>)::{lambda(boost::context::detail::transfer_t)#1}::operator()(boost::context::detail::transfer_t) const (pT=..., __closure=0x0) at /var/lib/manticore/src/manticoresearch/src/coroutine.cpp:80 #18 Threads::CoRoutine_c::CreateContext(std::function<void ()>, std::pair<boost::context::stack_context, Threads::StackFlavour_E>)::{lambda(boost::context::detail::transfer_t)#1}::_FUN(boost::context::detail::transfer_t) () at /var/lib/manticore/src/manticoresearch/src/coroutine.cpp:81 #19 0x000000000210ef7f in make_fcontext () #20 0x0000000000000000 in ?? () ``` ### Manticore Search Version: Manticore 25.0.0 ce3c27828@26032712 (columnar 13.0.0 e60b083@26032708) (secondary 13.0.0 e60b083@26032708) (knn 13.0.0 e60b083@26032708) (embeddings 1.1.1 e60b083@26032708) ### Operating System Version: Almalinux 9 ### Have you tried the latest development version? Yes ### Internal Checklist: <sup>To be completed by the assignee. Check off tasks that have been completed or are not applicable.</sup> <details> - [ ] Implementation completed - [ ] Tests developed - [ ] Documentation updated - [ ] Documentation reviewed </details>
1 条评论