Memory leak with unbounded consumption causing rapid system crash

I've discovered a **critical memory leak** bug that results in unbounded memory consumption (I'm talking 10+GB in a matter of seconds on my system), which soon balloons out of control and crashes my entire operating system. Summary: - The `wrap` highlighter with the `-marker` or `-indent` option enabled has an interaction with the `replace-ranges` highlighter, which causes the bug. - It can be reproduced consistently with a single `-n -e '...'` command. In my investigation, the following conditions trigger the memory leak: - There must be a `wrap` highlighter: - The `-marker` switch must be provided. The arg can be any character. - **OR** the `-indent` switch. - Either switch will cause the bug. - There must be some `range-specs` option: - There must be at least one tuple which meets these criteria: - Uses the syntax `a.b+length|string` - `a` and `b` can be anything - `length` must be `0` - There must a `replace-ranges` highlighter which uses the above `range-specs` option. - The `replace-ranges` and `wrap` highlighters must be active on the same window at the same time. ## reproducing > [!WARNING] > This will cause unbounded memory growth **that will soon crash your computer**. I recommend opening a second terminal and being ready to kill `kak` at a moment's notice. It may be a good idea to type `pkill -f kak` and be prepared to hit enter on it. The following command will trigger the bug before Kakoune has even drawn the screen: ```sh kak -n -e ' add-highlighter window/ wrap -marker "x" declare-option range-specs broken set-option window broken %val{timestamp} "1.1+0|: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" add-highlighter window/ replace-ranges broken ' ``` ## discussion > [!NOTE] > Terminology reminder: `range-specs` use the syntax `a.b,c.d|string` or `a.b+length|string`. I believe that, the problem is to do with the length of the `string` in a `range-spec`. Specifically, if it's long enough to be wider than the screen -- because that's what triggers the `wrap` highlighter. Observe that, in the repro command, replacing the long `AAAA...` string with a single character prevents the crash. In the `wrap` highlighter, it's the `-marker` option that seems to cause the bug. Omitting this switch prevents the crash. `-indent` _also_ causes a memory leak. Combinations of other switches don't seem to affect the behavior. I tried different characters for `-marker`'s value, including ASCII and some Unicode chars, but it doesn't seem to affect behavior --- as long as some `-marker` is set, the crash happens. The order of setting the highlighters doesn't seem to matter either. For example, I tried setting the `replace-ranges` one first, and _then_ `wrap`, and the crash still happened. It seems to happen the moment both highlighters are active at once. ## extra information - I'm using Kakoune built from source commit `d82775b076cffae3ee9af7201a70adce9e0a6851`. - I ran Kakoune (built with debug symbols) through `heaptrack` and got this trace: ``` main in kak Kakoune::run_server in kak Kakoune::ClientManager::redraw_clients() const in kak Kakoune::Client::redraw_ifn() in kak Kakoune::Window::update_display_buffer(Kakoune::Context const&) in kak Kakoune::Highlighters::highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak Kakoune::Highlighters::highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak Kakoune::Highlighter::highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak Kakoune::HighlighterGroup::do_highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak Kakoune::Highlighter::highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak Kakoune::WrapHighlighter::do_highlight(Kakoune::HighlightContext, Kakoune::DisplayBuffer&, Kakoune::Range<Kakoune::BufferCoord>) in kak std::vector<Kakoune::DisplayLine, Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8> >::insert(__gnu_cxx::__normal_iterator<Kakoune::DisplayLine const*, std::vector<Kakoune::DisplayLine, Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8> > >, Kakoune::DisplayLine const&) in kak std::vector<Kakoune::DisplayLine, Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8> >::_Temporary_value::_Temporary_value<Kakoune::DisplayLine const&>(std::vector<Kakoune::DisplayLine, Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8> >*, Kakoune::DisplayLine const&) in kak void std::allocator_traits<Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8> >::construct<Kakoune::DisplayLine, Kakoune::DisplayLine const&>(Kakoune::Allocator<Kakoune::DisplayLine, (Kakoune::MemoryDomain)8>&, Kakoune::DisplayLine*, Kakoune::DisplayLine const&) in kak void std::_Construct<Kakoune::DisplayLine, Kakoune::DisplayLine const&>(Kakoune::DisplayLine*, Kakoune::DisplayLine const&) in kak Kakoune::DisplayLine::DisplayLine(Kakoune::DisplayLine const&) in kak std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> >::vector(std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> > const&) in kak Kakoune::DisplayAtom* std::__uninitialized_copy_a<__gnu_cxx::__normal_iterator<Kakoune::DisplayAtom const*, std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> > >, __gnu_cxx::__normal_iterator<Kakoune::DisplayAtom const*, std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> > >, Kakoune::DisplayAtom*, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> >(__gnu_cxx::__normal_iterator<Kakoune::DisplayAtom const*, std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> > >, __gnu_cxx::__normal_iterator<Kakoune::DisplayAtom const*, std::vector<Kakoune::DisplayAtom, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> > >, Kakoune::DisplayAtom*, Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8>&) in kak void std::allocator_traits<Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8> >::construct<Kakoune::DisplayAtom, Kakoune::DisplayAtom const&>(Kakoune::Allocator<Kakoune::DisplayAtom, (Kakoune::MemoryDomain)8>&, Kakoune::DisplayAtom*, Kakoune::DisplayAtom const&) in kak void std::_Construct<Kakoune::DisplayAtom, Kakoune::DisplayAtom const&>(Kakoune::DisplayAtom*, Kakoune::DisplayAtom const&) in kak Kakoune::DisplayAtom::DisplayAtom(Kakoune::DisplayAtom const&) in kak Kakoune::String::String(Kakoune::String const&) in kak Kakoune::String::Data::Data(Kakoune::String::Data const&) in kak Kakoune::String::Data::Data(char const*, unsigned long) in kak Kakoune::String::Data::Data(char const*, unsigned long, unsigned long) in kak Kakoune::String::Data::Data(char const*, unsigned long, unsigned long) in kak Kakoune::Allocator<char, (Kakoune::MemoryDomain)1>::allocate(unsigned long) in kak ```