ITADN

[BUG] govaluate: `slice bounds out of range` panic in `readUntilFalse` on invalid UTF-8 input

#7462Opengnuletik 创建于 2026-06-12
Type: Bug
G
gnuletikcommented
### Is there an existing issue for this? - [x] I have searched the existing issues. ### Current Behavior Nuclei panics and the whole process aborts while building an HTTP request, when the evaluated expression contains invalid UTF-8 bytes: ``` panic: runtime error: slice bounds out of range [:418] with length 404 ``` The panic originates in `github.com/projectdiscovery/govaluate` (the expression engine), called from `expressions.Evaluate` in `http.(*requestGenerator).Make`. Because the expression is built from request data that can include values extracted from a scanned target's responses (`operators.MakeDynamicValuesCallback`), a target can supply the invalid bytes that trigger the crash. `expressions.evaluate` already handles a govaluate *error* gracefully, but this is a panic, so it bypasses that error handling and propagates up a worker goroutine (`core.(*Engine).executeTemplateWithTargets`), killing the engine. ### Expected Behavior Invalid UTF-8 in an expression should not panic. `govaluate.NewEvaluableExpressionWithFunctions` should return an `error` (which nuclei already handles) — or normalize the input — instead of performing an out-of-bounds slice. ### Steps To Reproduce Minimal, reproducible directly against govaluate (no nuclei needed), using the exact version nuclei v3.9.0 pins (`v0.0.0-20260504230327-80320480bb6e`): ```go package main import "github.com/projectdiscovery/govaluate" func main() { // A quoted string containing invalid UTF-8 bytes, followed by a trailing // ASCII token. The invalid bytes inflate the lexer's byte cursor; the next // token's reuseString fast-path then slices the source string out of range. body := make([]byte, 0, 8) for i := 0; i < 7; i++ { body = append(body, 0xff) } expr := "\"" + string(body) + "\"+aaaaaaaaaa" _, _ = govaluate.NewEvaluableExpressionWithFunctions(expr, nil) } ``` ``` go run . # panic: runtime error: slice bounds out of range [:24] with length 20 ``` In nuclei, the same crash happens when an HTTP template interpolates a dynamic value (extracted from a prior response) that contains non-UTF-8 bytes into a request expression, and that expression is compiled at `build_request.go:239`. ### Relevant log output ```shell panic: runtime error: slice bounds out of range [:418] with length 404 goroutine [running]: github.com/projectdiscovery/govaluate.readUntilFalse(...) github.com/projectdiscovery/govaluate/parsing.go:384 github.com/projectdiscovery/govaluate.readToken(...) github.com/projectdiscovery/govaluate/parsing.go:229 github.com/projectdiscovery/govaluate.parseTokens({... 0x194 ...}, ...) github.com/projectdiscovery/govaluate/parsing.go:37 github.com/projectdiscovery/govaluate.NewEvaluableExpressionWithFunctions(...) github.com/projectdiscovery/govaluate/EvaluableExpression.go:100 github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions.evaluate(...) github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions/expressions.go:61 github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions.Evaluate(...) github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions/expressions.go:31 github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*requestGenerator).Make(...) github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/build_request.go:239 github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1(...) github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/request.go:542 github.com/projectdiscovery/nuclei/v3/pkg/operators.MakeDynamicValuesCallback(...) github.com/projectdiscovery/nuclei/v3/pkg/operators/operators.go:143 ... created by github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets github.com/projectdiscovery/nuclei/v3/pkg/core/executors.go:97 `parseTokens({..., 0x194}, ...)`: `0x194` = 404 = the expression's byte length. The slice end `418` = 404 + 14 = 404 + (2 × 7): seven invalid UTF-8 bytes earlier in the expression, each over-counting the byte cursor by 2. ``` ### Environment ```markdown - OS: Linux / amd64 - Nuclei: v3.9.0 - govaluate: v0.0.0-20260504230327-80320480bb6e (and reproduced on govaluate `master`) - Go: 1.26 ``` ### Anything else? **Root cause (in govaluate).** `lexerStream` (`lexerStream.go`) keeps two cursors over the expression: `position` (rune index into `source []rune`) and `strPosition` (a *byte* offset into `sourceString string`). `re adCharacter` advances the byte cursor by the `RuneLen` of the *decoded* rune: ```go func (this *lexerStream) readCharacter() rune { character := this.source[this.position] this.position += 1 this.strPosition += utf8.RuneLen(character) // RuneLen of the decoded rune return character } ``` `newLexerStream` decodes the source with `for _, character := range source`. On invalid UTF-8, `range` yields `utf8.RuneError` (U+FFFD) and advances 1 byte, but `utf8.RuneLen(utf8.RuneError) == 3`. So every invalid byte pushes `strPosition` 2 bytes ahead of the true byte offset. (Valid multi-byte runes stay aligned.) `readUntilFalse` then has a `reuseString` fast-path that slices the raw source bytes (`parsing.go:384`): ```go if reuseString { ret := stream.sourceString[startPosition:stream.strPosition] // strPosition can exceed len(sourceString) ... } ``` The guard `if character > utf8.RuneSelf { reuseString = false }` only disables the fast-path for the *current* token; it does not undo the global `strPosition` drift the invalid bytes already caused. So: an earlier token containing invalid UTF-8 takes the safe buffered path (no panic) but inflates `strPosition`; a later all-ASCII token takes the `reuseString` path and slices past `len(sourceString)`. **Suggested fix (govaluate):** advance `strPosition` by the bytes actually consumed for the rune (1 for a `RuneError` from an invalid byte), so the byte cursor stays aligned with the source — or reject/normalize non -UTF-8 input in `NewEvaluableExpressionWithFunctions` (e.g. `strings.ToValidUTF8`). **Suggested hardening (nuclei):** sanitize request data / extracted dynamic values with `strings.ToValidUTF8` before `expressions.Evaluate`, so target-controlled bytes never reach the lexer in a malformed state.
2 条评论