[BUG] govaluate: `slice bounds out of range` panic in `readUntilFalse` on invalid UTF-8 input
Type: Bug
### Is there an existing issue for this?
- [x] I have searched the existing issues.
### Current Behavior
Nuclei panics and the whole process aborts while building an HTTP request, when the evaluated expression contains invalid UTF-8 bytes:
```
panic: runtime error: slice bounds out of range [:418] with length 404
```
The panic originates in `github.com/projectdiscovery/govaluate` (the expression engine), called from `expressions.Evaluate` in `http.(*requestGenerator).Make`. Because the expression is built from request data that can include values extracted from a scanned target's responses (`operators.MakeDynamicValuesCallback`), a target can supply the invalid bytes that trigger the crash.
`expressions.evaluate` already handles a govaluate *error* gracefully, but this is a panic, so it bypasses that error handling and propagates up a worker goroutine (`core.(*Engine).executeTemplateWithTargets`), killing the engine.
### Expected Behavior
Invalid UTF-8 in an expression should not panic. `govaluate.NewEvaluableExpressionWithFunctions` should return an `error` (which nuclei already handles) — or normalize the input — instead of performing an out-of-bounds slice.
### Steps To Reproduce
Minimal, reproducible directly against govaluate (no nuclei needed), using the exact version nuclei v3.9.0 pins (`v0.0.0-20260504230327-80320480bb6e`):
```go
package main
import "github.com/projectdiscovery/govaluate"
func main() {
// A quoted string containing invalid UTF-8 bytes, followed by a trailing
// ASCII token. The invalid bytes inflate the lexer's byte cursor; the next
// token's reuseString fast-path then slices the source string out of range.
body := make([]byte, 0, 8)
for i := 0; i < 7; i++ {
body = append(body, 0xff)
}
expr := "\"" + string(body) + "\"+aaaaaaaaaa"
_, _ = govaluate.NewEvaluableExpressionWithFunctions(expr, nil)
}
```
```
go run .
# panic: runtime error: slice bounds out of range [:24] with length 20
```
In nuclei, the same crash happens when an HTTP template interpolates a dynamic value (extracted from a prior response) that contains non-UTF-8 bytes into a request expression, and that expression is compiled at `build_request.go:239`.
### Relevant log output
```shell
panic: runtime error: slice bounds out of range [:418] with length 404
goroutine [running]:
github.com/projectdiscovery/govaluate.readUntilFalse(...)
github.com/projectdiscovery/govaluate/parsing.go:384
github.com/projectdiscovery/govaluate.readToken(...)
github.com/projectdiscovery/govaluate/parsing.go:229
github.com/projectdiscovery/govaluate.parseTokens({... 0x194 ...}, ...)
github.com/projectdiscovery/govaluate/parsing.go:37
github.com/projectdiscovery/govaluate.NewEvaluableExpressionWithFunctions(...)
github.com/projectdiscovery/govaluate/EvaluableExpression.go:100
github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions.evaluate(...)
github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions/expressions.go:61
github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions.Evaluate(...)
github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/expressions/expressions.go:31
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*requestGenerator).Make(...)
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/build_request.go:239
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1(...)
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/request.go:542
github.com/projectdiscovery/nuclei/v3/pkg/operators.MakeDynamicValuesCallback(...)
github.com/projectdiscovery/nuclei/v3/pkg/operators/operators.go:143
...
created by github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets
github.com/projectdiscovery/nuclei/v3/pkg/core/executors.go:97
`parseTokens({..., 0x194}, ...)`: `0x194` = 404 = the expression's byte length. The slice end `418` = 404 + 14 = 404 + (2 × 7): seven invalid UTF-8 bytes earlier in the expression, each over-counting the byte cursor by 2.
```
### Environment
```markdown
- OS: Linux / amd64
- Nuclei: v3.9.0
- govaluate: v0.0.0-20260504230327-80320480bb6e (and reproduced on govaluate `master`)
- Go: 1.26
```
### Anything else?
**Root cause (in govaluate).** `lexerStream` (`lexerStream.go`) keeps two cursors over the expression: `position` (rune index into `source []rune`) and `strPosition` (a *byte* offset into `sourceString string`). `re
adCharacter` advances the byte cursor by the `RuneLen` of the *decoded* rune:
```go
func (this *lexerStream) readCharacter() rune {
character := this.source[this.position]
this.position += 1
this.strPosition += utf8.RuneLen(character) // RuneLen of the decoded rune
return character
}
```
`newLexerStream` decodes the source with `for _, character := range source`. On invalid UTF-8, `range` yields `utf8.RuneError` (U+FFFD) and advances 1 byte, but `utf8.RuneLen(utf8.RuneError) == 3`. So every invalid
byte pushes `strPosition` 2 bytes ahead of the true byte offset. (Valid multi-byte runes stay aligned.)
`readUntilFalse` then has a `reuseString` fast-path that slices the raw source bytes (`parsing.go:384`):
```go
if reuseString {
ret := stream.sourceString[startPosition:stream.strPosition] // strPosition can exceed len(sourceString)
...
}
```
The guard `if character > utf8.RuneSelf { reuseString = false }` only disables the fast-path for the *current* token; it does not undo the global `strPosition` drift the invalid bytes already caused. So: an earlier
token containing invalid UTF-8 takes the safe buffered path (no panic) but inflates `strPosition`; a later all-ASCII token takes the `reuseString` path and slices past `len(sourceString)`.
**Suggested fix (govaluate):** advance `strPosition` by the bytes actually consumed for the rune (1 for a `RuneError` from an invalid byte), so the byte cursor stays aligned with the source — or reject/normalize non
-UTF-8 input in `NewEvaluableExpressionWithFunctions` (e.g. `strings.ToValidUTF8`).
**Suggested hardening (nuclei):** sanitize request data / extracted dynamic values with `strings.ToValidUTF8` before `expressions.Evaluate`, so target-controlled bytes never reach the lexer in a malformed state.
2 条评论