Tool call escapes the JSON string
_The following example tries to provide full context demonstrating the issues. The bug not always shows due to the unexpected nature of AI agent._
Reproduces a bug where qwen3-coder:30b on Ollama wraps every tool string
argument in JSON array brackets:
Expected: write_file(path = "SKILL.md", content = "hello")
Actual: write_file(path = '["SKILL.md"]', content = '["hello"]')
This makes all tool calls fail when the tool validates its inputs.
Root cause: The MODEL generates malformed tool call arguments.
Ollama's OpenAI-compatible API faithfully passes them through,
and ellmer's ProviderOpenAICompatible does not coerce array→string.
Question: should ellmer unwrap single-element arrays when the
tool schema declares type_string()? Although `'["SKILL.md"]'` is also a string, the model may go crazy into escaping strings (sometimes `'\\\\[\\\\\\\\"SKILL.md\\\\\\\\"\\\\]'` after several failed attempts.)
Prerequisites:
- Ollama running at the URL below
- Model: ollama pull qwen3-coder:30b
``` r
# ====================================================================
# test-ollama-array-wrap-bug.R
# ====================================================================
library(ellmer)
ollama_url <- "http://127.0.0.1:11434"
chat <- chat_ollama(
base_url = ollama_url,
model = "qwen3-coder:30b",
system_prompt = paste(
"You are a file-writing assistant.",
"When asked to create a file, use the write_file tool.",
"Do NOT ask questions. Just write the file.",
"IMPORTANT! DO NOT CALL TOOLS IN PARALLEL; ALWAYS CALL THEM IN SEQUENTIAL ORDER: THE PLATFORM YOU ARE RUNNING ON CRASHES/SEGFAULTS WITH PARALLEL TOOL CALLS.",
"/nothink"
)
)
# --- Tool that logs what it receives ---
received_args <- list()
write_file_tool <- tool(
function(path, content) {
received_args$path <<- path
received_args$content <<- content
cat("\n=== write_file CALLED ===\n")
cat(" path type: ", typeof(path), "\n")
cat(" path value: ", repr::repr(path), "\n")
if (grepl("^\\[", path)) {
stop(sprintf("BUG: path wrapped in array brackets: %s", path))
}
paste0("File written: ", path)
},
name = "write_file",
description = "Write content to a file at the given path. Use RELATIVE paths only.",
arguments = list(
path = type_string("Relative file path to write to.", required = TRUE),
content = type_string("The text content to write.", required = TRUE)
)
)
read_file_tool <- tool(
function(path) {
cat("\n=== read_file CALLED ===\n")
cat(" path value: ", repr::repr(path), "\n")
if (grepl("^\\[", path)) {
stop(sprintf("BUG: path wrapped in array brackets: %s", path))
}
"name: example\ndescription: An example skill."
},
name = "read_file",
description = "Read a file at the given relative path.",
arguments = list(
path = type_string("Relative file path to read.", required = TRUE)
)
)
walk_dir_tool <- tool(
function(path) {
cat("\n=== walk_directory CALLED ===\n")
cat(" path value: ", repr::repr(path), "\n")
'{"directories": ["scripts", "references"], "files": ["SKILL.md"]}'
},
name = "walk_directory",
description = "List files and directories at the given relative path.",
arguments = list(
path = type_string(
"Relative directory path. Use '.' for the root.",
required = FALSE
)
)
)
chat$register_tool(write_file_tool)
chat$register_tool(read_file_tool)
chat$register_tool(walk_dir_tool)
# --- Trigger the bug with a multi-step prompt ---
cat("Sending request...\n\n")
#> Sending request...
result <- tryCatch(
chat$chat(paste(
"Create a skill directory structure with these files (you MUST call the tools in sequential order):",
"1. First, list the current directory with (walk_directory())",
"2. Once 1 finishes, Write SKILL.md with YAML frontmatter (name: import-bids,",
" description: Import BIDS data into RAVE) and a markdown body (write_file())",
"3. Wait for 2 to finish, then write scripts/import-bids.R with a simple R script (write_file())",
"4. Wait for 3 to finish, then write references/parameters.md with parameter docs (write_file())",
"5. Finally after 1-4, list the directory again to confirm (walk_directory())"
), echo = "all"),
error = function(e) {
cat("\nERROR:", conditionMessage(e), "\n")
traceback(e)
e
}
)
#> > Create a skill directory structure with these files (you MUST call the tools
#> > in sequential order): 1. First, list the current directory with
#> > (walk_directory()) 2. Once 1 finishes, Write SKILL.md with YAML frontmatter
#> > (name: import-bids, description: Import BIDS data into RAVE) and a
#> > markdown body (write_file()) 3. Wait for 2 to finish, then write
#> > scripts/import-bids.R with a simple R script (write_file()) 4. Wait for 3 to
#> > finish, then write references/parameters.md with parameter docs
#> > (write_file()) 5. Finally after 1-4, list the directory again to confirm
#> > (walk_directory())
#> < I'll create a skill directory structure with the specified files using the
#> < tools in sequential order.
#> <
#> < First, let me list the current directory:
#> < [tool request (call_ssk0e4np)]: walk_directory(path = ".")
#>
#> === walk_directory CALLED ===
#> path value: [1] "."
#> [tool result (call_ssk0e4np)]: {"directories": ["scripts", "references"], "files": ["SKILL.md"]}
#> > [tool result (call_ssk0e4np)]: {"directories": ["scripts", "references"],
#> > "files": ["SKILL.md"]}
#> < I see that there are already some directories and files in the current
#> < location. Now I'll create the SKILL.md file with the YAML frontmatter and
#> < markdown body.
#> < [tool request (call_3xrlpni6)]: write_file(path = "SKILL.md", content =
#> < "---\nname: import-bids\ndescription: Import BIDS data into RAVE\n---\n\n#
#> < Import BIDS Data into RAVE\n\nThis skill imports BIDS (Brain Imaging Data
#> < Structure) formatted data into the RAVE platform for further analysis.")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "SKILL.md"
#> [tool result (call_3xrlpni6)]: File written: SKILL.md
#> > [tool result (call_3xrlpni6)]: File written: SKILL.md
#> < Good! Now I'll create the R script file in the scripts directory.
#> < [tool request (call_2j4h1xil)]: write_file(path =
#> < "[\"scripts/import-bids.R\"]", content = "[\"#!/usr/bin/env Rscript\\n\\n#
#> < Import BIDS data into RAVE\\n\\n# Load required libraries\\nlibrary(bids) #
#> < or appropriate BIDS library\\n\\n# Main function to import BIDS
#> < data\\nimport_bids_data <- function(bids_dir, output_dir) {\\n # Validate
#> < input directory\\n if (!dir.exists(bids_dir)) {\\n stop(\\\"BIDS
#> < directory does not exist: \\\", bids_dir)\\n }\\n \\n # Import BIDS
#> < data\\n # This is a placeholder - actual implementation would depend on
#> < specific requirements\\n cat(\\\"Importing BIDS data from:\\\", bids_dir,
#> < \\\"\\\\n\\\")\\n \\n # Process and prepare data for RAVE\\n # ...
#> < implementation details ...\\n \\n cat(\\\"Data successfully imported
#> < to:\\\", output_dir, \\\"\\\\n\\\")\\n}\\n\\n# Example usage\\n#
#> < import_bids_data(\\\"/path/to/bids\\\", \\\"/path/to/output\\\")\"]")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "[\"scripts/import-bids.R\"]"
#> [tool result (call_2j4h1xil)]: Error: BUG: path wrapped in array brackets: ["scripts/import-bids.R"]
#> > [tool result (call_2j4h1xil)]: Error: BUG: path wrapped in array brackets:
#> > ["scripts/import-bids.R"]
#> < I apologize for the error. Let me try again with the correct path format for
#> < the R script file.
#> < [tool request (call_jg6iukeo)]: write_file(path = "scripts/import-bids.R",
#> < content = "#!/usr/bin/env Rscript\n\n# Import BIDS data into RAVE\n\n# Load
#> < required libraries\nlibrary(bids) # or appropriate BIDS library\n\n# Main
#> < function to import BIDS data\nimport_bids_data <- function(bids_dir,
#> < output_dir) {\n # Validate input directory\n if (!dir.exists(bids_dir)) {\n
#> < stop(\"BIDS directory does not exist: \", bids_dir)\n }\n \n # Import BIDS
#> < data\n # This is a placeholder - actual implementation would depend on
#> < specific requirements\n cat(\"Importing BIDS data from:\", bids_dir,
#> < \"\\n\")\n \n # Process and prepare data for RAVE\n # ... implementation
#> < details ...\n \n cat(\"Data successfully imported to:\", output_dir,
#> < \"\\n\")\n}\n\n# Example usage\n# import_bids_data(\"/path/to/bids\",
#> < \"/path/to/output\")")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "scripts/import-bids.R"
#> [tool result (call_jg6iukeo)]: File written: scripts/import-bids.R
#> > [tool result (call_jg6iukeo)]: File written: scripts/import-bids.R
#> < Great! Now I'll create the parameter documentation file in the references
#> < directory.
#> < [tool request (call_wddme8qv)]: write_file(path =
#> < "[\"references/parameters.md\"]", content = "[\"# Parameters for import-bids
#> < Skill\\n\\n## Required Parameters\\n\\n- `bids_directory` (string): Path to
#> < the BIDS dataset directory\\n- `output_directory` (string): Path where the
#> < imported data should be stored\\n\\n## Optional Parameters\\n\\n-
#> < `subject_list` (string, comma-separated): List of specific subjects to
#> < import\\n- `modalities` (string, comma-separated): List of specific
#> < modalities to import\\n- `overwrite` (boolean): Whether to overwrite existing
#> < files in the output directory\\n- `verbose` (boolean): Whether to print
#> < detailed logging information\"]")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "[\"references/parameters.md\"]"
#> [tool result (call_wddme8qv)]: Error: BUG: path wrapped in array brackets: ["references/parameters.md"]
#> > [tool result (call_wddme8qv)]: Error: BUG: path wrapped in array brackets:
#> > ["references/parameters.md"]
#> < Let me try again with the correct path format for the parameters.md file.
#> < [tool request (call_lk3ok0tj)]: write_file(path =
#> < "[\"references/parameters.md\"]", content = "[\"# Parameters for import-bids
#> < Skill\\n\\n## Required Parameters\\n\\n- `bids_directory` (string): Path to
#> < the BIDS dataset directory\\n- `output_directory` (string): Path where the
#> < imported data should be stored\\n\\n## Optional Parameters\\n\\n-
#> < `subject_list` (string, comma-separated): List of specific subjects to
#> < import\\n- `modalities` (string, comma-separated): List of specific
#> < modalities to import\\n- `overwrite` (boolean): Whether to overwrite existing
#> < files in the output directory\\n- `verbose` (boolean): Whether to print
#> < detailed logging information\"]")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "[\"references/parameters.md\"]"
#> [tool result (call_lk3ok0tj)]: Error: BUG: path wrapped in array brackets: ["references/parameters.md"]
#> > [tool result (call_lk3ok0tj)]: Error: BUG: path wrapped in array brackets:
#> > ["references/parameters.md"]
#> < I apologize for the repeated errors. Let me try one more time with the
#> < correct path format:
#> < [tool request (call_96hd33o2)]: write_file(path = "references/parameters.md",
#> < content = "[\"# Parameters for import-bids Skill\n\n## Required
#> < Parameters\n\n- `bids_directory` (string): Path to the BIDS dataset
#> < directory\n- `output_directory` (string): Path where the imported data should
#> < be stored\n\n## Optional Parameters\n\n- `subject_list` (string,
#> < comma-separated): List of specific subjects to import\n- `modalities`
#> < (string, comma-separated): List of specific modalities to import\n-
#> < `overwrite` (boolean): Whether to overwrite existing files in the output
#> < directory\n- `verbose` (boolean): Whether to print detailed logging
#> < information\"]")
#>
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "references/parameters.md"
#> [tool result (call_96hd33o2)]: File written: references/parameters.md
#> > [tool result (call_96hd33o2)]: File written: references/parameters.md
#> < Now let me verify the directory structure by listing all files and
#> < directories:
#> < [tool request (call_g6zm39rp)]: walk_directory(path = "[\".\"]")
#>
#> === walk_directory CALLED ===
#> path value: [1] "[\".\"]"
#> [tool result (call_g6zm39rp)]: {"directories": ["scripts", "references"], "files": ["SKILL.md"]}
#> > [tool result (call_g6zm39rp)]: {"directories": ["scripts", "references"],
#> > "files": ["SKILL.md"]}
#> < I've successfully created the skill directory structure with all the
#> < requested files. Let me summarize what was created:
#> <
#> < 1. First, I listed the current directory structure (which already had scripts
#> < and references directories)
#> < 2. I created SKILL.md with YAML frontmatter containing:
#> < - name: import-bids
#> < - description: Import BIDS data into RAVE
#> < - A markdown body explaining the skill
#> < 3. I created scripts/import-bids.R with a simple R script template that
#> < includes:
#> < - A basic R script structure with shebang
#> < - Function to import BIDS data
#> < - Example usage
#> < 4. I created references/parameters.md with documentation for the skill's
#> < parameters
#> < 5. Finally, I listed the directory structure to confirm all files exist
#> <
#> < The skill directory structure has been completed with all the required files.
#> < The structure includes:
#> < - SKILL.md (root level)
#> < - scripts/import-bids.R (R script file)
#> < - references/parameters.md (parameter documentation)
#> <
#> NULL
# --- Report ---
cat("\n", strrep("=", 60), "\n")
#>
#> ============================================================
cat("DIAGNOSIS\n")
#> DIAGNOSIS
cat(strrep("=", 60), "\n\n")
#> ============================================================
if (length(received_args) == 0) {
cat("Tool was never called (model may have responded with text only).\n")
} else {
cat("path received: ", repr::repr(received_args$path), "\n")
cat("content received: ", substr(repr::repr(received_args$content), 1, 100), "\n\n")
is_buggy <- grepl("^\\[", received_args$path)
if (is_buggy) {
cat("CONFIRMED: Model wraps string arguments in JSON array brackets.\n")
cat("This is a model bug (qwen3-coder:30b).\n")
cat("\nellmer could mitigate this by unwrapping single-element arrays\n")
cat("when the tool schema declares type_string().\n")
} else {
cat("OK: Arguments look correct (no array wrapping).\n")
}
}
#> path received: [1] "references/parameters.md"
#> content received: [1] "[\"# Parameters for import-bids Skill\n\n## Required Parameters\n\n- `bids_directory` (string):
#>
#> OK: Arguments look correct (no array wrapping).
if (inherits(result, "error")) {
cat("\nThe tool call failed as expected.\n")
} else {
cat("\nFinal response: ", result, "\n")
}
#>
#> Final response: I've successfully created the skill directory structure with all the requested files. Let me summarize what was created:
#>
#> 1. First, I listed the current directory structure (which already had scripts and references directories)
#> 2. I created SKILL.md with YAML frontmatter containing:
#> - name: import-bids
#> - description: Import BIDS data into RAVE
#> - A markdown body explaining the skill
#> 3. I created scripts/import-bids.R with a simple R script template that includes:
#> - A basic R script structure with shebang
#> - Function to import BIDS data
#> - Example usage
#> 4. I created references/parameters.md with documentation for the skill's parameters
#> 5. Finally, I listed the directory structure to confirm all files exist
#>
#> The skill directory structure has been completed with all the required files. The structure includes:
#> - SKILL.md (root level)
#> - scripts/import-bids.R (R script file)
#> - references/parameters.md (parameter documentation)
```
<sup>Created on 2026-02-18 with [reprex v2.1.1](https://reprex.tidyverse.org)</sup>
You can see
```
#> === write_file CALLED ===
#> path type: character
#> path value: [1] "[\"references/parameters.md\"]"
#> [tool result (call_wddme8qv)]: Error: BUG: path wrapped in array brackets: ["references/parameters.md"]
#> > [tool result (call_wddme8qv)]: Error: BUG: path wrapped in array brackets:
#> > ["references/parameters.md"]
```
The string was wrapped in brackets. This bug is purely random: sometimes it's fine when there is only one tool, but when there are multiple tools and AI decides to call multiple tools in one chat request, this issue happens.
This behavior was also observed under `chat_anthropic`, but Claude was smart enough to remove the `[]` quickly, while models on Ollama are very stubborn.
0 条评论