ITADN

Move tokenizer information into pte to reduce ExecuTorch runner args

#1484ClosedJack-Khuu 创建于 2025-01-30
enhancementgood first issueactionableExecuTorchtriaged
J
Jack-Khuucommented
### 🚀 The feature, motivation and pitch After an ExecuTorch model is exported to a `pte`, tokenization information must be passed in as an arg (`-l <#>`) to the runner. This can be avoided by writing this information into the `pte` file itself since the tokenizer is known at export time (sentencepiece => 2, tiktoken =>3). Tokenization information can be stored during export as a [constant_method](https://github.com/pytorch/executorch/blob/073397357118feef0fca91326ed612ce5c60d53b/exir/program/_program.py#L1188). For example: https://github.com/pytorch/torchchat?tab=readme-ov-file#deploy-and-run-on-android ``` cmake-out/et_run llama3.1.pte -z `python3 torchchat.py where llama3.1`/tokenizer.model -l 3 -i "Once upon a time" ``` --- **Task:** 1) Update ExecuTorch exporting to save tokenization information in the pte artifact 2) Update the ExecuTorch runner to read the newly saved metadata For a similar optimization made for aoti: https://github.com/pytorch/torchchat/pull/1159. See https://github.com/pytorch/torchchat/pull/1439 for conversation/more context ### Alternatives Continue to pass tokenizer arguments to the runner ### Additional context _No response_ ### RFC (Optional) _No response_
关闭于 2025-03-14 0 条评论