Wide chars, UTF-8, terminal escapes and colors, etc.
bug
Cowsay doesn't handle variant character widths well. It kind of assumes all characters are 1 char wide (in display) and (I think) 1 byte (in the input encoding). This means that non-English/Latin characters in the cows or the message text are not handled well.
This is an expansion of [#65 "Use Debian's UTF-8 Patch"](https://github.com/cowsay-org/cowsay/issues/65).
Aspects:
* Messages with multi-byte encoded characters don't look right. The speech balloon width and word wrapping are wrong.
* Terminal control (escape) sequences, like color codes, are treated as visible characters.
Bad-wrapped multi-byte char [example](https://github.com/cowsay-org/cowsay/issues/65):
<img width="373" alt="image" src="https://github.com/user-attachments/assets/603645d4-f9d4-4367-82a8-5b21593a45c1">
## Considerations
The cow files distributed with cowsay are all UTF-8, regardless of what locale the user is running in or how their system is set up. (I think? Or are they actually ASCII/Latin-1, since they are Perl source code?)
Message input might be in the user's locale while the cow files are UTF-8. Custom cows (including in third-party cow herd packages) might be in other encodings, which may or may not be the same encoding as
Perl's standard library doesn't support char width detection, I don't think. Would need a CPAN module for that. We currently don't take any deps on modules. Would need to figure out how to do that. I think we'd vendor the module (ship a copy of it in cowsay itself), to avoid creating any external dependencies or a more complicated install process.
## Testing
Examples:
* `cowsay "MÖÖÖ"`
* `cowsay 'Привет, мир!'`
* `cowsay 'Ищу свое лицо. Особых примет нет.'`
Wide chars:
* `cowsay "我愛中國人"`
* from [Debian 769565 "widechar not good"](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=769565):
* `cowsay 'でびあん/Debian'`
* `cowsay 谢谢你`
ANSI [terminal escapes](https://bugs.launchpad.net/ubuntu/+source/cowsay/+bug/1027033):
* `echo 'Hello, World!' | toilet -w 100 --metal | cowsay -n`
* `figlet "Hello World!" | toilet -f term --metal | /usr/games/cowsay -n`
## TODO
* [ ] Determine and document which encoding(s) are supported for cowfiles.
* [ ] Make our source code UTF-8 (with `use utf8;`)?
* [-] Add support for detecting, and maybe explicitly setting, message input encoding.
* [-] Bump required Perl to 5.8.1 (or 5.8.7, or later), which added Unicode fixes relevant to this UTF-8 stuff?
* 5.8.0 added Unicode support and UTF8-ness of stdin/out/err/ARGV.
* 5.8.1 restored behavior of stdin/out/err and ARGV not being interpreted as UTF8 by default.
* ${^UTF8LOCALE} was added in 5.8.7.
* [ ] Support wide chars and invisible chars (like terminal escapes).
* ("Wide" in the sense that they are displayed 2-char width; not that they're a `wchar` type in encoding.)
* [ ] Add tests for multibyte and wide characters.
## References
* https://github.com/cowsay-org/cowsay/issues/65
* Third-party patches
* [Debian patch for UTF-8 input and char sizes](https://sources.debian.org/patches/cowsay/3.03%2Bdfsg2-8/utf8_width/)
* [Debian bug 2544557](http://bugs.debian.org/254557)
* [Ubuntu bug 393212](https://bugs.launchpad.net/ubuntu/+source/cowsay/+bug/393212)
* ANSI escape codes
* [Ubuntu bug 1027033 "should ignore ANSI codes for width"](https://bugs.launchpad.net/ubuntu/+source/cowsay/+bug/1027033)
* [Ubuntu bug 1437804 "confused by input with ANSI colour codes"](https://bugs.launchpad.net/ubuntu/+source/cowsay/+bug/1437804)
* Perl doco
* [${^UTF8LOCALE}](https://perldoc.perl.org/variables/$%7B%5EUTF8LOCALE%7D)
* [perlrun -C Unicode features](https://perldoc.perl.org/perlrun#-C-%5Bnumber%2Flist%5D)
* ["UTF-8 no longer default under UTF-8 locales"](https://perldoc.perl.org/perl581delta#UTF-8-no-longer-default-under-UTF-8-locales)
5 条评论