ITADN

extract_text: space extraction fails due to bad width estimation for composite fonts without the " " (space) glyph

#3978Openjbylund 创建于 12 天前
workflow-text-extractionneeds-pdf
J
jbylundcommented
For TeX produced pdfs using a composite font there is no " " glyph in the document. Instead the word gaps are drawn by using adjustments in TJ arrays. pypdf must choose a TJ adjustment which separates kerning (intraword adjustments) from spacing (inter word). It does this by searching the character map for the " " glyph. If there is none found it returns the literal " " char. Then _add_space_width looks up the width of that glyph at that index. In our case " " is not found in the doc, " " = chr(32), and it finds the width of the glyph at 32 in character widths - 32 in my font happened to be Q, and the width it came up with was much wider than the TJ adjustment between words, meaning no spaces are inserted into the extracted text. I think there are 3 cases: 1) space glyph in the character map - the correct width is used here 2) no space glyph in the char map, and there is something in slot 32 of the widths map - the width of whatever is in that slot is used as the space width which then feeds into the kerning vs space threshold - which frequently results in no spaces being extracted 3) no space glyph in the char map, and nothing in slot 32 of the widths map - /DW // 2 is used - in my limited experiment this sometimes extracts intraword spaces where there are none [minimal_bug.pdf](https://github.com/user-attachments/files/31231935/minimal_bug.pdf) - has one extra entry in the width map for glyph 32, glyph 32 is never drawn in the document, so this shouldn't make any difference? [minimal_ok.pdf](https://github.com/user-attachments/files/31231936/minimal_ok.pdf) - missing 32 in the width map (falls back to /DW derived value) [xetex_lato.pdf](https://github.com/user-attachments/files/31232401/xetex_lato.pdf) The two minimal ones render in some (firefox), but don't in a pdf preview program I had. xetex_lato.pdf is a more "real world" example. xetex_lato.pdf has no space width and /W doesn't have the space because the glyph is never drawn. The embedded font subset has no cmap table — so nothing in the file says which glyph is U+0020, so I'm not sure how to recover the space width? Checked on 5.3.0 and 6.16.1 and main - all reproduce the same behavior.
3 条评论