tar archive has too small of rewind buffer for zstandard worst case scenario
Due to format limitations to tell if a file is a valid compressed tar archive we need to decompress the first block and check for the tar header. The problem is our default rewind buffer is 81KB and zstandard can exceed that.
zstandard for one block has 128KB total or 131KB all in for first block on a tar archive.
Now officially BZip2 at -9 has 900KB as the first block max but our bzip2 reader is lazy so only reads bytes as needed vs zstandard so at least for the tar header check with the exact current code i think bzip2 is safe to not throw. I am not sure if we want to just set it to 2x current (161) or potentially higher. Users can fix this on their own but the error message is cryptic they may not know the problem.
4 条评论