ITADN

Block checksum

#2015Openbraingram 创建于 2026-04-03
B
braingramcommented
The asdf specification [section describing the block header](https://github.com/spacetelescope/romancal/actions/runs/23652437489/job/68900650432#step:11:151) contains the following description of the checksum: > checksum (16-byte string): An optional MD5 checksum of the used data in the block. The special value of all zeros indicates that no checksum verification should be performed. The mention of "used data" presumably means the compressed bytes (for compressed data). However this library is storing the checksum for the uncompressed bytes. ```python import hashlib import asdf, numpy as np arr = np.arange(100) uncompressed_checksum = hashlib.md5(arr).digest() print(f"Uncompressed checksum: {uncompressed_checksum}") fn = "foo.asdf" asdf.dump({"arr": arr}, fn, all_array_compression="lz4") af = asdf.open(fn) blk = af._blocks.blocks[0] assert blk.header["checksum"] == uncompressed_checksum f = open(fn, "rb") f.seek(blk.data_offset) bs = f.read(blk.header["used_size"]) compressed_checksum = hashlib.md5(bs).digest() print(f"Compressed checksum: {compressed_checksum}") assert blk.header["checksum"] != compressed_checksum ``` This was uncovered by ASDF.jl developers: https://juliaastro.org/ASDF.jl/previews/PR19/examples/roman/#Note-cbaa8683607656d3
6 条评论