Block checksum
The asdf specification [section describing the block header](https://github.com/spacetelescope/romancal/actions/runs/23652437489/job/68900650432#step:11:151) contains the following description of the checksum:
> checksum (16-byte string): An optional MD5 checksum of the used data in the block. The special value of all zeros indicates that no checksum verification should be performed.
The mention of "used data" presumably means the compressed bytes (for compressed data). However this library is storing the checksum for the uncompressed bytes.
```python
import hashlib
import asdf, numpy as np
arr = np.arange(100)
uncompressed_checksum = hashlib.md5(arr).digest()
print(f"Uncompressed checksum: {uncompressed_checksum}")
fn = "foo.asdf"
asdf.dump({"arr": arr}, fn, all_array_compression="lz4")
af = asdf.open(fn)
blk = af._blocks.blocks[0]
assert blk.header["checksum"] == uncompressed_checksum
f = open(fn, "rb")
f.seek(blk.data_offset)
bs = f.read(blk.header["used_size"])
compressed_checksum = hashlib.md5(bs).digest()
print(f"Compressed checksum: {compressed_checksum}")
assert blk.header["checksum"] != compressed_checksum
```
This was uncovered by ASDF.jl developers:
https://juliaastro.org/ASDF.jl/previews/PR19/examples/roman/#Note-cbaa8683607656d3
6 条评论