Quantizing q8p
I was quantizing weights using :
```
graph.openStore(
full_f16_path, flags: .truncateWhenClose
) { store in
let keys = store.keys
graph.openStore(
f8_path,
flags: .truncateWhenClose
) {
for key in keys {
guard let tensor = store.read(key) else { continue }
print("quantizing \(key) \(tensor)")
$0.write(key, tensor: tensor, codec: [.q8p ])
}
}
}
```
but it looks like some params with less number of elements are not being quantized. Like layers with 320 params.
did you add a check or something? where is it ?
关闭于 2024-09-28 5 条评论