ITADN

Off-by-one bug in Kinesis shard ending hash

#9909Openmrdziuban 创建于 2026-03-24
bug
M
mrdziubancommented
I'm developing a KCL app in Scala and using Moto to test it. In the process, I ran into an error: ``` 12:33:09.687 [io-compute-blocker-6] ERROR [ShardSyncTask.java:87] s.a.kinesis.leases.ShardSyncTask - Caught exception while sync'ing Kinesis shards and leases software.amazon.kinesis.exceptions.internal.KinesisClientLibIOException: Hash range of shards returned for test-stream was incomplete after 3 retries. at software.amazon.kinesis.leases.HierarchicalShardSyncer.getShardListAtInitialPosition(HierarchicalShardSyncer.java:325) at software.amazon.kinesis.leases.HierarchicalShardSyncer.checkAndCreateLeaseForNewShards(HierarchicalShardSyncer.java:133) at software.amazon.kinesis.leases.ShardSyncTask.call(ShardSyncTask.java:75) at software.amazon.kinesis.metrics.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:54) at software.amazon.kinesis.leases.ShardSyncTaskManager.callShardSyncTask(ShardSyncTaskManager.java:160) at software.amazon.kinesis.coordinator.PeriodicShardSyncManager.syncShardsOnce(PeriodicShardSyncManager.java:189) at software.amazon.kinesis.coordinator.Scheduler.initialize(Scheduler.java:491) at software.amazon.kinesis.coordinator.Scheduler.run(Scheduler.java:456) at com.example.MyAppTest.testRun$$anonfun$2$$anonfun$1(MyAppTest.scala:149) at com.example.MyAppTest.testRun$$anonfun$2$$anonfun$adapted$1(MyAppTest.scala:149) at cats.effect.IOFiber.runLoop(IOFiber.scala:1023) at cats.effect.IOFiber.execR(IOFiber.scala:1399) at cats.effect.IOFiber.run(IOFiber.scala:122) at cats.effect.unsafe.WorkerThread.lookForWork$1(WorkerThread.scala:549) at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:942) ``` With the help of Claude, I found that this seems to be an [off-by-one issue in `init_shards`](https://github.com/getmoto/moto/blob/7d9d153b66eb3dded684a0cd322f245a12a448a0/moto/kinesis/models.py#L240) -- [KCL is expecting `2**128 - 1`](https://github.com/awslabs/amazon-kinesis-client/blob/3c119a9caf47c4d95b926b85ab22774c5ac7f272/amazon-kinesis-client/src/main/java/software/amazon/kinesis/leases/HierarchicalShardSyncer.java#L85-L86) but Moto is using `2**128`. I confirmed that if I patch line 240 of `moto/kinesis/models.py` like so, then my KCL app works as expected: ```diff diff --git a/moto/kinesis/models.py b/moto/kinesis/models.py index 7dd22816c..c56143204 100644 --- a/moto/kinesis/models.py +++ b/moto/kinesis/models.py @@ -237,7 +237,7 @@ class Stream(CloudFormationModel): step = 2**128 // shard_count hash_ranges = itertools.chain( ((i, i * step, (i + 1) * step - 1) for i in range(shard_count - 1)), - [(shard_count - 1, (shard_count - 1) * step, 2**128)], + [(shard_count - 1, (shard_count - 1) * step, 2**128 - 1)], ) for index, start, end in hash_ranges: shard = Shard(index, start, end) ``` Would you be open to a pull request with this change?
1 条评论