[source-mongodb-v2] _id field schema discovery is extremely expensive
type/bugarea/connectorscommunityneeds-triageteam/extensibilityautoteam
### Connector Name
source-mongodb-v2
### Connector Version
2.0.7
### What step the error happened?
During the sync
### Relevant information
We are syncing a collection with tens of millions of documents. We noticed in our MongoDB logs that we had some queries taking **30 minutes**. It seems that it's this query:
https://github.com/airbytehq/airbyte/blob/a8e626e457d88014d2236624751e787f6040cb5d/airbyte-integrations/connectors/source-mongodb-v2/src/main/java/io/airbyte/integrations/source/mongodb/InitialSnapshotHandler.java#L125
Here's a screenshot from our MongoDB Atlas
<img width="423" height="643" alt="Image" src="https://github.com/user-attachments/assets/3f72f5f3-9e0d-491e-bd79-6e1703b0b849" />
It seems that the purpose of this query is to discover if any `_id` fields have a type other that `ObjectId`. In our case we are willing to guarantee that they don't. Could you introduce an option to skip this discovery? Doing a full collection scan is very expensive for this operation, it's unindexable and so it will always be slow.
### Relevant log output
```shell
N/A
```
### Contribute
- [x] Yes, I want to contribute
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/12891
2 条评论