gcp pubsub source: at_least_once #3767
Conversation
Actually I am now looking how to test the |
You can move the ack logic outside the |
b39649d
to
4cd9949
Compare
351d161
to
8e89963
Compare
| "num_bytes_processed": 54, | ||
| "num_messages_processed": 6, | ||
| "num_invalid_messages": 0, | ||
| "num_consecutive_empty_batches": 4, | ||
| }); | ||
| assert_eq!(exit_state, expected_exit_state); | ||
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn test_gcp_pubsub_source_ack() { |
There was a problem hiding this comment.
We could drop the line self.subscription.ack(message_ids).await, and this test would still pass. Does the PubSub API have some way to tell us that we acknowledged the messages correctly?
There was a problem hiding this comment.
Yeah... sadly I think there is no way to really know if the message are ack.
My guess is because gcp doesn't garantee you that the message will not be redeliver even when you call ack.
When I was working with go, we usually tested that with an interface mock...
Options1:
One way, would be to set an ack deadline really small. And at the end of the test just subscribe and wait to see if we receive message. It will not be perfect but it should be "fine". Problem is we might need to wait few seconds...
Options2:
having some interface so we can mock / ensure that the function was call.
There was a problem hiding this comment.
I'm leaning towards Option 2. We can only mock the call to ack for now. There are examples in the codebase of mocked traits (Metastore, code-generated clients) with mockall.
Description
Following on #3720 and #3744 for #1107
check_connectivityat_least_oncedelivery, and add test for it.How was this PR tested?
It is adding some test, to ensure that the message are ACK in order
Next step
In next PR we can focus on: