Repository navigation
feat(storage): implement atomic compose via DeleteSourceObjects - #4804
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4804 +/- ##
==========================================
+ Coverage 83.93% 83.97% +0.04%
==========================================
Files 176 176
Lines 21563 21630 +67
==========================================
+ Hits 18098 18163 +65
- Misses 2765 2766 +1
- Partials 700 701 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
482c0ab to
0163678
Compare
73be66d to
96d222d
Compare
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
7 similar comments
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
bcef5d1 to
f71af26
Compare
f71af26 to
290be5f
Compare
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
4 similar comments
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
|
Hi @abhishek10004, @meet2mky, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you! |
abhishek10004
left a comment
There was a problem hiding this comment.
Left one minor comment. Rest looks good. Haven't checked the UTs, will rely on previous reviewer's review for that.
Description
This pull request implements an atomic compose operation by introducing a
DeleteSourceObjectsboolean inside theComposeObjectsRequest. It leverages the new GCS Compose API functionality that allows atomically deleting source objects on the server-side the exact millisecond a compose operation completes.Impact:
This change fundamentally resolves race conditions, early deletion billing spikes, and "soft delete" charges commonly encountered during append-heavy AI/ML workloads.
Key Changes:
DeleteSourceObjects: trueflag during GCS object composition viaComposeObjectCreatorand removed the manualDeleteObjectdeferred calls.fake/bucket.goto simulate source deletion and added unit tests inbucket_tests.go,bucket_handle_test.go, andcompose_object_creator_test.go.Link to the issue in case of a bug fix:
b/430606937
Testing details
make build.go test -race ./internal/storage/... ./internal/gcsx/...passes cleanly.execute-integration-tests,execute-integration-tests-on-zb).Any backward incompatible change? If so, please explain.
N/A