Repository navigation
fix(worker): batch flag comparison updates in ComputeComparison - #2460
Open
sentry[bot] wants to merge 1 commit into
Open
sentry[bot] wants to merge 1 commit into
sentry[bot] wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2460 +/- ##
=======================================
Coverage 91.65% 91.65%
=======================================
Files 1341 1341
Lines 53342 53352 +10
Branches 1649 1649
=======================================
+ Hits 48888 48898 +10
Misses 4133 4133
Partials 321 321
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses an N+1 query performance issue in the
ComputeComparisonTask.create_or_update_flag_comparisonsmethod withinapps/worker/tasks/compute_comparison.py.Previously, the method would update existing
CompareFlagrows and create new ones in a loop, leading to individualUPDATEstatements anddb_session.flush()calls for each new row. This resulted in an N+1 query pattern, as observed in Sentry traces.The fix refactors the logic to:
CompareFlagobjects into a list of dictionaries, ensuring uniform keys for efficient batching by SQLAlchemy.CompareFlagobjects into a separate list without immediate flushing.db_session.flush()call fromstore_flag_comparison.db_session.bulk_update_mappingsfor all existing rows and a singledb_session.add_allfor all new rows after the loop completes.db_session.flush()at the end to persist all changes.CompareFlagobjects already loaded in the session after the bulk update to ensure subsequent reads reflect the latest data.This change significantly reduces the number of database queries, improving the performance of the
ComputeComparisontask.Legal Boilerplate
Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.
Fixes WORKER-PG3
@sentry <feedback>: Autofix iterates on these changes@sentry stop iterating: Autofix stops iterating on this runThis PR was automatically generated by Sentry. You can adjust this setting at any time.
Note
Medium Risk
Changes how comparison flag rows are persisted (bulk ORM updates bypass the identity map until expire/flush), but behavior and fields should match prior logic in a background worker path.
Overview
Batches flag comparison DB writes in
ComputeComparisonTask.create_or_update_flag_comparisonsto remove per-flagUPDATE/flushN+1 behavior seen in worker traces.Instead of mutating each existing
CompareFlagin the loop and callingstore_flag_comparisonwith immediateadd/flush, the loop now collects new rows and update dicts (with a sharedupdated_atfromget_utc_now()), then applies onebulk_update_mappings,expires loaded instances,add_allfor inserts, and a singleflush.store_flag_comparisononly constructs and returns aCompareFlagusing foreign-key IDs.Reviewed by Cursor Bugbot for commit b48860b. Bugbot is set up for automated code reviews on this repo. Configure here.