Skip to content

fix: Make ObjectValue thread safe. #5699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

tom-andersen
Copy link
Contributor

@tom-andersen tom-andersen commented Feb 8, 2024

Internal tracking: b/323342714

There are potential race conditions in ObjectValue.

Past work in response to NullPointerException made access to internal collection more thread safe, but protection is not complete. #3062

Recent support issue has made me suspicious whether thread safety is at fault. #5683

This PR should make thread safety complete. In addition, memoization of proto is improved, thereby reducing contention.

Copy link
Contributor

github-actions bot commented Feb 8, 2024

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

Copy link
Contributor

github-actions bot commented Feb 8, 2024

Unit Test Results

   180 files  ±0     180 suites  ±0   3m 55s ⏱️ -11s
1 226 tests ±0  1 210 ✔️ ±0  16 💤 ±0  0 ±0 
2 476 runs  ±0  2 444 ✔️ ±0  32 💤 ±0  0 ±0 

Results for commit 78d1c1e. ± Comparison against base commit 6a97c30.

♻️ This comment has been updated with latest results.

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 8, 2024

Coverage Report 1

Affected Products

  • firebase-firestore

    Overall coverage changed from 44.83% (6a97c30) to 44.88% (eac26e6) by +0.05%.

    FilenameBase (6a97c30)Merge (eac26e6)Diff
    DeleteMutation.java90.48%95.24%+4.76%
    ObjectValue.java99.06%98.41%-0.64%
    PatchMutation.java98.39%100.00%+1.61%

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/Dx0xDmtSe5.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 8, 2024

Size Report 1

Affected Products

  • firebase-firestore

    TypeBase (6a97c30)Merge (eac26e6)Diff
    aar1.41 MB1.41 MB+297 B (+0.0%)
    apk (release)11.4 MB11.4 MB+384 B (+0.0%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/yD1MDlNaDe.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 8, 2024

Startup Time Report 1

Note: Layout is sometimes suboptimal due to limited formatting support on GitHub. Please check this report on GCS.

Notes

Startup Times

  • fire-fst

    DeviceStatisticsDistributions
    oriole-32
    Percentile6a97c30eac26e6DiffSignificant (?)
    p10418 ±204 μs562 ±251 μs+145 μs (+34.7%)NO
    p25435 ±206 μs580 ±253 μs+145 μs (+33.4%)NO
    p50476 ±206 μs615 ±244 μs+139 μs (+29.3%)NO
    p75536 ±206 μs662 ±234 μs+126 μs (+23.5%)NO
    p90645 ±249 μs763 ±230 μs+117 μs (+18.2%)NO

    20 test runs in comparison
    CommitTest Runs
    6a97c30
    • 2024-02-12_19:35:34.996370_ggkf
    • 2024-02-12_19:35:34.996415_MKmp
    • 2024-02-12_19:35:34.996427_Dicg
    • 2024-02-12_19:35:34.996447_sRGe
    • 2024-02-12_19:35:34.996458_PNQL
    • 2024-02-12_19:35:34.996466_mvig
    • 2024-02-12_19:35:34.996473_tJnr
    • 2024-02-12_19:35:34.996479_olLr
    • 2024-02-12_19:35:34.996486_qGNa
    • 2024-02-12_19:35:34.996493_vamR
    eac26e6
    • 2024-02-13_21:16:01.811143_UDNT
    • 2024-02-13_21:16:01.811179_udeF
    • 2024-02-13_21:16:01.811190_elgP
    • 2024-02-13_21:16:01.811205_MSeF
    • 2024-02-13_21:16:01.811215_cWoy
    • 2024-02-13_21:16:01.811223_TUbd
    • 2024-02-13_21:16:01.811228_AhRU
    • 2024-02-13_21:16:01.811232_VWNa
    • 2024-02-13_21:16:01.811236_kEwX
    • 2024-02-13_21:16:01.811240_VfVW
    redfin-30
    Percentile6a97c30eac26e6DiffSignificant (?)
    p10608 ±20 μs683 ±155 μs+75.0 μs (+12.3%)NO
    p25630 ±23 μs705 ±152 μs+75.7 μs (+12.0%)NO
    p50668 ±33 μs737 ±152 μs+68.5 μs (+10.2%)NO
    p75713 ±43 μs791 ±150 μs+78.4 μs (+11.0%)NO
    p90777 ±70 μs956 ±239 μs+179 μs (+23.0%)NO

    20 test runs in comparison
    CommitTest Runs
    6a97c30
    • 2024-02-12_19:35:34.996370_ggkf
    • 2024-02-12_19:35:34.996415_MKmp
    • 2024-02-12_19:35:34.996427_Dicg
    • 2024-02-12_19:35:34.996447_sRGe
    • 2024-02-12_19:35:34.996458_PNQL
    • 2024-02-12_19:35:34.996466_mvig
    • 2024-02-12_19:35:34.996473_tJnr
    • 2024-02-12_19:35:34.996479_olLr
    • 2024-02-12_19:35:34.996486_qGNa
    • 2024-02-12_19:35:34.996493_vamR
    eac26e6
    • 2024-02-13_21:16:01.811143_UDNT
    • 2024-02-13_21:16:01.811179_udeF
    • 2024-02-13_21:16:01.811190_elgP
    • 2024-02-13_21:16:01.811205_MSeF
    • 2024-02-13_21:16:01.811215_cWoy
    • 2024-02-13_21:16:01.811223_TUbd
    • 2024-02-13_21:16:01.811228_AhRU
    • 2024-02-13_21:16:01.811232_VWNa
    • 2024-02-13_21:16:01.811236_kEwX
    • 2024-02-13_21:16:01.811240_VfVW
  • timeToInitialDisplay

    DeviceStatisticsDistributions
    oriole-32
    Percentile6a97c30eac26e6DiffSignificant (?)
    p10203 ±3 ms208 ±3 ms+4.95 ms (+2.4%)NO
    p25208 ±3 ms214 ±3 ms+6.07 ms (+2.9%)NO
    p50216 ±3 ms222 ±5 ms+6.36 ms (+2.9%)NO
    p75224 ±3 ms232 ±6 ms+8.30 ms (+3.7%)NO
    p90233 ±4 ms245 ±8 ms+11.8 ms (+5.1%)NO

    20 test runs in comparison
    CommitTest Runs
    6a97c30
    • 2024-02-12_19:35:34.996370_ggkf
    • 2024-02-12_19:35:34.996415_MKmp
    • 2024-02-12_19:35:34.996427_Dicg
    • 2024-02-12_19:35:34.996447_sRGe
    • 2024-02-12_19:35:34.996458_PNQL
    • 2024-02-12_19:35:34.996466_mvig
    • 2024-02-12_19:35:34.996473_tJnr
    • 2024-02-12_19:35:34.996479_olLr
    • 2024-02-12_19:35:34.996486_qGNa
    • 2024-02-12_19:35:34.996493_vamR
    eac26e6
    • 2024-02-13_21:16:01.811143_UDNT
    • 2024-02-13_21:16:01.811179_udeF
    • 2024-02-13_21:16:01.811190_elgP
    • 2024-02-13_21:16:01.811205_MSeF
    • 2024-02-13_21:16:01.811215_cWoy
    • 2024-02-13_21:16:01.811223_TUbd
    • 2024-02-13_21:16:01.811228_AhRU
    • 2024-02-13_21:16:01.811232_VWNa
    • 2024-02-13_21:16:01.811236_kEwX
    • 2024-02-13_21:16:01.811240_VfVW
    redfin-30
    Percentile6a97c30eac26e6DiffSignificant (?)
    p10247 ±3 ms273 ±6 ms+26.0 ms (+10.5%)YES
    p25253 ±3 ms280 ±6 ms+26.5 ms (+10.5%)MAYBE
    p50261 ±3 ms288 ±7 ms+27.3 ms (+10.4%)MAYBE
    p75269 ±3 ms299 ±9 ms+29.8 ms (+11.1%)MAYBE
    p90279 ±4 ms318 ±14 ms+38.8 ms (+13.9%)MAYBE

    20 test runs in comparison
    CommitTest Runs
    6a97c30
    • 2024-02-12_19:35:34.996370_ggkf
    • 2024-02-12_19:35:34.996415_MKmp
    • 2024-02-12_19:35:34.996427_Dicg
    • 2024-02-12_19:35:34.996447_sRGe
    • 2024-02-12_19:35:34.996458_PNQL
    • 2024-02-12_19:35:34.996466_mvig
    • 2024-02-12_19:35:34.996473_tJnr
    • 2024-02-12_19:35:34.996479_olLr
    • 2024-02-12_19:35:34.996486_qGNa
    • 2024-02-12_19:35:34.996493_vamR
    eac26e6
    • 2024-02-13_21:16:01.811143_UDNT
    • 2024-02-13_21:16:01.811179_udeF
    • 2024-02-13_21:16:01.811190_elgP
    • 2024-02-13_21:16:01.811205_MSeF
    • 2024-02-13_21:16:01.811215_cWoy
    • 2024-02-13_21:16:01.811223_TUbd
    • 2024-02-13_21:16:01.811228_AhRU
    • 2024-02-13_21:16:01.811232_VWNa
    • 2024-02-13_21:16:01.811236_kEwX
    • 2024-02-13_21:16:01.811240_VfVW

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/VjZfdVhEOe/index.html

boolean modified = false;

@Nullable Value existingValue = extractNestedValue(partialValue, currentPath);
Copy link
Contributor Author

@tom-andersen tom-andersen Feb 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding currentValue as parameter as replacement for existingValue, such that this method can be static. This helps reason about side effects, and also eliminates redundant work of extracting existing value from root. The parameter currentValue follows same pattern as currentPath, such that each recursion passes in a child.

This change isn't strictly required for thread safety, but makes the code more efficient and readable.

@@ -195,7 +214,19 @@ private void setOverlay(FieldPath path, @Nullable Value value) {
}
}

currentLevel.put(path.getLastSegment(), value);
String lastSegment = path.getLastSegment();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than just putting value every time. We more carefully only put value when there is a change. Otherwise, we will set dirty to true, causing needless work, when values actually were the same.

Copy link
Contributor

@MarkDuckworth MarkDuckworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tom-andersen, I have a few questions. But overall the direction looks good to me

}
}

private static Value extractSegmentValue(Value value, String segment) {
return Values.isMapValue(value) ? value.getMapValue().getFieldsOrDefault(segment, null) : null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was it intentional to invert the logic for extracting a value from a MapValue? Previously the code ran Values.isMapValue(value) on the value contained in the input map and returned null if it was not a map. Now that check is not performed on the extracted value. Seems like this could change what this method returns?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semantically, this should be equivalent. The helper function is used in other contexts where value might be null, so I had to add check. Since this is being done in helper, it seemed like overkill to also do it in the loop. Technically the loop will work fine even without early return, but the short circuiting behavior has nice performance characteristics.

It isn't the easiest code to read, so any recommendations are welcome.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new code already is more readable. So thanks for the improvement, and clarifying comment. I don't have any recommendations for improvement, but consider a unit test if you have any concerns about equivalency.

private static @Nullable MapValue applyOverlay(
FieldPath currentPath, Map<String, Object> currentOverlays, Value currentValue) {
if (currentOverlays.isEmpty()) {
return null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this return currentValue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. The method description states:

@return The merged data at `currentPath` or null if no modifications were applied.

This basically just short circuiting logic before more heavy lifting of creating builder objects is done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see where this is called expects return value of null if there is no overlay. Seems good then, I guess I would just expect a different behavior if the function was being written for the first time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests look like they may be related to other work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct. Let me revert that.

private static @Nullable MapValue applyOverlay(
FieldPath currentPath, Map<String, Object> currentOverlays, Value currentValue) {
if (currentOverlays.isEmpty()) {
return null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see where this is called expects return value of null if there is no overlay. Seems good then, I guess I would just expect a different behavior if the function was being written for the first time.

}
}

private static Value extractSegmentValue(Value value, String segment) {
return Values.isMapValue(value) ? value.getMapValue().getFieldsOrDefault(segment, null) : null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new code already is more readable. So thanks for the improvement, and clarifying comment. I don't have any recommendations for improvement, but consider a unit test if you have any concerns about equivalency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants