Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Messaging CI on Xcode 15 #12028

Merged
merged 8 commits into from
Nov 1, 2023
Merged

Run Messaging CI on Xcode 15 #12028

merged 8 commits into from
Nov 1, 2023

Conversation

paulb777
Copy link
Member

@paulb777 paulb777 commented Oct 31, 2023

Run FirebaseMessaging CI on Xcode 15.0.1 along with some changes:

  • Add SPM visionOS testing for Messaging and Storage
  • Defer Messaging integration tests on Xcode 15 pending PubSub test integration test failures
  • Only run unit tests via pod_lib_lint instead of doubling up in the messaging job
  • Rename messaging job to indicate that it is only to run the integration tests
  • Run all SPM platform tests in non-cron
  • Test --use-static-frameworks in cron instead of deprecated --use-modular-headers
  • Restore Storage cron
  • Move Storage integration tests back to Xcode 14 because of flakiness

#no-changelog

@google-oss-bot
Copy link

google-oss-bot commented Oct 31, 2023

Coverage Report 1

Affected Products

  • FirebaseMessaging-iOS-FirebaseMessaging.framework

    Overall coverage changed from 60.67% (24be340) to 60.56% (24bffb0) by -0.11%.

    FilenameBase (24be340)Merge (24bffb0)Diff
    FIRMessagingCheckinPreferences.m98.02%93.07%-4.95%
    FIRMessagingPendingTopicsList.m90.59%89.80%-0.78%

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/1LEer18DOr.html

@paulb777
Copy link
Member Author

paulb777 commented Nov 1, 2023

Consistently getting this integration test failure with Xcode 15.0.1:

FIRMessagingPubSubTest
    ✗ testSubscribeTopic, XCTAssertNil failed: "Error Domain=com.google.fcm Code=0 "Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field." UserInfo={NSLocalizedFailureReason=Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field.}"
    ✗ testUnsubscribeTopic, XCTAssertNil failed: "Error Domain=com.google.fcm Code=0 "Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field." UserInfo={NSLocalizedFailureReason=Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field.}"
FIRMessagingTokenRefreshTests
    ✓ testDeleteDataWithTokenRefreshDelegatesAndNotifications (1.587 seconds)
    ✓ testDeleteDefaultTokenWithTokenRefreshDelegatesAndNotifications (0.518 seconds)
    ✓ testDeleteTokenWithTokenRefreshDelegatesAndNotifications (1.053 seconds)


FirebaseMessaging_Unit_integration.FIRMessagingPubSubTest
  testSubscribeTopic, XCTAssertNil failed: "Error Domain=com.google.fcm Code=0 "Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field." UserInfo={NSLocalizedFailureReason=Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field.}"
  /Users/runner/work/firebase-ios-sdk/firebase-ios-sdk/FirebaseMessaging/Tests/IntegrationTests/FIRMessagingPubSubTest.swift:52
  messaging.subscribe(toTopic: "cat_video") { error in
    XCTAssertNil(error)
    expectation.fulfill()

testUnsubscribeTopic, XCTAssertNil failed: "Error Domain=com.google.fcm Code=0 "Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field." UserInfo={NSLocalizedFailureReason=Invalid registration response :'Error=INTERNAL_SERVER_ERROR'. It is missing 'token' field.}"
/Users/runner/work/firebase-ios-sdk/firebase-ios-sdk/FirebaseMessaging/Tests/IntegrationTests/FIRMessagingPubSubTest.swift:63
  messaging.unsubscribe(fromTopic: "cat_video") { error in
    XCTAssertNil(error)
    expectation.fulfill()


   Executed 5 tests, with 2 failures (0 unexpected) in 28.335 (28.394) seconds
2023-10-31 23:47:22.212 xcodebuild[14521:81384] [MT] IDETestOperationsObserverDebug: 898.275 elapsed -- Testing started completed.
2023-10-31 23:47:22.212 xcodebuild[14521:81384] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2023-10-31 23:47:22.212 xcodebuild[14521:81384] [MT] IDETestOperationsObserverDebug: 898.275 sec, +898.275 sec -- end
Failing tests:
  FIRMessagingPubSubTest.testSubscribeTopic()
  FIRMessagingPubSubTest.testUnsubscribeTopic()

** TEST FAILED **

@paulb777 paulb777 mentioned this pull request Nov 1, 2023
@paulb777 paulb777 enabled auto-merge (squash) November 1, 2023 15:15
@paulb777 paulb777 requested a review from ncooke3 November 1, 2023 15:16
@ncooke3
Copy link
Member

ncooke3 commented Nov 1, 2023

Only run unit tests via pod_lib_lint instead of doubling up in the messaging job

Do you think this is something we should extend to other products as well?

Copy link
Member

@ncooke3 ncooke3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work, the runs-on: ${{ matrix.os }} is super useful!

@paulb777 paulb777 merged commit 7351504 into master Nov 1, 2023
108 checks passed
@paulb777 paulb777 deleted the pb-msg-xc15-ci branch November 1, 2023 15:22
@paulb777
Copy link
Member Author

paulb777 commented Nov 1, 2023

Only run unit tests via pod_lib_lint instead of doubling up in the messaging job

Do you think this is something we should extend to other products as well?

In most cases, we don't. I'll review as I work through the Xcode 15 changes.

@paulb777
Copy link
Member Author

paulb777 commented Nov 1, 2023

@andrewheard verified in #12035 that the visionOS changes don't work. I'll remove in a subsequent PR

@firebase firebase locked and limited conversation to collaborators Dec 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants