VertexAI: add support for mock responses versioning system #13206
Conversation
| run: scripts/update_vertexai_responses.sh | ||
| - name: Find cloned and latest versions | ||
| run: | | ||
| echo "current_tag=$(git describe --tags | awk -F'/' '{print $NF}')" >> $GITHUB_ENV |
There was a problem hiding this comment.
When running this command I'm getting
v0.8.0-47-g2bfe9e3
in a misc repo. This happens when the latest commit isn't annotated with a tag.
This would make the equality check below never pass.
There was a problem hiding this comment.
I believe the commit that git describe sees will always be annotated with a tag since update_vertexai_responses.sh clones a specific tag.
For example, the current latest commit in vertexai-sdk-test-data is not annotated with a tag, but cloning with
git clone --depth 1 --branch v1.0 https://github.com/FirebaseExtended/vertexai-sdk-test-data
and running git describe --tags shows v1.0.
There was a problem hiding this comment.
Then you only need git tag -l as there's only one tag available, right?
| echo "latest_tag=$(git -c 'versionsort.suffix=-' ls-remote --tags --sort='v:refname' https://github.com/FirebaseExtended/vertexai-sdk-test-data.git | tail -n1 | awk -F'/' '{print $NF}')" >> $GITHUB_ENV | ||
| working-directory: FirebaseVertexAI/Tests/Unit/vertexai-sdk-test-data | ||
| - name: Find comment from previous run if exists | ||
| uses: peter-evans/find-comment@v3 |
There was a problem hiding this comment.
since this isn't an official github action, could you pin to a commit rather to the tag?, same below
There was a problem hiding this comment.
Andrew is looking into whether we can use unofficial GitHub actions at all. I'll pin it to a commit if we end up using this action. Thanks!
| ### Vertex AI Mock Responses Check :warning: | ||
| A newer major version of the mock responses for Vertex AI unit tests is available. | ||
| [update_vertexai_responses.sh](https://github.com/firebase/firebase-ios-sdk/blob/main/scripts/update_vertexai_responses.sh) should be updated to clone the latest version of the responses. | ||
| - name: Fail job if newer version is available |
There was a problem hiding this comment.
@andrewheard do you want the test to fail if not using the latest version? IMO just warning should be enough as to not block changes to other sdks
| @@ -17,6 +17,10 @@ | |||
| # This script replaces mock response files for Vertex AI unit tests with a fresh | |||
| # clone of the shared repository of Vertex AI test data. | |||
|
|
|||
| RESPONSES_VERSION='v1.*' # the major version of mock responses to use | |||
| REPO="https://github.com/FirebaseExtended/vertexai-sdk-test-data.git" | |||
| TAG=$(git -c 'versionsort.suffix=-' ls-remote --tags --sort='v:refname' "$REPO" | grep "$RESPONSES_VERSION" | tail -n1 | awk -F'/' '{print $NF}') | |||
There was a problem hiding this comment.
Add a comment here to explain what the command does
There was a problem hiding this comment.
Does the output include any / ? it's not clear either why the awk is needed here.
| - name: Find cloned and latest versions | ||
| run: | | ||
| echo "current_tag=$(git describe --tags | awk -F'/' '{print $NF}')" >> $GITHUB_ENV | ||
| echo "latest_tag=$(git -c 'versionsort.suffix=-' ls-remote --tags --sort='v:refname' https://github.com/FirebaseExtended/vertexai-sdk-test-data.git | tail -n1 | awk -F'/' '{print $NF}')" >> $GITHUB_ENV |
There was a problem hiding this comment.
Why you need the last awk command?
There was a problem hiding this comment.
The output before awk includes the commit hash and looks like this:
0b55b397d27706407eb4062dd70f18cad47edcfc refs/tags/v1.0
I'm using the awk command to extract the tag v1.0 so that it can be compared with the output of git describe --tags.
But I see that the awk command above with git describe --tags is unnecessary.
There was a problem hiding this comment.
Yes, thanks. I meant to comment on the command above.
|
The iOS SDK team has decided to clone the mock responses from HEAD rather than a tag. |
Vertex AI Mock Responses Check
|
This updates update_vertexai_responses.sh to clone the latest minor version within a hard-coded major version of the mock responses.
It also adds a CI job that fails and leaves a comment on the PR if update_vertexai_responses.sh is cloning an outdated major version.