-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI/CD] Add E2E Cypress workflow for Dashboards Reporting #262
Merged
mengweieric
merged 4 commits into
opensearch-project:main
from
RyanL1997:reporting-cypress-workflow
Dec 18, 2023
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,180 @@ | ||
name: Cypress E2E Dashboards Reporting Plugin Test | ||
|
||
on: [pull_request, push] | ||
|
||
env: | ||
CI: 1 | ||
# avoid warnings like "tput: No value for $TERM and no -T specified" | ||
TERM: xterm | ||
OPENSEARCH_DASHBOARDS_VERSION: 'main' | ||
OPENSEARCH_VERSION: '3.0.0' | ||
OPENSEARCH_PLUGIN_VERSION: '3.0.0.0' | ||
PLUGIN_NAME: dashboards-reporting | ||
|
||
jobs: | ||
tests: | ||
name: Run Cypress E2E Dashboards Reporting Plugin Tests | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
os: [ ubuntu-latest ] | ||
jdk: [ 11 ] | ||
runs-on: ${{ matrix.os }} | ||
|
||
steps: | ||
- name: Set up JDK | ||
uses: actions/setup-java@v1 | ||
with: | ||
java-version: ${{ matrix.jdk }} | ||
|
||
- name: Download Job Scheduler artifact | ||
uses: suisei-cn/[email protected] | ||
with: | ||
url: https://aws.oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=org.opensearch.plugin&a=opensearch-job-scheduler&v=${{ env.OPENSEARCH_PLUGIN_VERSION }}-SNAPSHOT&p=zip | ||
target: plugin-artifacts/ | ||
filename: job-scheduler.zip | ||
|
||
- name: Download Reports Scheduler artifact | ||
uses: suisei-cn/[email protected] | ||
with: | ||
url: https://aws.oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=org.opensearch.plugin&a=opensearch-reports-scheduler&v=${{ env.OPENSEARCH_PLUGIN_VERSION }}-SNAPSHOT&p=zip | ||
target: plugin-artifacts/ | ||
filename: reports-scheduler.zip | ||
|
||
- name: Download OpenSearch | ||
uses: peternied/download-file@v2 | ||
with: | ||
url: https://artifacts.opensearch.org/snapshots/core/opensearch/${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/opensearch-min-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT-linux-x64-latest.tar.gz | ||
|
||
- name: Extract OpenSearch | ||
run: | | ||
tar -xzf opensearch-*.tar.gz | ||
rm -f opensearch-*.tar.gz | ||
shell: bash | ||
|
||
- name: Install Job Scheduler | ||
run: | | ||
/bin/bash -c "yes | ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/bin/opensearch-plugin install file:$(pwd)/plugin-artifacts/job-scheduler.zip" | ||
shell: bash | ||
|
||
- name: Install Reports Scheduler | ||
run: | | ||
/bin/bash -c "yes | ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/bin/opensearch-plugin install file:$(pwd)/plugin-artifacts/reports-scheduler.zip" | ||
shell: bash | ||
|
||
- name: Run OpenSearch | ||
run: | | ||
/bin/bash -c "./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/bin/opensearch &" | ||
sleep 30 | ||
shell: bash | ||
|
||
- name: Check OpenSearch Running on Linux | ||
if: ${{ runner.os != 'Windows'}} | ||
run: curl http://localhost:9200/ | ||
shell: bash | ||
|
||
- name: Show OpenSearch Logs | ||
if: always() | ||
run: cat ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/logs/opensearch.log | ||
shell: bash | ||
|
||
- name: Checkout OpenSearch Dashboards | ||
uses: actions/checkout@v2 | ||
with: | ||
path: OpenSearch-Dashboards | ||
repository: opensearch-project/OpenSearch-Dashboards | ||
ref: ${{ env.OPENSEARCH_DASHBOARDS_VERSION }} | ||
fetch-depth: 0 | ||
filter: | | ||
cypress | ||
test | ||
|
||
- name: Checkout Dashboards Reporting Plugin in OpenSearch Dashboards Plugins Dir | ||
uses: actions/checkout@v2 | ||
with: | ||
path: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }} | ||
|
||
- id: tool-versions | ||
run: | | ||
echo "node_version=$(cat .node-version)" >> $GITHUB_OUTPUT | ||
echo "yarn_version=$(jq -r '.engines.yarn' package.json)" >> $GITHUB_OUTPUT | ||
working-directory: OpenSearch-Dashboards | ||
shell: bash | ||
|
||
- uses: actions/setup-node@v1 | ||
with: | ||
node-version: ${{ steps.tool-versions.outputs.node_version }} | ||
registry-url: 'https://registry.npmjs.org' | ||
|
||
- name: Setup Opensearch Dashboards | ||
run: | | ||
npm uninstall -g yarn | ||
echo "Installing yarn ${{ steps.tool-versions.outputs.yarn_version }}" | ||
npm i -g yarn@${{ steps.tool-versions.outputs.yarn_version }} | ||
yarn cache clean | ||
working-directory: OpenSearch-Dashboards | ||
shell: bash | ||
|
||
- name: OpenSearch Dashboards Plugin Bootstrap | ||
run: | | ||
max_retries=3 | ||
count=0 | ||
until yarn osd bootstrap; do | ||
exit_code=$? | ||
count=$(($count + 1)) | ||
echo "Attempt $count/$max_retries failed with exit code $exit_code, retrying..." | ||
if [ $count -ge $max_retries ]; then | ||
echo "Reached maximum retry attempts, stopping..." | ||
exit $exit_code | ||
fi | ||
sleep 5 # Wait for 5 seconds before retrying | ||
done | ||
working-directory: OpenSearch-Dashboards | ||
shell: bash | ||
|
||
- name: Run Opensearch Dashboards with Dashboards Reporting Plugin Installed | ||
run: | | ||
nohup yarn start --no-base-path --no-watch | tee dashboard.log & | ||
working-directory: OpenSearch-Dashboards | ||
|
||
- name : Check If OpenSearch Dashboards Is Ready | ||
if: ${{ runner.os == 'Linux' }} | ||
run: | | ||
if timeout 600 grep -q "bundles compiled successfully after" <(tail -n0 -f dashboard.log); then | ||
echo "OpenSearch Dashboards compiled successfully." | ||
else | ||
echo "Timeout for 600 seconds reached. OpenSearch Dashboards did not finish compiling." | ||
exit 1 | ||
fi | ||
working-directory: OpenSearch-Dashboards | ||
|
||
- name: Install Cypress | ||
run: | | ||
npx cypress install | ||
shell: bash | ||
working-directory: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }} | ||
|
||
- name: Get Cypress version | ||
id: cypress_version | ||
run: | | ||
echo "::set-output name=cypress_version::$(cat ./package.json | jq '.dependencies.cypress' | tr -d '"')" | ||
working-directory: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }} | ||
|
||
- name: Run Cypress tests | ||
run: | | ||
yarn cypress:run --browser chrome --headless --spec '.cypress/integration/*' | ||
working-directory: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }} | ||
|
||
- name: Capture failure screenshots | ||
uses: actions/upload-artifact@v1 | ||
if: failure() | ||
with: | ||
name: cypress-screenshots-${{ matrix.os }} | ||
path: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }}/.cypress/screenshots | ||
|
||
- name: Capture test video | ||
uses: actions/upload-artifact@v1 | ||
if: failure() | ||
with: | ||
name: cypress-videos-${{ matrix.os }} | ||
path: OpenSearch-Dashboards/plugins/${{ env.PLUGIN_NAME }}/.cypress/videos |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
const { defineConfig } = require('cypress'); | ||
|
||
module.exports = defineConfig({ | ||
video: true, | ||
chromeWebSecurity: true, | ||
fixturesFolder: '.cypress/fixtures', | ||
screenshotsFolder: '.cypress/screenshots', | ||
videosFolder: '.cypress/videos', | ||
downloadsFolder: '.cypress/downloads', | ||
viewportWidth: 2000, | ||
viewportHeight: 1320, | ||
requestTimeout: 60000, | ||
responseTimeout: 60000, | ||
defaultCommandTimeout: 60000, | ||
//experimentalNetworkStubbing: true, | ||
//experimentalMemoryManagement: true, | ||
numTestsKeptInMemory: 10, //Default value 50, chrome crashes without lowering | ||
env: { | ||
opensearch: 'localhost:9200', | ||
opensearchDashboards: 'localhost:5601', | ||
security_enabled: true, | ||
}, | ||
'cypress-watch-and-reload': { | ||
watch: ['common/**', 'public/**', 'server/**'], | ||
}, | ||
e2e: { | ||
// We've imported your old cypress plugins here. | ||
// You may want to clean this up later by importing these. | ||
setupNodeEvents(on, config) { | ||
return require('./.cypress/plugins/index.js')(on, config); | ||
}, | ||
baseUrl: 'http://localhost:5601', | ||
specPattern: '.cypress/integration/**/*.spec.{js,jsx,ts,tsx}', | ||
supportFile: '.cypress/support/index.js', | ||
}, | ||
}); |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here goes a super wired error: https://github.com/opensearch-project/dashboards-reporting/actions/runs/7228046528/job/19696789121?pr=262#step:17:154
I can reproduce it on my local only during the first run of yarn osd bootstrap . And if I re-run it after it fails, then it works again..
So in the current commit, I added max 3 re-tries for the bootstrap step, and it got the CI works.. but in the 1st try, you can see the exactly the same error: https://github.com/opensearch-project/dashboards-reporting/actions/runs/7228507642/job/19698061142?pr=262#step:17:166
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Transferring some of the conversation between me and @ps48 here:
We think it is fine to do this CI work here first and then we address the above bootstrap issue, since we have a current workaround on that, so that it is not a hard blocker. I will file an issue to track that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tracking issue is here: #263