Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update info about what happens when not specifying a branch when running script that deploys Dataverse on AWS #355

Open
jggautier opened this issue May 24, 2024 · 22 comments

Comments

@jggautier
Copy link

jggautier commented May 24, 2024

If we don't specify a branch with the -b option, the script for spinning up an instance of Dataverse on AWS doesn't default to the develop branch but to the latest version of Dataverse, such as v6.2 if that's the latest version.

In order to correct what users are told about what happens when -b is not used when running the script for creating a Dataverse instance on AWS, I'm proposing in a PR I'll open a bit that we remove the line * the default branch is the develop from the documentation at https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/tree/develop/ec2.

Running the script with the -h option also tells the user that the develop branch is used when the -b option isn't used to specify a branch. This needs to be up updated, too, but I'm not sure how to do that. Does the usage part of the ec2-create-instance.sh script need to be changed?

The documentation in the Dataverse Developers Guide is also being updated. See IQSS/dataverse#10572 and conversation in the PR at IQSS/dataverse#10589.

@jggautier
Copy link
Author

Scratch that, I can't create a branch in this repo in order to create a PR to remove the line * the default branch is the develop from the documentation at https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/tree/develop/ec2.

Could someone else do this? @donsizemore or @pdurbin maybe?

@pallinger
Copy link
Contributor

pallinger commented May 28, 2024

But you can create a pull request, you just need to fork the repo, and create your own branch there. Or not even create your own branch and create a PR from your develop branch!

@pallinger
Copy link
Contributor

pallinger commented May 28, 2024

Moreover, as far as I see in ec2-create-instance.sh:

BRANCH_DEFAULT="develop"

so it should deploy the develop branch. Which may say about itself, that it is v6.2. You should actually login to the ec2 instance, and see the checked-out dataverse directory and see which branch is checked out. I unfortunately cannot try this out, as I do not have an aws subscription.

@jggautier
Copy link
Author

jggautier commented May 28, 2024

Thanks for looking into what should be deployed by default when the -b option isn't included when we run ec2-create-instance.sh.

On May 17 my colleague @landreev created an instance this way, that is without including -b, and we saw "v 6.2 build 1603-a218417" on the bottom right of the webpage where the Dataverse version is indicated. Is that what you mean by seeing the checked-out dataverse directory?

v6.2 was released on April 2, a month before that instance was created. And the develop branch doesn't really have a version number yet, right? So would we really expect to see v6.2 at the bottom of an instance that would be based on the develop branch?

@donsizemore
Copy link
Member

@jggautier my addled memory is hazy, but due to Ansible's group_var priorities, if you give the script a -g (group_vars file) and a -b (Dataverse branch), group_vars will win out? I forget. I haven't had time to dig into this or anything fun, but pull requests are most welcome.

@jggautier
Copy link
Author

jggautier commented May 28, 2024

Hi @donsizemore. I'm not sure if the group_vars will win out but I can try to find out.

I haven't had luck spinning up instances, even when running ec2-create-instance.sh alone (without -b, -g or any other options), which is why @landreev created the instance on May 17, (which I think he did without -g and -b).

@donsizemore
Copy link
Member

@jggautier do you have the current version of the script? (and how far do you get? error message?)

@jggautier
Copy link
Author

Hey @donsizemore. Yeah I always sync my copy of the dataverse-ansible repo so that I have the latest version of the script.

When I run the script with no arguments, it seems to finish. At the end it tells me the URL of the instance, how to ssh into it and how to delete it, but the URL leads to a page that says that "This site can’t be reached" and it "refused to connect."

Before that, the last error message I see is:

rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/8e85887f-758c-11ee-a904-2a65a1af8551/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(1404) [receiver=2.6.9]

And before that I see other errors like NO MORE HOSTS LEFT, "ERROR: compilation failed for package ‘DescTools’" and failed: [localhost] (item=DescTools). The last two seem similar to what @pdurbin wrote in #357.

There are other things that looks like errors and warnings. I can Slack you the complete output if you'd like.

@pallinger
Copy link
Contributor

we saw "v 6.2 build 1603-a218417"

a218417 is really the commit for the v6.2 tag, so that is really not develop.
So the script has a bug, we should fix it!
After inspecting ec2/ec2-create-instance.sh I cannot really find where v6.2 would come from. It seems to me that dataverse_branch is correctly overridden on the ansible-playbook command line. This needs a thorough debugging...

@donsizemore
Copy link
Member

@jggautier you're seeing this: IQSS/dataverse#10452 which is dependent on DescTools being fixed upstream (and likely won't happen any time soon).

I suggest switching to Rocky 9: https://raw.githubusercontent.com/GlobalDataverseCommunityConsortium/dataverse-ansible/243_rocky_9/ec2/ec2-create-instance.sh (which will be updated to 9.4 once AWS publishes Rocky's new images)

@jggautier
Copy link
Author

a218417 is really the commit for the v6.2 tag, so that is really not develop.
So the script has a bug, we should fix it!

@pallinger I've been assuming that we intend for the instance to use the latest released version of Dataverse when no branch is specified. So it's the documentation that's wrong and not a bug with the script. Does that make sense? That's why this GitHub issue is about updating documentation to match what's happening and not about changing what's happening.

Do you think the develop branch should be used when the script is run with no arguments?

@donsizemore and @pdurbin what you think?

I don't mind either way, as long as it's documented of course.

@jggautier you're seeing this: IQSS/dataverse#10452 which is dependent on DescTools being fixed upstream (and likely won't happen any time soon).

I suggest switching to Rocky 9: https://raw.githubusercontent.com/GlobalDataverseCommunityConsortium/dataverse-ansible/243_rocky_9/ec2/ec2-create-instance.sh (which will be updated to 9.4 once AWS publishes Rocky's new images)

Ah okay. Thanks @donsizemore. I suppose that's what @landreev did? I'll try running the script you linked to without any arguments and if it works see what version of Dataverse is used.

@donsizemore
Copy link
Member

@jggautier I bet with no arguments it's finding the default main.yml - I remember some screwiness around this, but it's years past. As a workaround, passing it a group_vars file should always work.

@pallinger
Copy link
Contributor

pallinger commented May 30, 2024

@jggautier

Do you think the develop branch should be used when the script is run with no arguments?

I do not think that, but reading the script, it seems to use the develop branch by default, or the parameter after -b otherwise.
My main concern is that if it does not use the develop branch by default, then maybe the -b option also does not work...

@jggautier
Copy link
Author

I was able to spin up an instance using the script you shared @donsizemore, and v6.2 was on the bottom right. Seems like more evidence for your bet about it finding the default main.yml.

@pallinger Using -b has worked for me, although it's been a while since I've needed to use it.

I'm getting less certain about what to do now.

Should I create a PR that does the following?:

@donsizemore
Copy link
Member

@jggautier last time I ran into this I just added an exit 0 in the script before it actually did anything, which along with a few echo commands told me what I needed. Let me tinker for a minute.

@pdurbin
Copy link
Member

pdurbin commented May 30, 2024

v6.2 was on the bottom right

Was there a build number? Did it match the one from the 6.2 release?

@jggautier
Copy link
Author

jggautier commented May 30, 2024

Yeah there was more that came after the v6.2 but I don't remember what it was and if it matches the one from the v6.2 release.

I could spin up an instance again and see.

"build 1603-a218417" is the build number of the 6.2 release, right?

@pdurbin
Copy link
Member

pdurbin commented May 30, 2024

Yep, I unzipped the 6.2 war and it's the same build number:

% grep build.number WEB-INF/classes/BuildNumber.properties
build.number=1603-a218417

@jggautier
Copy link
Author

I spun up an instance again and saw build 1603-a218417, so it matches the one from the 6.2 release.

@pdurbin
Copy link
Member

pdurbin commented Jun 3, 2024

@donsizemore
Copy link
Member

@jggautier @pdurbin I'm finally looking at this (sorry). Looks like Ansible is being launched on the instance as /usr/bin/python3.12 /usr/bin/ansible-playbook -i dataverse/inventory dataverse/dataverse.pb --connection=local -e dataverse_branch=develop which according to https://docs.ansible.com/ansible/latest/reference_appendices/general_precedence.html#command-line-options should override the default group_vars file. Still poking.

@donsizemore
Copy link
Member

@jggautier @pdurbin I just launched the script with

$ ec2/ec2-create-instance.sh -b develop

and I'm seeing

{"version":"6.2","build":"develop-"}

but on the instance, in /opt/dataverse/dataverse I issue sudo -u dataverse git status and I see

On branch develop
Your branch is up to date with 'origin/develop'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants