wget receives 403 during bootstrap

[thoward27]

While trying to run ./install_software.sh on my new EC2 nodes, I am seeing 403 errors.

[centos@ip-10-0-3-64 ~]$ ./install_software.sh '2.13.2.0'
~/yugabyte-db ~
Setting appropriate YB ulimits..
installing wget
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: download.cf.centos.org
 * extras: download.cf.centos.org
 * updates: download.cf.centos.org
Package wget-1.14-18.el7_6.1.x86_64 already installed and latest version
Nothing to do
Fetching package https://downloads.yugabyte.com/yugabyte-2.13.2.0-linux.tar.gz...
--2022-06-11 15:21:40--  https://downloads.yugabyte.com/yugabyte-2.13.2.0-linux.tar.gz
Resolving downloads.yugabyte.com (downloads.yugabyte.com)... 172.66.42.235, 172.66.41.21, 2606:4700:3108::ac42:2aeb, ...
Connecting to downloads.yugabyte.com (downloads.yugabyte.com)|172.66.42.235|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2022-06-11 15:21:41 ERROR 403: Forbidden.

I am only seeing this because I removed the -q flag from the ./install_software.sh script.

[thoward27]

Relevant snippet from the install script

###############################################################################
#
# This is a simple script to install yugabyte-db software on a machine.
#
###############################################################################
YB_VERSION=$1
YB_HOME=/home/${USER}/yugabyte-db
YB_PACKAGE_URL="https://downloads.yugabyte.com/yugabyte-${YB_VERSION}-linux.tar.gz"
YB_PACKAGE_NAME="${YB_PACKAGE_URL##*/}"

###############################################################################
# Create the necessary directories.
###############################################################################
mkdir -p ${YB_HOME}/yb-software
mkdir -p ${YB_HOME}/master/conf
mkdir -p ${YB_HOME}/tserver/conf

# Save the current directory.
pushd ${YB_HOME}

###############################################################################
# Set appropriate ulimits according to https://docs.yugabyte.com/latest/deploy/manual-deployment/system-config/#setting-ulimits
###############################################################################
echo "Setting appropriate YB ulimits.."

cat > /tmp/99-yugabyte-limits.conf <<EOF
$USER   soft    core    unlimited
$USER   hard    core    unlimited
$USER   soft    data    unlimited
$USER   hard    data    unlimited
$USER   soft    priority        0
$USER   hard    priority        0
$USER   soft    fsize   unlimited
$USER   hard    fsize   unlimited
$USER   soft    sigpending      119934
$USER   hard    sigpending      119934
$USER   soft    memlock 64
$USER   hard    memlock 64
$USER   soft    nofile  1048576
$USER   hard    nofile  1048576
$USER   soft    stack   8192
$USER   hard    stack   8192
$USER   soft    rtprio  0
$USER   hard    rtprio  0
$USER   soft    nproc   12000
$USER   hard    nproc   12000
EOF

sudo cp /tmp/99-yugabyte-limits.conf /etc/security/limits.d/99-yugabyte-limits.conf
###############################################################################
# Download and install the software.
###############################################################################
echo "installing wget"
sudo yum install -y wget
echo "Fetching package $YB_PACKAGE_URL..."
wget $YB_PACKAGE_URL

[thoward27]

If I copy/paste the same command into a local terminal, I see the same output

11:23 ❯ wget https://downloads.yugabyte.com/yugabyte-2.13.2.0-linux.tar.gz
--2022-06-11 11:23:37--  https://downloads.yugabyte.com/yugabyte-2.13.2.0-linux.tar.gz
Resolving downloads.yugabyte.com (downloads.yugabyte.com)... 172.66.41.21, 172.66.42.235
Connecting to downloads.yugabyte.com (downloads.yugabyte.com)|172.66.41.21|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2022-06-11 11:23:37 ERROR 403: Forbidden.

[thoward27]

This is related to this issue: yugabyte/utilities#30