Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass env variable as argument #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion acmeair/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ service/acmeair-service-1 created
# Openshift
To deploy the benchmark use `acmeair-deploy-openshift.sh`

`./scripts/acmeair-deploy-openshift.sh -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM]`
`./scripts/acmeair-deploy-openshift.sh -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]`

- **BENCHMARK_SERVER**: Name of the cluster you are using
- **SERVER_INSTANCES**: Number of acmeair instances to be deployed. It is optional, if is not specified then by default it will be considered as 1 instance.
Expand All @@ -92,6 +92,7 @@ To deploy the benchmark use `acmeair-deploy-openshift.sh`
- **MEM_REQ**: Memory request
- **CPU_LIM**: CPU limit
- **MEM_LIM**: Memory limit
- **ENV_VAR**: Environment variable

Example to deploy and run acmeair application on openshift cluster

Expand Down
8 changes: 4 additions & 4 deletions acmeair/scripts/acmeair-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
#

CURRENT_DIR="$(dirname "$(realpath "$0")")"
pushd "${CURRENT_DIR}" >> setup.log
pushd ".." >> setup.log
pushd "${CURRENT_DIR}" >> /dev/null
pushd ".." >> /dev/null

# Set the defaults for the app
JMX_FILE="${PWD}/jmeter-driver/acmeair-jmeter/scripts/AcmeAir.jmx"
Expand Down Expand Up @@ -70,7 +70,7 @@ function get_ip() {
# output:build the application from scratch and create the acmeair docker image
function build_acmeair() {
# Build acmeair monolithic application docker image
pushd acmeair >>${LOGFILE}
pushd acmeair >> /dev/null
# Build the application
docker run --rm -v "${PWD}":/home/gradle/project -w /home/gradle/project dinogun/gradle:5.5.0-jdk8-openj9 gradle build 2>>${LOGFILE} >>${LOGFILE}
err_exit "Error: gradle build of acmeair monolithic application failed."
Expand All @@ -84,7 +84,7 @@ function build_acmeair() {
# Build the acmeair driver application
function build_acmeair_driver() {
# Build acmeair driver
pushd jmeter-driver >>${LOGFILE}
pushd jmeter-driver >> /dev/null
docker run --rm -v "${PWD}":/home/gradle/project -w /home/gradle/project dinogun/gradle:5.5.0-jdk8-openj9 gradle build 2>>${LOGFILE} >>${LOGFILE}
err_exit "Error: gradle build of acmeair driver failed."
popd >>${LOGFILE}
Expand Down
11 changes: 10 additions & 1 deletion acmeair/scripts/acmeair-deploy-openshift.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ CLUSTER_TYPE="openshift"
# Describes the usage of the script
function usage() {
echo
echo "Usage: $0 -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM]"
echo "Usage: $0 -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]"
echo " "
echo "Example: $0 -s rouging.os.fyre.ibm.com -i 2 -a dinogun/acmeair-monolithic:latest --cpulim=4 --cpureq=2 --memlim=1024Mi --memreq=512Mi"
exit -1
Expand Down Expand Up @@ -78,6 +78,9 @@ do
memlim=*)
MEM_LIM=${OPTARG#*=}
;;
env=*)
ENV_VAR=${OPTARG#*=}
;;
*)
esac
;;
Expand Down Expand Up @@ -159,6 +162,12 @@ function createInstances() {
sed -i '/limits:/a \ \ \ \ \ \ \ \ \ \ cpu: '${CPU_LIM}'' ${MANIFESTS_DIR}/acmeair-${inst}.yaml
fi

# Pass environment variables
if [ ! -z ${ENV_VAR} ]; then
sed -i '/env:/a \ \ \ \ \ \ \ \ - name: "JVM_ARGS"' ${MANIFESTS_DIR}/acmeair-${inst}.yaml
sed -i '/- name: "JVM_ARGS"/a \ \ \ \ \ \ \ \ \ \ value: '${ENV_VAR}'' ${MANIFESTS_DIR}/acmeair-${inst}.yaml
fi

oc create -f ${MANIFESTS_DIR}/acmeair-${inst}.yaml -n ${NAMESPACE}
err_exit "Error: Issue in deploying."
((ACMEAIR_PORT=ACMEAIR_PORT+1))
Expand Down
3 changes: 2 additions & 1 deletion acmeair/scripts/perf/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Test with multiple instances

`./scripts/perf/run_acmeair_openshift.sh -s BENCHMARK_SERVER -e RESULTS_DIR_PATH [-u JMETER_LOAD_USERS] [-d JMETER_LOAD_DURATION] [-w WARMUPS] [-m MEASURES] [-i TOTAL_INST] [--iter=TOTAL_ITR] [-r= set redeploy to true] [-n NAMESPACE] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM]`
`./scripts/perf/run_acmeair_openshift.sh -s BENCHMARK_SERVER -e RESULTS_DIR_PATH [-u JMETER_LOAD_USERS] [-d JMETER_LOAD_DURATION] [-w WARMUPS] [-m MEASURES] [-i TOTAL_INST] [--iter=TOTAL_ITR] [-r= set redeploy to true] [-n NAMESPACE] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]`

- **BENCHMARK_SERVER_NAME** : Name of the cluster you are using
- **RESULTS_DIR_PATH** : Location where you want to store the results
Expand All @@ -18,6 +18,7 @@
- **MEM_REQ**: Memory request
- **CPU_LIM**: CPU limit
- **MEM_LIM**: Memory limit
- **ENV_VAR**: Environment variable

Example to test with multiple instances

Expand Down
13 changes: 8 additions & 5 deletions acmeair/scripts/perf/run_acmeair_openshift.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@
# Ex of ARGS : -s wobbled.os.fyre.ibm.com default -e /acmeair/results -u 400 -d 300 -w 5 -m 3

CURRENT_DIR="$(dirname "$(realpath "$0")")"
pushd "${CURRENT_DIR}" >> setup.log
pushd ".." >> setup.log
pushd "${CURRENT_DIR}" >> /dev/null
pushd ".." >> /dev/null
SCRIPT_REPO=${PWD}
pushd ".." >> setup.log
pushd ".." >> /dev/null
JMX_FILE="${PWD}/jmeter-driver/acmeair-jmeter/scripts/AcmeAir.jmx"
ACMEAIR_DEFAULT_IMAGE="dinogun/acmeair-monolithic"
LOG_FILE="${PWD}/logs/jmeter.log"
Expand All @@ -33,7 +33,7 @@ LOGFILE="${PWD}/logs/setup.log"
# Describes the usage of the script
function usage() {
echo
echo "Usage: $0 -s BENCHMARK_SERVER -e RESULTS_DIR_PATH [-u JMETER_LOAD_USERS] [-d JMETER_LOAD_DURATION] [-w WARMUPS] [-m MEASURES] [-i TOTAL_INST] [--iter=TOTAL_ITR] [-r= set redeploy to true] [-n NAMESPACE] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] "
echo "Usage: $0 -s BENCHMARK_SERVER -e RESULTS_DIR_PATH [-u JMETER_LOAD_USERS] [-d JMETER_LOAD_DURATION] [-w WARMUPS] [-m MEASURES] [-i TOTAL_INST] [--iter=TOTAL_ITR] [-r= set redeploy to true] [-n NAMESPACE] [-a ACMEAIR_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]"
exit -1
}

Expand All @@ -58,6 +58,9 @@ do
memlim=*)
MEM_LIM=${OPTARG#*=}
;;
env=*)
ENV_VAR=${OPTARG#*=}
;;
*)
esac
;;
Expand Down Expand Up @@ -553,7 +556,7 @@ function runIterations() {
for (( itr=0; itr<${TOTAL_ITR}; itr++ ))
do
if [ ${RE_DEPLOY} == "true" ]; then
${SCRIPT_REPO}/acmeair-deploy-openshift.sh -s ${BENCHMARK_SERVER} -n ${NAMESPACE} -i ${SCALING} -a ${ACMEAIR_IMAGE} --cpureq=${CPU_REQ} --memreq=${MEM_REQ} --cpulim=${CPU_LIM} --memlim=${MEM_LIM} >> setup.log
${SCRIPT_REPO}/acmeair-deploy-openshift.sh -s ${BENCHMARK_SERVER} -n ${NAMESPACE} -i ${SCALING} -a ${ACMEAIR_IMAGE} --cpureq=${CPU_REQ} --memreq=${MEM_REQ} --cpulim=${CPU_LIM} --memlim=${MEM_LIM} --env=${ENV_VAR} >> setup.log
err_exit "Error: acmeair deployment failed"
fi
# Start the load
Expand Down
48 changes: 47 additions & 1 deletion galaxies/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ service/galaxies-service-1 created
# Openshift
To deploy the benchmark use `galaxies-deploy-openshift.sh`

`./scripts/galaxies-deploy-openshift.sh -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-g GALAXIES_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM]`
`./scripts/galaxies-deploy-openshift.sh -s BENCHMARK_SERVER [-n NAMESPACE] [-i SERVER_INSTANCES] [-g GALAXIES_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]`

- **BENCHMARK_SERVER**: Name of the cluster you are using
- **SERVER_INSTANCES**: Number of galaxies instances to be deployed. It is optional, if is not specified then by default it will be considered as 1 instance.
Expand All @@ -85,6 +85,7 @@ To deploy the benchmark use `galaxies-deploy-openshift.sh`
- **MEM_REQ**: Memory request
- **CPU_LIM**: CPU limit
- **MEM_LIM**: Memory limit
- **ENV_VAR**: Environment variable

Example to deploy and run galaxies application on openshift cluster

Expand All @@ -103,6 +104,51 @@ route.route.openshift.io/galaxies-service-1 exposed

```

# Run the load
Simulating the load on galaxies benchmarks using hyperfoil/wrk
`./scripts/galaxies-load.sh -c CLUSTER_TYPE [-i SERVER_INSTANCES] [--iter MAX_LOOP] [-n NAMESPACE] [-a IP_ADDR] [-t THREAD] [-R REQUEST_RATE] [-d DURATION] [--connection=CONNECTIONS]`

- **CLUSTER_TYPE**: docker|minikube|openshift
- **SERVER_INSTANCES**: Number of galaxies instances to which you want to run the load. It is optional, if is not specified then by default it will be considered as 1 instance.
- **MAX_LOOP**: Number of times you want to run the load. It is optional, if is not specified then by default it will be considered as 5 iterations.
- **NAMESPACE**: Namespace in which galaxies application is deployed(Required only in the case of openshift cluster and if the application is deployed in other namespaces except `openshift-monitoring`)
- **IP_ADDR**: IP address of the machine. It is optional, if it is not specified then the get_ip function written inside the script will get the IP address of the machine.
- **THREAD**: Number of threads
- **REQUEST_RATE**: Requests rate
- **DURATION**: Test duration
- **CONNECTIONS**: Number of connections

Example to run the load on 1 galaxies instances for 2 iterations in openshift cluster

**`$./scripts/galaxies-load.sh -c docker --iter=2`**
```
#########################################################################################
Starting Iteration 1
#########################################################################################

Running wrk load for galaxies instance 1 with the following parameters
CMD=./wrk2.sh --threads=10 --connections=700 --duration=60s --rate=2000 http://192.168.122.105:32000/galaxies
wrk logs Dir : /home/shruthi/galaxies-12-mar/benchmarks/galaxies/logs/galaxies-2021-03-12:12:30

#########################################################################################
Starting Iteration 2
#########################################################################################

Running wrk load for galaxies instance 1 with the following parameters
CMD=./wrk2.sh --threads=20 --connections=700 --duration=60s --rate=2000 http://192.168.122.105:32000/galaxies
wrk logs Dir : /home/shruthi/galaxies-12-mar/benchmarks/galaxies/logs/galaxies-2021-03-12:12:30
#########################################################################################
Displaying the results
#########################################################################################
RUN, THROUGHPUT, RESPONSE_TIME, MAX_RESPONSE_TIME, STDDEV_RESPONSE_TIME, ERRORS
1, 2008.90, 1.31, 150.99, 4.40, 0
2, 2001.05, 939.76, 84.93, 1.84, 0

```
Above image shows the logs of the load run, it processes and displays the output for each run. See Displaying the results section of the log for information about throughput, Number of pages it has retreived, average response time and errors if any.

To run the load for multiple instances in case of openshift cluster follow [README.md](/galaxies/scripts/perf/README.md)

# Cleanup
`$ ./scripts/galaxies-cleanup.sh -c CLUSTER_TYPE[docker|minikube|openshift] [-n NAMESPACE]`

Expand Down
1 change: 1 addition & 0 deletions galaxies/manifests/galaxies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ spec:
- name: galaxies
image: dinogun/galaxies:1.2-jdk-11.0.10_9
imagePullPolicy: IfNotPresent
env:
resources:
requests:
limits:
Expand Down
73 changes: 65 additions & 8 deletions galaxies/scripts/galaxies-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
### Script containing common functions ###

CURRENT_DIR="$(dirname "$(realpath "$0")")"
pushd "${CURRENT_DIR}" >> setup.log
pushd ".." >> setup.log
pushd "${CURRENT_DIR}" > /dev/null
pushd ".." > /dev/null

# Set the defaults for the app
export GALAXIES_PORT="32000"
Expand All @@ -42,6 +42,30 @@ function err_exit() {
fi
}

# Check if java is installed and it is of version 11 or newer
function check_load_prereq() {
echo
echo -n "Info: Checking prerequisites..." >> ${LOGFILE}
# check if java exists
if [ ! `which java` ]; then
echo " "
echo "Error: java is not installed."
exit 1
else
JAVA_VER=$(java -version 2>&1 >/dev/null | egrep "\S+\s+version" | awk '{print $3}' | tr -d '"')
case "${JAVA_VER}" in
1[1-9].*.*)
echo "done" >> ${LOGFILE}
;;
*)
echo " "
echo "Error: Hyperfoil requires Java 11 or newer and current java version is ${JAVA_VER}"
exit 1
;;
esac
fi
}

# Check for all the prereqs
function check_prereq() {
docker version > ${LOGFILE} 2>&1
Expand Down Expand Up @@ -69,7 +93,7 @@ function check_app() {
for status in "${CMD[@]}"
do
if [ -z "${status}" ]; then
echo "Application pod did not come up"
echo "Application pod did not come up"
exit -1;
fi
done
Expand All @@ -84,11 +108,11 @@ function build_galaxies() {
err_exit "Error: Building of docker image of galaxies."
}

# Run the galaxies application
# Run the galaxies application
# input:galaxies image to be used and JVM arguments if any
# output:Create network bridge "kruize-network" and run galaxies application container on the same network
function run_galaxies() {
GALAXIES_IMAGE=$1
GALAXIES_IMAGE=$1
INST=$2

# Create docker network bridge "kruize-network"
Expand All @@ -98,14 +122,47 @@ function run_galaxies() {
echo "Creating Kruize network: ${NETWORK}..."
docker network create --driver bridge ${NETWORK} 2>>${LOGFILE} >>${LOGFILE}
else
echo "${NETWORK} already exists..."
echo "${NETWORK} already exists..."
fi

# Run the galaxies app container on "kruize-network"
cmd="docker run -d --name=galaxies-app-${INST} -p ${GALAXIES_PORT}:8080 --network=${NETWORK} ${GALAXIES_IMAGE} "
${cmd}s 2>>${LOGFILE} >>${LOGFILE}
cmd="docker run -d --name=galaxies-app-${INST} -p ${GALAXIES_PORT}:8080 --network=${NETWORK} ${GALAXIES_IMAGE}"
${cmd} 2>>${LOGFILE} >>${LOGFILE}
err_exit "Error: Unable to start galaxies container"
((GALAXIES_PORT=GALAXIES_PORT+1))
# Check if the application is running
check_app
}

# Parse the Throughput Results
# input:result directory , Number of iterations of the wrk load
# output:Throughput log file (tranfer per sec, Number of requests per second, average latency and errors if any)
function parse_galaxies_results() {
RESULTS_DIR=$1
TOTAL_LOGS=$2
echo "RUN, THROUGHPUT, RESPONSE_TIME, MAX_RESPONSE_TIME, STDDEV_RESPONSE_TIME, ERRORS" > ${RESULTS_DIR}/Throughput.log
for (( iteration=1 ; iteration<=${TOTAL_LOGS} ;iteration++))
do
RESULT_LOG=${RESULTS_DIR}/wrk-${inst}-${iteration}.log
throughput=`cat ${RESULT_LOG} | grep "Requests" | cut -d ":" -f2 `
responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f2 `
max_responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f6 `
stddev_responsetime=`cat ${RESULT_LOG} | grep "Latency" | cut -d ":" -f2 | tr -s " " | cut -d " " -f4 `
weberrors=`cat ${RESULT_LOG} | grep "Non-2xx" | cut -d ":" -f2`
if [ "${weberrors}" == "" ]; then
weberrors="0"
fi
echo "${iteration}, ${throughput}, ${responsetime}, ${max_responsetime}, ${stddev_responsetime}, ${weberrors}" >> ${RESULTS_DIR}/Throughput.log
done
}

# Download the required dependencies
# output: Check if the hyperfoil/wrk dependencies is already present, If not download the required dependencies to apply the load
function load_setup(){
if [ ! -d "${PWD}/hyperfoil-0.13" ]; then
wget https://github.com/Hyperfoil/Hyperfoil/releases/download/release-0.13/hyperfoil-0.13.zip >> ${LOGFILE} 2>&1
unzip hyperfoil-0.13.zip
fi
pushd hyperfoil-0.13/bin > /dev/null
}

27 changes: 21 additions & 6 deletions galaxies/scripts/galaxies-deploy-openshift.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ CLUSTER_TYPE="openshift"
# Describes the usage of the script
function usage() {
echo
echo "Usage: $0 -s BENCHMARK_SERVER [-i SERVER_INSTANCES] [-n NAMESPACE] [-g GALAXIES_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] "
echo "Usage: $0 -s BENCHMARK_SERVER [-i SERVER_INSTANCES] [-n NAMESPACE] [-g GALAXIES_IMAGE] [--cpureq=CPU_REQ] [--memreq=MEM_REQ] [--cpulim=CPU_LIM] [--memlim=MEM_LIM] [--env=ENV_VAR]"
echo " "
echo "Example: $0 -s rouging.os.fyre.ibm.com -i 2 -g dinogun/galaxies:1.1-jdk-11.0.10_9 --cpulim=4 --cpureq=2 --memlim=1024Mi --memreq=512Mi"
echo "Example: $0 -s rouging.os.fyre.ibm.com -i 2 -g dinogun/galaxies:1.2-jdk-11.0.10_9 --cpulim=4 --cpureq=2 --memlim=1024Mi --memreq=512Mi"
exit -1
}

Expand Down Expand Up @@ -72,16 +72,15 @@ do
;;
memreq=*)
MEM_REQ=${OPTARG#*=}
# check memory request for unit
check_memory_unit ${MEM_REQ}
;;
cpulim=*)
CPU_LIM=${OPTARG#*=}
;;
memlim=*)
MEM_LIM=${OPTARG#*=}
# check memory limit for unit
check_memory_unit ${MEM_LIM}
;;
env=*)
ENV_VAR=${OPTARG#*=}
;;
*)
esac
Expand Down Expand Up @@ -119,6 +118,16 @@ if [ -z "${NAMESPACE}" ]; then
NAMESPACE="${DEFAULT_NAMESPACE}"
fi

# check memory limit for unit
if [ ! -z "${MEM_LIM}" ]; then
check_memory_unit ${MEM_LIM}
fi

# check memory request for unit
if [ ! -z "${MEM_REQ}" ]; then
check_memory_unit ${MEM_REQ}
fi

# Create multiple yamls based on instances and Update the template yamls with names and create multiple files
# input:galaxies and service-monitor yaml file
function createInstances() {
Expand Down Expand Up @@ -154,6 +163,12 @@ function createInstances() {
sed -i '/limits:/a \ \ \ \ \ \ \ \ \ \ cpu: '${CPU_LIM}'' ${MANIFESTS_DIR}/galaxies-${inst}.yaml
fi

# Pass environment variables
if [ ! -z ${ENV_VAR} ]; then
sed -i '/env:/a \ \ \ \ \ \ \ \ - name: "JVM_ARGS"' ${MANIFESTS_DIR}/galaxies-${inst}.yaml
sed -i '/- name: "JVM_ARGS"/a \ \ \ \ \ \ \ \ \ \ value: '${ENV_VAR}'' ${MANIFESTS_DIR}/galaxies-${inst}.yaml
fi

oc create -f ${MANIFESTS_DIR}/galaxies-${inst}.yaml -n ${NAMESPACE}
err_exit "Error: Issue in deploying."
((GALAXIES_PORT=GALAXIES_PORT+1))
Expand Down
Loading