From a41d1c2bc5511f1b4bd52a332a81a0d955220b08 Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Sat, 15 Jul 2023 11:40:15 +0200 Subject: [PATCH 1/8] Update Readme.md --- Readme.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Readme.md b/Readme.md index 6b61f75..152cc83 100644 --- a/Readme.md +++ b/Readme.md @@ -16,7 +16,7 @@ # Docker Tags # tag | Description ----------------|------------------------------------------------------------------- -22.4.19 | This is the latest based on GVMd 22.5.3 available on x86_64, arm64, and armv7. +22.4.20 | This is the latest based on GVMd 22.5.4 available on x86_64, arm64, and armv7. 21.04.09 | This is the last 21.4 build. 20.08.04.6 | The last 20.08 image pre-20.08 | This is the last image from before the 20.08 update. @@ -27,6 +27,13 @@ v1.0 | old out of date image for posterity. (Dont` use this one. . . ## Documentation ## The current docs are maintained on github [here](https://immauss.github.io/openvas/) - - - - +# 13 July 2023 # +## 22.4.20 ## + +GB pushed a new gmvd (22.5.4) that resolved some issues with PostgreSQL connections. + +-Scott +- - - - # 9 July 2023 # ## 22.4.19 ## From c6b87e75a78d6dbd23adf831eac1d0492ad7a7eb Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Sat, 15 Jul 2023 11:46:52 +0200 Subject: [PATCH 2/8] Update Readme.md grammar and spelling fixes --- docs/Readme.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/Readme.md b/docs/Readme.md index 5477069..76ef621 100644 --- a/docs/Readme.md +++ b/docs/Readme.md @@ -17,25 +17,25 @@ The Greenbone Source code can be found at: [Greenbone Source Code](https://github.com/greenbone) The advantages of the Immauss container image vs the Greenbone images: -- Able to run a full scanner in a sinlge image with or without volumes. +- Able to run a full scanner in a single image with or without volumes. - Image contains a full database. -- Speed to scanning. The Immauss image can be up and scanning in 15-20 minutes. ( With sufficent machine resources). +- Speed to scanning. The Immauss image can be up and scanning in 15-20 minutes. ( With sufficient machine resources). - The image on docker hub is updated weekly to ensure the database is up to date. -The the latest image is based on GVM 22.4.x In single container mode, it runs all the components needed to create a scanner in a single container including: -- gvmd - the Greenbone Vulnerability Managedment daemon +The latest image is based on GVM 22.5.x In single container mode, it runs all the components needed to create a scanner in a single container including: +- gvmd - the Greenbone Vulnerability Management daemon - openvas scanner - the scanner component of GVM -- ospd - the openvas scanner protocal daemon +- ospd - the openvas scanner protocol daemon - notusscanner - the new piece from Greenbone that handles the local scans of machines. - postgresql - the database backend for the scanner and gvm -- redis - in memory database store used by gvmd +- redis - in-memory database store used by gvmd - postfix mail server for delivering email notices from GVM - A copy of the baseline data feeds and associated database -- Option to restore from existing postgresql database dump +- Option to restore from existing Postgresql database dump - Option to skip the data sync on startup - Proper database shutdown on container stop to prevent db corruption. -In multi-container mode it creates individual containers for each of the components. Since most of the Greenbone components utlize unix sockets for comunication, the contianers share a volume (the default name is: ovasrun) soley for the sharing of the sokets.` +In multi-container mode, it creates individual containers for each of the components. Since most of the Greenbone components utilize unix sockets for communication, the containers share a volume (the default name is: ovasrun) solely for the sharing of the sockets.` ## Deployment @@ -83,7 +83,7 @@ There is also a script in the container that will initiate the sync. ``` /scripts/sync.sh ``` -You can run the sync at anytime on a running container with: +You can run the sync at any time on a running container with: ``` docker exec -it /scripts/sync.sh ``` @@ -94,9 +94,9 @@ docker exec -it /scripts/sync.sh - /compose/docker-compose.yml - /multi-container/docker-compose.yml - The 'yml' in /compose is a single container immplementation. The 'yml' in /multi-container is for .... multiple containers. Both utilize a '.env" file. You can set the docker tag in the ".env" file. + The 'yml' in /compose is a single container implementation. The 'yml' in /multi-container is for .... multiple containers. Both utilize a '.env" file. You can set the docker tag in the ".env" file. - To utilze the docker-compose.yml files, change to the desired directory and run: + To utilize the docker-compose.yml files, change to the desired directory and run: ``` docker-compose up -d ``` @@ -105,7 +105,7 @@ docker-compose up -d docker-compose up -d ``` -* For upgrades from major versions, ensure you are using the most recent docker-compose.yml for the git repo. For instance, from 21.4 -> 22.4, the notus scanner was added. If you do not utilize the new docker-compose.yml with the mulit-container "yml", then there will be no container with the "notuscanner". * +* For upgrades from major versions, ensure you are using the most recent docker-compose.yml for the git repo. For instance, from 21.4 -> 22.4, the notus scanner was added. If you do not utilize the new docker-compose.yml with the multi-container "yml", then there will be no container with the "notuscanner". * # Database backup @@ -132,7 +132,7 @@ docker run -it -e RESTORE=true -v :/usr/lib/db-backup.sql -- # Full backup -There are a number of crucial items not stored in the database such as encryption keys for credentials, SSL certificates etc. All of these will however be stored on the persitent volume located in /data of the container filesystem. The easiest way to backup the entireity of the volume is shutdown the openvas container and use a new container to create the backup. This is the safest way to create the backup to ensure no files are changed during the backup process. The below commands assume a container name of openvas-prod and a volume name of openvas. +There are a number of crucial items not stored in the database such as encryption keys for credentials, SSL certificates etc. All of these will however be stored on the persistent volume located in /data of the container filesystem. The easiest way to backup the entirety of volume is shutdown the openvas container and use a new container to create the backup. This is the safest way to create the backup to ensure no files are changed during the backup process. The below commands assume a container name of openvas-prod and a volume name of openvas. **Stop the running container** ``` @@ -146,7 +146,7 @@ docker run -it --rm -v openvas:/opt -v $(pwd):/mnt alpine /bin/sh -c "cd /opt; t ``` docker start openvas-prod ``` -* Note: alpine is very lightweight linux container which is well suited for this purpose. +* Note: alpine is a very lightweight Linux container which is well-suited for this purpose. # Full restoral @@ -161,7 +161,7 @@ docker run --rm -it -v :/backup.tar.gz -v openvas:/mnt alpi ``` # Options -The following options can be set as environement variables when starting the container. To set an environement variable use "-e": +The following options can be set as environment variables when starting the container. To set an environment variable use "-e": - USERNAME : Use a different default username. Default = admin ``` @@ -172,10 +172,10 @@ The following options can be set as environement variables when starting the con -e PASSWORD='' ``` ## Important note about USERNAME and PASSWORD -**You should only use these for initial setup of the container. Always change the password aftewards. If you start the container from the command line with the PASSWORD env set, then the password is readily readible in your command history and in /proc etc ....** +**You should only use these for the initial setup of the container. Always change the password afterward. If you start the container from the command line with the PASSWORD env set, then the password is readily readable in your command history and in /proc etc ....** **If you choose to create a new user at startup, the "admin" user will still exist with the default admin password. The admin user is needed as it is the owner of the "feed import process" and gvmd will not let it be deleted. Make sure you change the password for admin in this scenario. you have been warned. :) -- RELAYHOST : The IP address or hostname of the email relay to send emails through. Default = 172.17.01 (This is default for the docker host. If you are running the mail relay on your docker host, this should work, but you will need to make sure you allow the conections through the host`s firewall/iptables) +- RELAYHOST : The IP address or hostname of the email relay to send emails through. Default = 172.17.01 (This is default for the docker host. If you are running the mail relay on your docker host, this should work, but you will need to make sure you allow the connections through the host`s firewall/iptables) ``` -e RELAYHOST=mail.example.com ``` @@ -183,7 +183,7 @@ you have been warned. :) ``` -e SMTPPORT=25 ``` -- REDISDBS : Number or redis databases to allow. (This was specific user request. In somecases, when running scans against a large number of targets, the default can be low and increasing the nubmer of redis databases can improve scan performance.) Default = 512 +- REDISDBS : Number or redis databases to allow. (This was a specific user request. In some cases, when running scans against a large number of targets, the default can be low, and increasing the number of redis databases can improve scan performance.) Default = 512 ``` -e REDISDBS=512 ``` @@ -191,11 +191,11 @@ you have been warned. :) ``` -e QUIET=true ``` -- SKIPSYNC : If you would prefer to skip the data feed synchronizations on container start, then set this to true. Thils will get the scanner operational faster, at the cost of using what might be slightly out of date NVTs. Default = false +- SKIPSYNC : If you would prefer to skip the data feed synchronizations on container start, then set this to true. This will get the scanner operational faster, at the cost of using what might be slightly out-of-date NVTs. Default = false ``` -e SKIPSYNC=true ``` -- RESTORE : Set this to true to in order to use the database restore function. After the db is restored, the container will exit. This is to prevent the possiblity of container restart with the RESTORE option still set which would again restore the DB from the backup file. (See Restore section above for more details) Default = false +- RESTORE : Set this to true to in order to use the database restore function. After the db is restored, the container will exit. This is to prevent the possibility of container restart with the RESTORE option still set which would again restore the DB from the backup file. (See the Restore section above for more details) Default = false ``` -e RESTORE=true ``` From 173c10f1e0d74535514afc56c64cb5bd9a5b17de Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Sat, 15 Jul 2023 12:33:38 +0200 Subject: [PATCH 3/8] Update Readme.md added connections to port 9390 to docs. --- docs/Readme.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/Readme.md b/docs/Readme.md index 76ef621..3bfe0fa 100644 --- a/docs/Readme.md +++ b/docs/Readme.md @@ -159,6 +159,11 @@ Then extract the backup into the volume with alpine. ``` docker run --rm -it -v :/backup.tar.gz -v openvas:/mnt alpine /bin/sh -c "cd /mnt; tar xvf /backup.tar.gz" ``` +# Scripting + +It's possible to interact with gvmd via custom scripts and scripts provided by Greenbone. If you are running openvas/gvmd on a single machine, then the scripts will usually communicate with gvmd via socket. With it running in the container, you would need to copy the scripts into the container to run them. Alternatively, your scripts can communicate with gvmd via tcp on port 9390. The easiest way to do this is to tell docker to expose the port by adding ''' -p 9390:9390 ''' to the start command. This forwards port 9390 on your host machine to port 9390 on the container. Now you should be able to send the connection to the IP of your host computer and and script will connect to the container. + +Alternatively, you could start a new container with the scripts installed on the same docker network, and have the script connect to the container IP on port 9390. # Options The following options can be set as environment variables when starting the container. To set an environment variable use "-e": From 8c24c25cdaf873d2b7fc3a1098f83d14badceb75 Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Fri, 21 Jul 2023 16:26:23 +0200 Subject: [PATCH 4/8] Update Readme.md --- Readme.md | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/Readme.md b/Readme.md index 152cc83..b631108 100644 --- a/Readme.md +++ b/Readme.md @@ -16,17 +16,46 @@ # Docker Tags # tag | Description ----------------|------------------------------------------------------------------- -22.4.20 | This is the latest based on GVMd 22.5.4 available on x86_64, arm64, and armv7. +22.4.21 | This is the latest based on GVMd 22.5.4 available on x86_64, arm64, and armv7. 21.04.09 | This is the last 21.4 build. 20.08.04.6 | The last 20.08 image pre-20.08 | This is the last image from before the 20.08 update. v1.0 | old out of date image for posterity. (Dont` use this one. . . . ever) +# Greenbone Versions in Latest image: # +Component | Version +----------|---------- +|pg_gvm|v22.5.1| +|notus_scanner|v22.5.0| +|gvmd|v22.5.5| +|openvas|v22.7.3| +|openvas_smb|v22.5.3| +|gvm_libs|v22.6.3| +|openvas_scanner|v22.7.3| +|gsad|v22.5.1| +|gsa|v22.5.0| +|ospd|v21.4.4| +|ospd_openvas|v22.5.3| +|python_gvm|v23.5.1| +|gvm_tools|v23.4.0| +|greenbone_feed_sync|v23.7.0| + - - - - ## Documentation ## -The current docs are maintained on github [here](https://immauss.github.io/openvas/) +The current container docs are maintained on github [here](https://immauss.github.io/openvas/) + +For docs on the web interface and scanning, use Greenbone's docs [here](https://docs.greenbone.net/GSM-Manual/gos-22.04/en/). Chapter's 8-14 cover the bits you'll need. - - - - +# 21 July 2023 # +## 22.4.21 ## +It's been a busy month. This latest release updates the openvas-scanner to prevent the "out-of-date" scanner warnings. There is also currently a bug with GSA that is preventing user creation from the web interface. GB has resolved the issue, but it has not been released yet. In the interim, if you need to create a new user, use the following: +``` +docker exec -it -u gvm openvas gvmd --role="Admin" --create-user="$USERNAME" --password="$PASSWORD" +``` + +-Scott + # 13 July 2023 # ## 22.4.20 ## From 4ec48436023a585c68b98d357551bb0bb6d14ca4 Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Fri, 21 Jul 2023 17:04:12 +0200 Subject: [PATCH 5/8] Update Readme.md --- Readme.md | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/Readme.md b/Readme.md index b631108..969331e 100644 --- a/Readme.md +++ b/Readme.md @@ -23,22 +23,16 @@ pre-20.08 | This is the last image from before the 20.08 update. v1.0 | old out of date image for posterity. (Dont` use this one. . . . ever) # Greenbone Versions in Latest image: # -Component | Version -----------|---------- -|pg_gvm|v22.5.1| -|notus_scanner|v22.5.0| -|gvmd|v22.5.5| -|openvas|v22.7.3| -|openvas_smb|v22.5.3| -|gvm_libs|v22.6.3| -|openvas_scanner|v22.7.3| -|gsad|v22.5.1| -|gsa|v22.5.0| -|ospd|v21.4.4| -|ospd_openvas|v22.5.3| -|python_gvm|v23.5.1| -|gvm_tools|v23.4.0| -|greenbone_feed_sync|v23.7.0| +Component | Version | | Component | Version +----------|----------|-|----------|--------- +|pg_gvm|v22.5.1| |gvmd|v22.5.5| +|notus_scanner|v22.5.0| |openvas|v22.7.3| +|openvas_smb|v22.5.3| |gvm_libs|v22.6.3| +|openvas_scanner|v22.7.3| |gsad|v22.5.1| +|gsa|v22.5.0| |ospd|v21.4.4| +|ospd_openvas|v22.5.3| |python_gvm|v23.5.1| +|gvm_tools|v23.4.0| |greenbone_feed_sync|v23.7.0| + - - - - From 33d6a6019ae6b02bbc97be4161b70961d4674428 Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Fri, 21 Jul 2023 18:06:05 +0200 Subject: [PATCH 6/8] Updated get-gvm-releases.sh to create a versions.md with all component versions in a table. --- bin/get-gvm-releases.sh | 19 +++++++++++++++++++ build.rc | 18 +++++++++--------- compose/.env | 2 +- 3 files changed, 29 insertions(+), 10 deletions(-) diff --git a/bin/get-gvm-releases.sh b/bin/get-gvm-releases.sh index e883141..f027d5d 100755 --- a/bin/get-gvm-releases.sh +++ b/bin/get-gvm-releases.sh @@ -4,15 +4,34 @@ echo "Checking github for the latest releases." rm build.rc # Source the api token . .token +COUNT=0 +rm versions.md +echo "# Greenbone Versions in Latest image: # +Component | Version | | Component | Version +----------|----------|-|----------|---------" > versions.md for repo in pg-gvm notus-scanner gvmd openvas openvas-smb gvm-libs openvas-scanner gsa ospd ospd-openvas ; do VERSION=$(curl -s -H "Authorization: token $Oauth" -L https://api.github.com/repos/greenbone/$repo/releases/latest | jq -r ".assets[].browser_download_url" | sed "s/^.*download\/\(v.*\)\/.*$/\1/" | head -1) echo "$repo current version is $VERSION" VAR=$( echo $repo | tr - _ ) echo "$VAR=$VERSION" >> build.rc + COUNT=$( expr $COUNT + 1 ) + LF=$( expr $COUNT % 2) + if [ $LF -eq 1 ]; then + echo -n "| $VAR | \$${repo} |" >> versions.md + else + echo " | $VAR | \$${repo} |" >> versions.md + fi done for repo in python-gvm gvm-tools; do python_gvm=$(curl -s -H "Authorization: token $Oauth" -L https://api.github.com/repos/greenbone/$repo/releases/latest | jq -r ".tarball_url" | awk -F/ '{print $NF}' ) echo "$repo current version is $python_gvm" VAR=$(echo $repo | tr - _ ) echo "$VAR=$python_gvm" >> build.rc + COUNT=$( expr $COUNT + 1 ) + LF=$( expr $COUNT % 2) + if [ $LF -eq 1 ]; then + echo -n "| $VAR | \$${repo} |" >> versions.md + else + echo " | $VAR | \$${repo} |" >> versions.md + fi done diff --git a/build.rc b/build.rc index f268774..b6b97aa 100644 --- a/build.rc +++ b/build.rc @@ -1,12 +1,12 @@ -pg_gvm=v22.4.0 +pg_gvm=v22.5.1 notus_scanner=v22.5.0 -gvmd=v22.4.2 -openvas=v22.5.0 -openvas_smb=v22.5.0 -gvm_libs=v22.5.2 -openvas_scanner=v22.5.0 -gsa=v22.4.1 +gvmd=v22.5.5 +openvas=v22.7.3 +openvas_smb=v22.5.3 +gvm_libs=v22.6.3 +openvas_scanner=v22.7.3 +gsa=v22.5.0 ospd=v21.4.4 -ospd_openvas=v22.5.0 -python_gvm=v23.4.2 +ospd_openvas=v22.5.3 +python_gvm=v23.5.1 gvm_tools=v23.4.0 diff --git a/compose/.env b/compose/.env index 91fc207..9096604 100644 --- a/compose/.env +++ b/compose/.env @@ -1 +1 @@ -TAG="22.4-beta" +TAG="22.4.20" From 5722f112b50103ba414d7fdd26d68651c16a941f Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Fri, 21 Jul 2023 18:14:13 +0200 Subject: [PATCH 7/8] fixed variable in get-gvm-releases.sh --- bin/get-gvm-releases.sh | 8 ++++---- versions.md | 14 +++++++------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/bin/get-gvm-releases.sh b/bin/get-gvm-releases.sh index 00ecd32..4e65ba6 100755 --- a/bin/get-gvm-releases.sh +++ b/bin/get-gvm-releases.sh @@ -17,9 +17,9 @@ for repo in pg-gvm notus-scanner gvmd openvas openvas-smb gvm-libs openvas-scann COUNT=$( expr $COUNT + 1 ) LF=$( expr $COUNT % 2) if [ $LF -eq 1 ]; then - echo -n "| $VAR | \$${repo} |" >> versions.md + echo -n "| $VAR | $VERSION |" >> versions.md else - echo " | $VAR | \$${repo} |" >> versions.md + echo " | $VAR | $VERSION |" >> versions.md fi done for repo in python-gvm gvm-tools greenbone-feed-sync; do @@ -30,8 +30,8 @@ for repo in python-gvm gvm-tools greenbone-feed-sync; do COUNT=$( expr $COUNT + 1 ) LF=$( expr $COUNT % 2) if [ $LF -eq 1 ]; then - echo -n "| $VAR | \$${repo} |" >> versions.md + echo -n "| $VAR | $VERSION |" >> versions.md else - echo " | $VAR | \$${repo} |" >> versions.md + echo " | $VAR | $VERSION |" >> versions.md fi done diff --git a/versions.md b/versions.md index 3b241b9..1716b70 100644 --- a/versions.md +++ b/versions.md @@ -1,10 +1,10 @@ # Greenbone Versions in Latest image: # Component | Version | | Component | Version ----------|----------|-|----------|--------- -| pg_gvm | $pg-gvm | | notus_scanner | $notus-scanner | -| gvmd | $gvmd | | openvas | $openvas | -| openvas_smb | $openvas-smb | | gvm_libs | $gvm-libs | -| openvas_scanner | $openvas-scanner | | gsa | $gsa | -| gsad | $gsad | | ospd | $ospd | -| ospd_openvas | $ospd-openvas | | python_gvm | $python-gvm | -| gvm_tools | $gvm-tools | | greenbone_feed_sync | $greenbone-feed-sync | +| pg_gvm | v22.5.1 | | notus_scanner | v22.5.0 | +| gvmd | v22.5.5 | | openvas | v22.7.3 | +| openvas_smb | v22.5.3 | | gvm_libs | v22.6.3 | +| openvas_scanner | v22.7.3 | | gsa | v22.5.0 | +| gsad | v22.5.1 | | ospd | v21.4.4 | +| ospd_openvas | v22.5.3 | | python_gvm | v22.5.3 | +| gvm_tools | v22.5.3 | | greenbone_feed_sync | v22.5.3 | From cc6ab5d899e9f684e2c6ed3e5d50d38c89a4ad1c Mon Sep 17 00:00:00 2001 From: GE Scott Knauss Date: Fri, 21 Jul 2023 18:18:11 +0200 Subject: [PATCH 8/8] python variable cleanup. --- bin/get-gvm-releases.sh | 6 +++--- versions.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/bin/get-gvm-releases.sh b/bin/get-gvm-releases.sh index 4e65ba6..9d95c10 100755 --- a/bin/get-gvm-releases.sh +++ b/bin/get-gvm-releases.sh @@ -23,10 +23,10 @@ for repo in pg-gvm notus-scanner gvmd openvas openvas-smb gvm-libs openvas-scann fi done for repo in python-gvm gvm-tools greenbone-feed-sync; do - python_gvm=$(curl -s -H "Authorization: token $Oauth" -L https://api.github.com/repos/greenbone/$repo/releases/latest | jq -r ".tarball_url" | awk -F/ '{print $NF}' ) - echo "$repo current version is $python_gvm" + VERSION=$(curl -s -H "Authorization: token $Oauth" -L https://api.github.com/repos/greenbone/$repo/releases/latest | jq -r ".tarball_url" | awk -F/ '{print $NF}' ) + echo "$repo current version is $VERSION" VAR=$(echo $repo | tr - _ ) - echo "$VAR=$python_gvm" >> build.rc + echo "$VAR=$VERSION" >> build.rc COUNT=$( expr $COUNT + 1 ) LF=$( expr $COUNT % 2) if [ $LF -eq 1 ]; then diff --git a/versions.md b/versions.md index 1716b70..8bdf151 100644 --- a/versions.md +++ b/versions.md @@ -6,5 +6,5 @@ Component | Version | | Component | Version | openvas_smb | v22.5.3 | | gvm_libs | v22.6.3 | | openvas_scanner | v22.7.3 | | gsa | v22.5.0 | | gsad | v22.5.1 | | ospd | v21.4.4 | -| ospd_openvas | v22.5.3 | | python_gvm | v22.5.3 | -| gvm_tools | v22.5.3 | | greenbone_feed_sync | v22.5.3 | +| ospd_openvas | v22.5.3 | | python_gvm | v23.5.1 | +| gvm_tools | v23.4.0 | | greenbone_feed_sync | v23.7.0 |