Resolving Oracle Cloud "Out of Capacity" issue and getting free VPS with 4 ARM cores / 24GB of memory
Very neat and useful configuration was recently announced at Oracle Cloud Infrastructure (OCI) blog as a part of Always Free tier. Unfortunately, as of July 2021, it's very complicated to launch an instance due to the "Out of Capacity" error. Here we're solving that issue as Oracle constantly adds capacity from time to time.
Each tenancy gets the first 3,000 OCPU hours and 18,000 GB hours per month for free to create Ampere A1 Compute instances using the VM.Standard.A1.Flex shape (equivalent to 4 OCPUs and 24 GB of memory).
This approach requires PHP ^7.4...^8.3 and composer installed and will call "LaunchInstance" OCI API endpoint. We'll utilise the package which I've written (and published) some time ago, here's the article.
If you prefer article style, here's a link to Medium
YouTube video instruction https://youtu.be/uzAqgjElc64 is a bit outdated regarding Configuration but still can be useful for the rest.
If you appreciate what I did please consider
- Generating API key
- Installation
- Configuration
- Running the script
- Periodic job setup (cron)
- How it works
- Assigning public IP address
- Troubleshooting
- Multiple configuration support
- Conclusion
After logging in to OCI Console, click profile icon and then "User Settings"
Go to Resources -> API keys, click "Add API Key" button
Make sure "Generate API Key Pair" radio button is selected, click "Download Private Key" and then "Add".
Copy the contents from textarea and save it to file with a name "config". I put it together with *.pem file in newly created directory /home/ubuntu/.oci
Clone this repository
git clone https://github.com/hitrov/oci-arm-host-capacity.git
run
cd oci-arm-host-capacity/
composer install
Copy .env.example
as .env
cp .env.example .env
You must modify .env
file below. Don't push/share it as it possibly contains sensitive information.
All parameters except OCI_AVAILABILITY_DOMAIN
are mandatory to be set. Please read the comments in .env
file as well.
Region, user, tenancy, fingerprint should be taken from textarea during API key generation step.
Adjust these values in .env
file accordingly:
OCI_REGION
OCI_USER_ID
OCI_TENANCY_ID
OCI_KEY_FINGERPRINT
OCI_PRIVATE_KEY_FILENAME
is an absolute path (including directories) or direct public accessible URL to your *.pem private key file.
You must start instance creation process from the OCI Console in the browser (Menu -> Compute -> Instances -> Create Instance)
Change image and shape. For Always free AMD x64 - make sure that "Always Free Eligible" availabilityDomain label is there:
ARMs can be created anywhere within your home region.
Adjust Networking section, set "Do not assign a public IPv4 address" checkbox. If you don't have existing VNIC/subnet, please create VM.Standard.E2.1.Micro instance before doing everything.
"Add SSH keys" section does not matter for us right now. Before clicking "Create"…
…open browser's dev tools -> network tab. Click "Create" and wait a bit most probably you'll get "Out of capacity" error. Now find /instances API call (red one)…
…and right click on it -> copy as curl. Paste the clipboard contents in any text editor and review the data-binary parameter.
Find subnetId
, imageId
and set OCI_SUBNET_ID
, OCI_IMAGE_ID
, respectively.
Note availabilityDomain
for yourself, then read the corresponding comment in .env
file regarding OCI_AVAILABILITY_DOMAIN
.
In order to have secure shell (SSH) access to the instance you need to have a keypair, besically 2 files:
- ~/.ssh/id_rsa
- ~/.ssh/id_rsa.pub
Second one (public key) contents (string) should be provided to a command below. The are plenty of tutorials on how to generate them (if you don't have them yet), we won't cover this part here.
cat ~/.ssh/id_rsa.pub
Output should be similar to
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFwZVQa+F41Jrb4X+p9gFMrrcAqh9ks8ATrcGRitK+R/ [email protected]
Change OCI_SSH_PUBLIC_KEY
inside double quotes - paste the contents above (or you won't be able to login into the newly created instance).
NB! No new lines allowed!
OCI_OCPUS
and OCI_MEMORY_IN_GBS
are set 4
and 24
by default. Of course, you can safely adjust them.
Possible values are 1/6, 2/12, 3/18 and 4/24, respectively.
Please notice that "Oracle Linux Cloud Developer" image can be created with at least 8GB of RAM (OCI_MEMORY_IN_GBS
).
If for some reason your home region is running out of Always free AMD x64 (1/8 OPCU + 1GB RAM), replace values below.
NB! Setting the OCI_AVAILABILITY_DOMAIN
to Always Free Eligible
is mandatory for non-ARM architecture!
OCI_SHAPE=VM.Standard.E2.1.Micro
OCI_OCPUS=1
OCI_MEMORY_IN_GBS=1
OCI_AVAILABILITY_DOMAIN=FeVO:EU-FRANKFURT-1-AD-2
If you don't have instances of selected shape at all, and need only one, leave the value of OCI_MAX_INSTANCES=1
.
When you managed to launch one and need more (or 2 from scratch), set to OCI_MAX_INSTANCES=2
.
php ./index.php
I bet that the output (error) will be similar to the one in a browser a few minutes ago
{
"code": "InternalError",
"message": "Out of host capacity."
}
or if you already have instances:
{
"code": "LimitExceeded",
"message": "The following service limits were exceeded: standard-a1-memory-count, standard-a1-core-count. Request a service limit increase from the service limits page in the console. "
}
You can now setup periodic job to run the command
Create log file:
touch /path/to/oci-arm-host-capacity/oci.log
Set permissions for PHP script to modify it:
chmod 777 /path/to/oci-arm-host-capacity/oci.log
Get full path to PHP binary
which php
Usually that's /usr/bin/php
Setup itself:
EDITOR=nano crontab -e
Add new line to execute the script every minute and append log the output:
* * * * * /usr/bin/php /path/to/oci-arm-host-capacity/index.php >> /path/to/oci-arm-host-capacity/oci.log
NB! Use absolute paths wherever possible
...and save the file (F2, press Y to confirm overwrite, then Enter).
There could be cases when cron user won't have some permissions, there're ways to solve it:
- Setup job for root user by executing
EDITOR=nano sudo crontab -e
- Move this directory (
oci-arm-host-capacity
) into web server's one e.g. /usr/share/nginx/html and setup cron this way:
* * * * * curl http://server.add.re.ss/oci-arm-host-capacity/index.php >> /path/to/oci-arm-host-capacity/oci.log
You can also visit the URL above and see the same command output as by running from the shell.
In order to test the script using GitHub runners (their virtual machines) please complete Setup. NB! To avoid the ban of your Github account Read This Carefully !!!
- Fork this repository
- Never push
.env
file, it's in.gitignore
for a reason - Instead of copying/modifying
.env
file, useSecrets
in your own repositorySettings
:
https://github.com/{your-username}/oci-arm-host-capacity/settings/secrets/actions
- Click
New repository secret
and set all the values (one by one) that you'd set in.env
file e.g.
NB! No need to double quote any value here!
- As for the private key, you have 2 options. Either:
- upload to any web server accessible from the Internet by using just URL or...
- upload in the bucket and
Create Pre-Authenticated Request
.
- Copy and save the URL from (5) as
OCI_PRIVATE_KEY_FILENAME
GitHub secret. - Go to any other directory e.g.
cd /Users/hitrov
git clone https://github.com/{your-username}/oci-arm-host-capacity
- Adjust the file
.github/workflows/tests.yml
according to this commit, just execute:
git checkout 67fe41ebfb9f385ae1614c97b74195ea318c8db7 -- .github/workflows/tests.yml
- Commit and push this file
git commit -m "Modify workflow to test out periodic job" .github/workflows/tests.yml
git push origin main
- Go to
https://github.com/{your-username}/oci-arm-host-capacity/actions
and check howRun script
job.
Here's the example https://github.com/hitrov/oci-arm-host-capacity/runs/4727904401?check_suite_focus=true
Specific GitHub Workflows commit
used in the Setup take an advantage of Scheduled events
and will endlessly run the script every 5-20 minutes (how exactly often - depends on runners' availability).
NB! After you're done with testing, immediately delete .github/workflows/tests.yml (because you don't need integration tests - they're written taking into account instances that I have) and push to the main
branch
because infinite run actually violates the Terms of Use:
Actions should not be used for:
...
- if using GitHub-hosted runners, any other activity unrelated to the production, testing, deployment, or publication
of the software project associated with the repository where GitHub Actions are used.
...
GitHub may monitor your use...
Misuse of GitHub Actions may result in termination of jobs, restrictions in your ability to use GitHub Actions,
or the disabling of repositories created to run Actions in a way that violates these Terms.
This is how you do:
git rm .github/workflows/tests.yml
git commit -m "Delete workflow file" .github/workflows/tests.yml
git push origin main
Before the instance creation, script will:
- Call ListAvailabilityDomains OCI API method
- Call ListInstances OCI API method
and check whether there're already existing instances with the same
OCI_SHAPE
, as well as number of themOCI_MAX_INSTANCES
(you can safely adjust the last one if you wanna e.g. twoVM.Standard.A1.Flex
with 2/12 - 2 OCPUs and 12GB RAM - each).
Script won't create new instance if current (actual) number return from the API exceeds the one from OCI_MAX_INSTANCES
variable.
In case of success the JSON output will be similar to
We are not doing this during the command run due to the default limitation (2 ephemeral addresses per compartment). That's how you can achieve this. When you'll succeed with creating an instance, open OCI Console, go to Instance Details -> Resources -> Attached VNICs by selecting it's name
Then Resources -> IPv4 Addresses -> Edit
Choose ephemeral and click "Update"
OCI_PRIVATE_KEY_FILENAME
doesn't exist.
PHP Fatal error: Uncaught Hitrov\OCI\Exception\PrivateKeyFileNotFoundException: Private key file does not exist: /path/to/oracleidentitycloudservice_***-07-14-10-35.pem in /Users/hitrov/Sites/oci-arm-host-capacity/vendor/hitrov/oci-api-php-request-sign/src/Hitrov/OCI/Signer.php:346
Make sure path is absolute (full including directories), you should see it's content by executing:
cat /path/to/oracleidentitycloudservice_***-07-14-10-35.pem
If that's URL make sure it's inside double quotes, and opens without redirections or additional actions:
curl "https://url.to/oracleidentitycloudservice_***-07-14-10-35.pem"
- Permission denied - private key file is inaccessible for this PHP script:
PHP Warning: file_get_contents(/path/to/oracleidentitycloudservice_***-07-14-10-35.pem): failed to open stream: Permission denied in /Users/hitrov/Sites/oci-arm-host-capacity/vendor/hitrov/oci-api-php-request-sign/src/Hitrov/OCI/Signer.php on line 225
PHP Fatal error: Uncaught TypeError: Return value of Hitrov\OCI\Signer::getPrivateKey() must be of the type string or null, bool returned in /Users/hitrov/Sites/oci-arm-host-capacity/vendor/hitrov/oci-api-php-request-sign/src/Hitrov/OCI/Signer.php:225
Fastest way to resolve:
chmod 777 /path/to/oracleidentitycloudservice_***-07-14-10-35.pem
- If you have new line(s) / line ending(s) in
OCI_SSH_PUBLIC_KEY
you will encounter:
{
"code": "InvalidParameter",
"message": "Unable to parse message body"
}
- If public key is incorrect:
{
"code": "InvalidParameter",
"message": "Invalid ssh public key; must be in base64 format"
}
Copy the proper contents of ~/.ssh/id_rsa.pub
again and make sure it's inside double quotes.
Or re-generate pair of keys. Make sure you won't unintentionally overwrite your existing ones.
If you need 2+ .env
files (or you have multiple Oracle Cloud Infrastructure accounts),
run the script in a way when you pass argument with environment filename e.g.
php index.php .env.my_acc1
Custom env filenames are supported only for CLI (command line interface).
If you call this script with browser/curl using web sapi (Apache, nginx),
find the best way to pass the 2nd argument here (e.g. $_GET
parameter):
$dotenv = Dotenv::createUnsafeImmutable(__DIR__, $envFilename);
as I don't want to overcomplicate this script for rare use cases.
That's how you will login when instance will be created (notice opc default username)
ssh -i ~/.ssh/id_rsa [email protected]
If you didn't assign public IP, you can still copy internal FQDN or private IP (10.x.x.x) from the instance details page and connect from your other instance in the same VNIC. e.g.
ssh -i ~/.ssh/id_rsa [email protected]