AutotestServerInstall

Installing an Autotest server (Ubuntu version)

Install script

We have developed a script to automate the steps described below on a Ubuntu 12.04 server. On an autotest source clone, this script is under contrib/install-autotest-server.sh:

https://github.com/autotest/autotest/blob/master/contrib/install-autotest-server.sh

If you want to download it straight from github, just

curl https://raw.github.com/autotest/autotest/master/contrib/install-autotest-server.sh > install-autotest-server.sh

Then make it executable and execute it:

chmod +x install-autotest-server.sh
./install-autotest-server.sh

Server Installation Steps

Install utility packages:

apt-get install -y unzip wget gnuplot makepasswd

Install webserver related packages (and Django):

apt-get install -y apache2-mpm-prefork libapache2-mod-wsgi python-django

Install database related packages:

apt-get install -y mysql-server python-mysqldb

Install java in order to compile the web interface, and git for cloning the autotest source code repository:

apt-get install git openjdk-7-jre-headless

Also, you'll need to install a bunch of auxiliary external packages

apt-get install python-imaging python-crypto python-paramiko python-httplib2 python-numpy python-matplotlib python-setuptools python-simplejson

You will create a special user for running autotest jobs.

sudo adduser autotest
cd /usr/local/
sudo mkdir autotest
sudo chown autotest:autotest autotest
su autotest

Make sure that the user Autotest runs as has passwordless ssh into root of any machines under test. There are many good guides here: Google

Clone the full Autotest repository:

git clone git://github.com/autotest/autotest.git /usr/local/autotest

Set up MySQL. The following commands will set up a read-only user called nobody and a user with full permissions called autotest:

mysql -u root -p
create database autotest_web;
grant all privileges on autotest_web.* TO 'autotest'@'localhost' identified by 'some_password';
grant SELECT on autotest_web.* TO 'nobody'@'%';
grant SELECT on autotest_web.* TO 'nobody'@'localhost';

If you use safesync for migrating the databases you will want to grant access to the test database.
```
GRANT ALL ON test_autotest_web.* TO 'autotest'@'localhost' identified by 'some_password';
```

If you want mysql available to hosts other than the localhost, you'll then want to comment out the bind-address = 127.0.0.1 line in the /etc/mysql/my.cnf.

In addition, you may want to increase the set-variable = max_connections to something like 6000, if you're running on a substantial server.

If you experience scalability issues, you may want to log slow queries for debugging purposes. This is done with the following lines:
log_slow_queries = /var/log/mysql/mysql-slow.log  # Log location
long_query_time = 30  # Time in seconds before we consider it slow
Note: Advanced setups may wish to use MySQL Replication

Run the build script to install necessary external packages (You will need unzip to be able to use setuptools). Currently, the packages it installs are setuptools, mysql-python, Django, numpy, matplotlib, paramiko, and GWT:
```
/usr/local/autotest/utils/build_externals.py
```
Always re-run this after a git pull if you notice it has changed, new dependencies may have been added. This is safe to rerun as many times as you want. It will only fetch and build what it doesn't already have.

NOTE: Set the HTTP_PROXY environment variable to

http://proxy:3128/ before running the above if your site requires a proxy to fetch urls.

Setup Apache config files. If the only thing you want to do with Apache is run Autotest, you can use the premade Apache conf:

   sudo rm /etc/apache2/sites-enabled/000-default
   sudo ln -s /usr/local/autotest/apache/conf/apache-conf /etc/apache2/sites-enabled/001-autotest

Also, you'll need to enable rewrite mod rules, which you can do by

 a2enmod rewrite


If you want to do other things on the Apache server as well, you'll
need to insert the following line into your Apache conf, under the
appropriate ``VirtualHost`` section:

::

    Include "/usr/local/autotest/apache/conf/all-directives"

Note: You will have to enable mod\_env on SuSE based distro's for the
all-directives to load properly when apache is started.

 Next, take a look in your ``/etc/apache2/apache2.conf`` for Debian
 based distros, ``/etc/apache2/mod_autoindex-defaults.conf`` for SuSE
 based distros or for newer distros it may also be in
 ``/etc/apache2/mods-available/autoindex.conf``. If the following
 line exists, comment it out; it's a broken "feature" of Apache that
 can't be overriden:

 ::

     IndexIgnore .??* *~ *# HEADER* RCS CVS *,t

 In the same file, either add the following line or un-comment it out
 if it already exists:

 ::

     AddHandler cgi-script .cgi

Enabled mod_headers for Apache
```
sudo a2enmod headers
```
Edit global_config.ini to match the passwords you set earlier.
Run DB migrations to set up DB schema and initial data:
```
database/migrate.py sync
```
If you have an existing database with data in it, you may want to use "safesync" instead of "sync". This will copy your data to a fresh DB and run the migration on that first, to ensure no errors occur during migration that could leave your database in an inconsistent state.
Compile the Autotest frontend web client and the new TKO client:
```
/usr/local/autotest/utils/compile_gwt_clients.py -a
```
You will need to re-compile after any changes/syncs of the frontend/client pages.

Run Django's syncdb.

/usr/local/autotest/frontend/manage.py syncdb
# you may have to run this twice, like if the debug_user does not have access to the Django Admin interface
/usr/local/autotest/frontend/manage.py syncdb

Make everything in the /usr/local/autotest directory world-readable, for Apache's sake:

chmod -R o+r /usr/local/autotest
find /usr/local/autotest/ -type d | xargs chmod o+x
chmod o+x /usr/local/autotest/tko/*.cgi

Restart apache
```
sudo apache2ctl restart
```
Check that the web frontend is working by browsing to http://localhost/afe/server/rpc_doc If you see an error or exception traceback, you need to go back and fix something.
Once that works, you should be able to access the web frontend at http://localhost/afe/.
Add client and server side tests to the frontend using utils/test_importer.py
```
/usr/local/autotest/utils/test_importer.py -A
```
To start the scheduler, you can run something like this (as user autotest):
```
/usr/local/autotest/scheduler/monitor_db.py /usr/local/autotest/results
```

To start the scheduler on reboot, you can setup init.d.

sudo cp /usr/local/autotest/utils/autotest.init /etc/init.d/
sudo update-rc.d /usr/local/autotest.init defaults

Then, you can reboot and you will autotest-scheduler-watcher and autotest-scheduler processess running.

Adding some initial frontend objects

Once you've got the frontend working, you'll want to add some objects so you can schedule some jobs.

Click the "Admin interface" link at the upper-right corner of the frontend.
Click "Labels" and then "Add label".
Type a name for the label, and check the "Platform" box, then click "Save".
Click "Home" at the upper-left corner, and click "Hosts" and then "Add host".
Type in a hostname, add your platform to the "chosen labels", and click "Save". If you want your job to succeed, you'll need to set things up so that the autotest user can ssh to the hostname you provide without needing a password.
Click "Home", and click "Tests" and then "Add test".
Fill in the following information:
- Name: sleeptest
- Test class: Kernel
- Test type: Client
- Path: client/tests/sleeptest/control
Click "Save".
Now click "Go to Frontend" at the upper-right corner.

You should now be able to create a job on the "Create Job" tab, submit it, and see it in the "Job List" and "View Job" tabs. You can also see the status of your host on the "Hosts" tab.

Additional Tips

Some really old distributions have similarly ancient SSH implementations. If so, it can take hours for a SSH connection to timeout if a machine dies. You can modify your /etc/sysctl.conf to contain the line net.ipv4.tcp_keepalive_time = 60 to reduce this time to around 10 minutes. This is typically unnecessary, though.
You'll probably want to back up your results filesystem and mysql db on a regular basis. A cron job is a simple way to accomplish this.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly