-
Notifications
You must be signed in to change notification settings - Fork 168
Orchestrator for developers
NOTE: orchestrator development is now active on https://github.com/github/orchestrator, where Issues and Pull Requests are accepted.
This repository is no longer the upstream and latest version of orchestrator.
The documentation in this repository is not up-to-date.
Orchestrator is open source and accepts pull requests.
If you would like to build orchestrator on your own machine, or eventually submit PRs, follow this guide.
Orchestrator is built on Linux. OS/X should generally follow same guidelines. I have no hint about MS Windows, and the build is incompatible with Windows.
You will need to have a Go environment. At this time (2016-03-24) orchestrator is built with Go 1.5 and with experimental vendor. You will need to
set GO15VENDOREXPERIMENT=1
This guide assumes you have set your Go environment, along with GOPATH.
To get started, issue
go get github.com/outbrain/orchestrator
Change directory into $GOPATH:/src/github.com/outbrain/orchestrator
Issue the following to resolve all dependencies:
go get ./...
Test that your code builds via
go run go/cmd/orchestrator/main.go
Orchestrator requires a MySQL backend to run. This could be installed anywhere. I usually use mysqlsandbox for local installations. You may choose to just install mysql-server on your dev machine.
Once your backend MySQL setup is complete, issue:
CREATE DATABASE IF NOT EXISTS orchestrator;
GRANT ALL PRIVILEGES ON `orchestrator`.* TO 'orc_server_user'@'127.0.0.1'
IDENTIFIED BY 'orc_server_password';
Replace 127.0.0.1 with hostname or orchestrator machine (or do your wildcards thing). Choose your password wisely. Edit orchestrator.conf.json to match:
Orchestrator uses a configuration file whose search path is either /etc/orchestrator.conf.json, conf/orchestrator.conf.json or orchestrator.conf.json.
The repository includes a file called conf/orchestrator-sample.conf.json with some basic settings. Issue:
cp conf/orchestrator-sample.conf.json conf/orchestrator.conf.json
The conf/orchestrator.conf.json file is not part of the repository and there is in fact a .gitignore entry for this file.
Edit orchestrator.conf.json to match the above as follows:
...
"MySQLOrchestratorHost": "127.0.0.1",
"MySQLOrchestratorPort": 3306,
"MySQLOrchestratorDatabase": "orchestrator",
"MySQLOrchestratorUser": "orc_server_user",
"MySQLOrchestratorPassword": "orc_server_password",
...
Edit the above as as fit for your MySQL backend install.
You should now be able to
go run go/cmd/orchestrator/main.go http
This will also invoke initial setup of your database environment (creating necessary tables in the orchestrator schema).
Browse into http://localhost:3000 or replace localhost with your dev hostname. You should see the orchestrator (empty) clusters dashboard.
Now to make stuff interesting.
For orchestrator to detect your replication topologies, it must also have an account on each and every topology. At this stage this has to be the same account (same user, same password) for all topologies. On each of your masters, issue the following:
GRANT SUPER, PROCESS, REPLICATION SLAVE, RELOAD ON *.*
TO 'orc_client_user'@'127.0.0.1' IDENTIFIED BY 'orc_client_password';
REPLICATION SLAVE is required if you intend to use Pseudo GTID
Replace 127.0.0.1 with hostname or orchestrator machine (or do your wildcards thing). Choose your password wisely. Edit orchestrator.conf.json to match:
"MySQLTopologyUser": "orc_client_user",
"MySQLTopologyPassword": "orc_client_password",
Go to the Discovery page at http://localhost:3000/web/discover. Type in a hostname & port for a known MySQL instance, preferably one that is part of a larger topology (again I like using MySQLSandbox for such test environments). Submit it.
Depending on your configuration (DiscoveryPollSeconds, InstancePollSeconds) this may take a few seconds to a minute for
orchestrator to fully scan the replication topology this instance belongs to, and present it under the clusters dashboard.
If you've made it this far, you've done 90% of the work. You may consider configuring Pseudo GTID queries, DC awareness etc. See "want to have" sub-sections under configuration.
To build an Orchestrator package, use the build.sh script:
bash build.sh
You will need:
-
fpm, which assumes you have
rubyandruby-gems rpmbuild-
go,gofmtin path tar
Current build.sh usage is:
usage() {
echo
echo "Usage: $0 [-t target ] [-a arch ] [ -p prefix ] [-h] [-d]"
echo "Options:"
echo "-h Show this screen"
echo "-t (linux|darwin) Target OS Default:(linux)"
echo "-a (amd64|386) Arch Default:(amd64)"
echo "-d debug output"
echo "-p build prefix Default:(/usr/local)"
echo
}
If you want to submit pull-requests you should first fork http://github.com/outbrain/orchestrator.
Setting up the environment is basically the same, except you don't want to
go get github.com/outbrain/orchestrator
But instead clone your own repository.
Assume you fork onto github.com/you-are-awesome/orchestrator. Golang has tight coupling between source code import paths and actual URIs. This leads to much confusion. Please consult Forking Golang repositories on GitHub and managing the import path as for ways to solve
that coupling.
Very briefly, you will either want to:
go get github.com/outbrain/orchestrator
git remote add awesome-fork https://github.com/you-are-awesome/orchestrator.git
Or you will workaround as follows:
cd $GOPATH
mkdir -p {src,bin,pkg}
mkdir -p src/github.com/outbrain/
cd src/github.com/outbrain/
git clone [email protected]:you-are-awesome/orchestrator.git # OR: git clone https://github.com/you-are-awesome/orchestrator.git
cd orchestrator/
go get ./...
You will have a fork of orchestrator to which you can push your changes and from which you can send pull requests. It is best that you first consult (use the project issues) whether some kind of development would indeed be merged.
You will need to license your code in Apache 2.0 license or compatible.
Thank you for considering contributions to orchestrator!