alonsodomin / quckoo   0.1.0

Website GitHub

Fault tolerant distributed task scheduler.

Scala versions: 2.11
Scala.js versions: 0.6

Quckoo

Build Status Codacy Badge codecov License Maven Central

Quckoo is a fault-tolerant distributed task scheduler platform that runs on the JVM. The aim of the project is the implementation of a reliable system able to run large amount of scheduled tasks without single points of failure.

To achieve that, Quckoo is composed of a cluster of scheduler nodes and ad hoc worker nodes that connect to this cluster and request for work to be sent to them. It's basically a generalization of the distributed worker pattern (and in fact the implementation owes a lot to the previous post).

Documentation

Building

First of all, ensure you have the following tools in your local machine:

  • Java JDK 1.8+ for running and compiling the source code
  • SBT 0.13.5+ for building the whole project
  • NodeJS 4.5.x+ for running and optimizing the ScalaJS code
  • PhantomJS 2.x+ for running the ScalaJS test suites
  • SASS 3.4.0+ for compiling UI's stylesheets.

Optionally, if you will want to be able to run Quckoo in your local machine using the Docker containers you will need one of the following (or both):

  • Docker 1.10+ for running the docker containers "natively" in your local machine
  • Vagrant 1.8.x+ plus a Virtual Machine provider (i.e.: VirtualBox 5.x+) for bootstrapping the sandbox.

To build the project input following commands in a terminal:

git clone https://github.com/alonsodomin/quckoo
cd quckoo
sbt package

Launching the cluster

The easiest way to launch the cluster right now is by building the Docker images locally and then starting the default docker compose cluster setup. This operations can easily be done from within SBT:

sbt launchLocal

After the previous commands, using docker ps you should be able to see 2 master nodes, one worker node and one Cassandra node running as docker containers in your local machine. The cluster will need a few seconds to fully stabilize and then it will be ready to serve requests. You should be able to access the Quckoo UI in following URL:

http://localhost:8095

Use admin and password as credentials when prompted.

Fully isolated environment

The repository ships with a Vagrantfile in its root folder for ease the setup of a fully isolated environment. This Vagrant configuration will create a virtual machine in the host computer and deploy into it the required components of the architecture as Docker containers. To start this environment issue following command in a terminal window from inside your working copy:

vagrant up

To access the UI in this environment, use this URL: http://192.168.50.25:8095

Build error when building the vagrant box

When trying to bootstrap the self-contained VM with Vagrant, SBT will be invoked from the inside the VM to publish the Docker images in the host's local registry. When this is done the first time SBT may fail to communicate with the local Docker daemon and you will see a build error. If this happens just re-provision the VM and it should work.

vagrant provision

Contributing

Quckoo is still right now in experimental phase, current codebase will be evolving until it reaches the level of robustness necessary to be able to trust on the system to be able to handle the big load we are aiming for. Feel free to fork this repository and contribute with your knowledge to make this project a reliable and really fault-tolerant task scheduling platform.

License

Copyright 2015 Antonio Alonso Dominguez

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.