Ensemble CI is designed to help businesses improve the quality, reliability and timeliness of their data pipelines by implementing an automated but controlled development lifecycle around their data transformation code.
For more details, please visit our introductory blog post.
In this section we will walk through the process of configuring your Ensemble CI deployment.
Ensemble CI is packaged and deployed using Docker and Docker Compose. You should therefore install a recent version on your deployment server.
Ensemble CI integrates with Github in order to build your repos and branches.
There is a small amount of manual configuration in Github to enable this interaction to take place with your own account.
- Visit the OAuth Apps area of Developer Settings - https://github.com/settings/developers
- Create a new app with the settings below, substituting in the publically accessible IP which you will use to host Ensemble CI
- Register your app
- Note down the Client ID and Client ID Secret for use in a subsequent step
Ensemble CI will interact with this application, meaning that you have no dependence on any centralised service hosted by us.
The next step is to clone the main Ensemble CI repository. This contains the docker-compose.yml file which we will use to start the system.
git clone https://github.com/ensembleanalytics/ensembleci.git cd ensembleci
Copy the example .env file:
cp .env.example .env
Edit the .env file and set the values correctly using the keys saved earlier, replacing the values below. You'll also need to set the publically accessible IP address of the server you are running on.
SERVER_IP_ADDRESS=184.108.40.206 GITHUB_CLIENT_ID=cc7666ffbafb59c4XXXX GITHUB_CLIENT_SECRET=377c0ae980a454082c7c4992968f56e881c9XXXX
Start the application using Docker compose.
If everything has gone well, the three services, frontend, backend, and CI server should start with healthy logs.
The frontend applications should be available at http://localhost:3000.
Github will call the CI server with webhooks whenever new code is checked into your monitored repos.
Depending on your networking setup, you may need to setup port forwarding to connect from your public facing router to the server where Ensemble CI is running.
If you would like to deploy from source instead of using Docker, please visit our contributing guide for details of how to clone and run the Ensemble CI repos.