Integration with Services

Checklist

In order for a service to be lcp-compose compatible, the following requirements must be satisfied.

Bootstrapping

A service should recognize that it is running in a bootstrappable environment and proceed to initialize it's dependencies prior to starting the application. The bootstrap actions could include creating databases, database users, elasticsearch indices, rabbitmq vhosts, etc. These actions must be idempotent (i.e, if run more than once, should not destroy/modify existing data). If a service depends on a resource owned by another service, it must assume that the resource already exists.

Configuration Templates

The service must store a version of it's configuration in a template(jinja2) format in the repo and ensure that it is included in any docker containers built from the project. lcp-compose will extract these template files at lcp-compose up-time and render them to be consumed by the service.

The template files could be either an ENV file (lcp-compose.env.j2) that gets injected to the container OR a configuration directory that gets mounted to the container at /config. The nature of configuration injection is determined by the lcp-compose configuration for each service. See here for more information on configuring lcp-compose for templated service configurations.

Refactor Integration Tests

Tests should simply issue lcp-compose commands to spin up the LCP stack. Use manage/unmanage to configure the services that should be not run, or built from Dockerfiles. Remove all hardcoded hostnames, URLs, etc from the tests and use a templated configuration file to source them.

Containerize the integration test runner by using the containerized build runner.

Gotchas

Integration Example

This is an overview of our process in migrating the Orders service to be lcp-compose compatible. Orders service currently uses both old style and new style configuration. lcp-compose is compatible with both.

Bootstrapping Databases

Update the orders_service's docker entry-point to provision the database only when an env var is supplied.

# ... 
if [ "${TEST_ORDERS_DB}" = "TRUE" ]; then
    echo "Bootstrapping databases"
    fab init_db
    fab create_bi_user
fi

# ... rest of container start up -- starting apache/gunicorn

Configuration Templates

# Restart gunicorn workers when code changes
GUNICORN_RELOAD=
# Port on which to run the orders service on.
LOAD_BALANCER_BACKEND_PORT={{ services.orders.url | port }}
SERVER_WORKERS=4

# Enable bootstrapping of databases
TEST_ORDERS_DB=TRUE

ACTIVE_SERVICES = [
  'orders.orders_rest_api',
]

# Security configuration
PERMISSION_SET_RESOURCE_URL = "/security/permission-sets/"
PERMISSION_SET_ALLOWED_RESOURCES_FOR_PRINCIPAL_URL = "/security/permission-sets/allowed-resources-for-principal/"
PERMISSION_CHECK_URL = "/security/permission-sets/resource-authorization"
KEYPAIR_RESOURCE_URL = "/security/key-pairs/"
PRINCIPALS_RESOURCE_URL = "/security/principals/"
GROUPS_RESOURCE_URL = "/security/groups/"
SEARCH_URL = "/search/"

INTERNAL_BASE_URLS = {
    'security': '{{ services.security.url | unslashed }}',
    'search': '{{ services.platform_core.url | unslashed }}',
    'events': '{{ services.platform_core.url | unslashed }}',
}

COUCHDB_ADMIN_USER = '{{ services.couchdb.user }}'
COUCHDB_ADMIN_PASSWORD = '{{ services.couchdb.password }}'

DATABASE_APPLICATION_USER = 'orders_user'
DATABASE_APPLICATION_PASSWORD = 'orders_user'

DATABASE_ADMIN_USER = 'orders_admin'
DATABASE_ADMIN_PASSWORD = 'orders_admin'

DATABASE_BI_USER = 'orders_bi_user'
DATABASE_BI_PASSWORD = 'orders_bi_user'

DATABASE_URL_BASE = '{{ services.couchdb.url }}'
DATABASE_NAME = 'orders'

DATABASE_MIGRATION_URL_BASE = '{{ services.couchdb.url }}'

Refactoring Tests

Update the fab task for system integration tests to use lcp-compose commands instead of lcpenv, while maintaining command compatibility with Orders' existing TC project.

# file: fabfile/tasks.py
import json
import os

from fabric.api import task, local, shell_env

TEAMCITY_VERSION = os.environ.get('TEAMCITY_VERSION')
ORDERS_ENDPOINT_PERMISSIONS = {'/orders/': ['POST'], '/search/orders/': ['GET']}


def _write_file(content, path):
    with open(path, 'w+') as fh:
        fh.write(content)


@task()
def test_system_integration(keeplcp=False):
    local('lcp-compose clean --config')
    local('lcp-compose init')
    local('lcp-compose manage orders --build-path=.')

    test_config_template_path = 'tests/integration/configuration/lcp-compose.ini.j2'
    test_config_path = 'tests/integration/configuration/lcp-compose.ini'
    local('lcp-compose assets.render_template --template={template} --destination={destination}'.format(
        template=test_config_template_path,
        destination=test_config_path
    ))

    try:
        local('lcp-compose up')

        # .. create accounts, permissions, etc.

        servicecontainer_path = '.lcp-compose/configurations/orders/servicecontainer.cfg'
        accounts_json = '.lcp-compose/accounts.json'

        environment = {
            'TEAMCITY_VERSION': TEAMCITY_VERSION if TEAMCITY_VERSION else '',
            'TEST_CONFIG': os.path.basename(test_config_path),
            'ACCOUNT_CLIENTS_JSON': accounts_json,
            'SERVICE_CONFIGURATION_PATH': servicecontainer_path
        }

        with shell_env(**environment):
            local('lcp-compose exec orders -- fab reset_db')
            local('nosetests -v --attr system_integration --nocapture --tests tests/integration', capture=False)
    finally:
        if not keeplcp:
            local('lcp-compose down -v')

Then we update the Makefile to run the tests in a containerized environment.

test_system_integration:
    docker pull dev-docker.points.com/lcpcompose-containerized:latest && \
    docker run --rm --network=host \
        --env UID=$$(id -u) \
        --env GID=$$(id -g) \
        --env PROJECT_NAME=orders \
        --env PROJECT_DIR=$(shell pwd) \
        --volume "/var/run/docker.sock:/var/run/docker.sock" \
        --volume "$(shell pwd):$(shell pwd)" \
        dev-docker.points.com/lcpcompose-containerized:latest '\
            virtualenv --no-site-packages /tmp/venv && . /tmp/venv/bin/activate && \
            easy_install --upgrade setuptools==26.0.0 && \
            pip install --upgrade pip==9.0.1 && \
            pip install -r requirements/development.txt && \
            $(TEAMCITY_REPORTING_SYS_INT) fab test_system_integration \
        '

Note: There were many other changes made to the testing code to remove hardcoded hostnames and other variables. These changes are left out from this guide for brevity.