KeiruaProd

I help my clients acquire new users and make more money with their web businesses. I have ten years of experience with the technical aspects of growing SaaS projects using data science for decision-making. If that’s something you need help with, we should get in touch!

Dynamic temporary demo environment with clever cloud

For my current engagement, we have review apps (“recette jetable”): throwaway containers generated dynamically with a singe click out of a specific git branch on our PaaS, which comes with their own sample database and subdomain.

This code explains this feature in detail, with all the code we use. It’s open-source anyway, under license GNU Affero.

I think is a very interesting but underused feature for projects. For us, it’s a very convenient, low-cost way to show a new feature to other devs, designers or stakeholders before merging it. We use this a lot for demos and QA, in order for quick adjustements before the launch of complicated features.

We use Github Actions for automation, with Clever-Cloud as a PaaS for hosting −it’s great!−.

A key element is Clever-Cloud’s utility clever tools. It allows to create/update/delete a machine on the fly through the command line. With some modifications to the code in this article, you can adapt it to other PaaSes that provide similar features.

Credits goes to my colleague Celine who did all this, I’m simply sharing this idea.

Not a tunnel to localhost

The feature described here is not similar to ngrok or localtunnel. They expose your localhost on the internet, which is useful too (say, for remote pair programming). However, with those solutions, if your code change locally (because you work on something else) the local tunnel will reflect those changes.

Deploying an branch on an external container allows you to do something else with your local machine. Some of our review apps live for multiple days, so work is not interrupted.

Behavior

The way it works is as follow. There are 3 parts that we’ll discuss in this article:

The code for this (as part of itou) has a lot of boilerplate (and may appear complex because yaml is very verbose), but it is surprisingly concise given how many things happen.

Creation

A dev opens a PR for his branch (my-awesome-feature), and attachs a label recette-jetable. A github action catchs the event “label recette-jetable has been added”. Inside the github action, it then:

# See https://developer.github.com/v3/ and https://help.github.com/en/actions
name: 🕵 Review app creation

on:
  pull_request:
    types: [ labeled ] # run this pipeline when a label is added to a PR

env:
  CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
  CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
  CLEVER_CLI: clever-tools-latest_linux/clever
  CLEVER_TOKEN: $
  CLEVER_SECRET: $
  REVIEW_APPS_ORGANIZATION_NAME: 
  BRANCH: $
  PYTHON_VERSION: 3.9

jobs:
  create:
    runs-on: ubuntu-latest
    if: github.event.action == 'labeled' && github.event.label.name == 'recette-jetable'

    steps:
    - name: 📥 Checkout to the PR branch
      uses: actions/checkout@v2
      with:
        ref: $

    - name: 📥 Fetch git branches
      run: git fetch --prune --unshallow

    - name: 🏷 Set some necessary environnment variables
      run:
        echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV
        echo "REVIEW_APP_DB_NAME=`echo $REVIEW_APP_NAME | sed -r 's/-/_/g'`" >> $GITHUB_ENV
        echo "DEPLOY_URL=`echo \"$REVIEW_APP_NAME.cleverapps.io\"`" >> $GITHUB_ENV

    - name: 🧫 Create a review app on Clever Cloud
      run: |
        curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
        tar -xvf $CLEVER_TAR_FILE
        $CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
        # Create a new application on Clever Cloud.
        # -t: application type (Python).
        # --org: organization name.
        # --region: server location ("par" means Paris).
        # --alias: custom application name, used to find it with the CLI.
        $CLEVER_CLI create $REVIEW_APP_NAME -t python --org $REVIEW_APPS_ORGANIZATION_NAME --region par --alias $REVIEW_APP_NAME
        $CLEVER_CLI env set CC_PYTHON_VERSION $ --alias $REVIEW_APP_NAME
        $CLEVER_CLI domain add $DEPLOY_URL --alias $REVIEW_APP_NAME
        $CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
    - name: 🗃 Create database addon
      run: |
        $CLEVER_CLI addon create postgresql-addon $REVIEW_APP_DB_NAME --org $REVIEW_APPS_ORGANIZATION_NAME --plan xxs_sml --yes
        $CLEVER_CLI service link-addon $REVIEW_APP_DB_NAME
    - name: 🗺 Add environment variables to the review app
      run: |
        $CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME
        $CLEVER_CLI service link-addon $CONFIGURATION_ADDON
        $CLEVER_CLI env import-vars REVIEW_APP_DB_NAME
        $CLEVER_CLI env import-vars DEPLOY_URL
    - name: 🚀 Deploy to Clever Cloud
      run: $CLEVER_CLI deploy --branch $BRANCH --force

    - name: 🍻 Add link to pull request
      uses: thollander/actions-comment-pull-request@main
      with:
        message: "🥁 La recette jetable est prête ! [👉 Je veux tester cette PR !](https://$)"
        GITHUB_TOKEN: $

The code is running through a github action, and asks Clever Cloud to setup a machine for us through clever tools. The twist is on what triggers this action, then it’s mostly configuration and running commands. Some things to note:

Pricing

Those containers are not free, but since there is very little trafic we create the smallest possible instances. This feature costs us in total less than 100€/month for hosting, which is quite a bargain given how useful it is.

Post-deploy hook

We need to populate the database with initial data, and with some sample user accounts. Fixtures are stored with the project. A small piece of code triggers their import. It is referenced in the shared review app configuration (on clever-cloud’s end) through a post-deploy hook:

CC_RUN_SUCCEEDED_HOOK="$APP_HOME/clevercloud/review-app-after-success.sh" 

Initial database data can be either a SQL dump or django fixtures.

#!/bin/sh

###################################################################
###################### Review apps entrypoint #####################
###################################################################

# Skip this step when redeploying a review app.
if [ "$SKIP_FIXTURES" = true ] ; then
    echo "Skipping fixtures."
    exit
fi

echo "Loading cities"
PGPASSWORD=$POSTGRESQL_ADDON_PASSWORD pg_restore -d $POSTGRESQL_ADDON_DB -h $POSTGRESQL_ADDON_HOST -p $POSTGRESQL_ADDON_PORT -U $POSTGRESQL_ADDON_USER --if-exists --clean --no-owner --no-privileges $APP_HOME/itou/fixtures/postgres/cities.sql

# `ls $APP_HOME` does not work as the current user does not have execution rights on the $APP_HOME directory.
echo "Loading fixtures"
ls -d $APP_HOME/itou/fixtures/django/* | xargs django-admin loaddata

Update

When a developper pushes a fix to such a branch already attached to a review app:

# See https://developer.github.com/v3/
# and https://help.github.com/en/actions
name: 🕵 Review app update

# Run this pipeline when a label is present and when a push is made on this PR.
# `types: [ synchronize ]` targets a push event made on a PR.
on:
  pull_request:
    types: [ synchronize ]

env:
  CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
  CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
  CLEVER_CLI: clever-tools-latest_linux/clever
  CLEVER_TOKEN: $
  CLEVER_SECRET: $
  REVIEW_APPS_ORGANIZATION_NAME: 
  BRANCH: $
  PYTHON_VERSION: 3.9

jobs:
  redeploy:
    runs-on: ubuntu-latest
    # A push event when the 'recette-jetable' label is present triggers a new deployment.
    if: github.event.action == 'synchronize' && contains( github.event.pull_request.labels.*.name, 'recette-jetable')

    steps:
    - name: 📥 Checkout to the PR branch
      uses: actions/checkout@v2
      with:
        ref: $

    - name: 📥 Fetch git branches
      run: git fetch --prune --unshallow

    - name: 🏷 Set review app name
      run: echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV

    - name: 🤝 Find the application on Clever Cloud
      run: |
        curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
        tar -xvf $CLEVER_TAR_FILE
        $CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
        $CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME

    - name: ⏭ Skip fixtures
      run: $CLEVER_CLI env set SKIP_FIXTURES true

    - name: 🚀 Deploy to Clever Cloud
      run: $CLEVER_CLI deploy --branch $BRANCH --force

Deletion

When a user closes a PR or removes the “recette-jetable” label, a github action catches the event and destroy all the containers used here.

name: 🔪 Review app removal

# We’ll run this pipeline:
#  - when a pull request having the label "review-app" is closed
#  - when the "recette-jetable" label is removed
on:
  pull_request:
    types: [ unlabeled, closed ]

env:
  CLEVER_TOOLS_DOWNLOAD_URL: https://clever-tools.clever-cloud.com/releases/latest/clever-tools-latest_linux.tar.gz
  CLEVER_TAR_FILE: clever-tools-latest_linux.tar.gz
  CLEVER_CLI: clever-tools-latest_linux/clever
  CLEVER_TOKEN: $
  CLEVER_SECRET: $
  REVIEW_APPS_ORGANIZATION_NAME: $
  BRANCH: $

jobs:
  delete:
    runs-on: ubuntu-latest
    if: github.event.label.name == 'recette-jetable' || contains( github.event.pull_request.labels.*.name, 'recette-jetable')

    steps:
    - name: 📥 Checkout to the PR branch
      uses: actions/checkout@v2

    - name: 🏷 Set environnment variables
      run:
        echo "REVIEW_APP_NAME=`echo \"c1-review-$BRANCH\" | sed -r 's/[-;\\/._]+/-/g'`" >> $GITHUB_ENV
        echo "REVIEW_APP_DB_NAME=`echo $REVIEW_APP_NAME | sed -r 's/-/_/g'`" >> $GITHUB_ENV

    - name: 🤝 Find the application on Clever Cloud
      run: |
        curl $CLEVER_TOOLS_DOWNLOAD_URL > $CLEVER_TAR_FILE
        tar -xvf $CLEVER_TAR_FILE
        $CLEVER_CLI login --token $CLEVER_TOKEN --secret $CLEVER_SECRET
        $CLEVER_CLI link $REVIEW_APP_NAME --org $REVIEW_APPS_ORGANIZATION_NAME

    - name: 🗑 Delete the review app and its database
      run: |
        $CLEVER_CLI delete --yes
        $CLEVER_CLI addon delete $REVIEW_APP_DB_NAME --org $REVIEW_APPS_ORGANIZATION_NAME --yes

Conclusion

That’s it for today! Hopefully this is clear enough.

See a typo ? You can suggest a modification on Github.