Architecture

Infrastructure and services description.
2 parts: 1 ensemble split within 2 vms and maintenance side

Client side

  • WEB a vm with docker services (docker-compose)
    • poll service: a flask app
    • worker a java jvm v8
    • result a flask app
    • and a traefik servcie
  • DB a vm with services installed
    • redis app
    • psql app
      These vms are reachable through a public ip auth with pem file. Protocol ssh.

TODO: we suggest storing pem fil on vault to get it from API.

REM: network is highly unstable!!!
app-diagram

monitoring side

  • monitoring aws node
    • an aws instance with grafana stack
  • backup aws node
    • perform db backup action
    • perform container state backups

Setup project

Here we describe how to start the project

Dependancies

1
2
3
docker -v
docker-compose -v
git -v

pull project

1
2
3
git clone $PROJECT && cd 
docker-compose pull
docker-compose up -d --build

Project Usecase

System view of the project

Requirements US

  • REQ100: as a maintainer I want to be able to read web vm status in order to perform monitoring
  • REQ200: as a maintainer I want to save database in order to be able to restore it later
  • REQ300: as a maitainer I want to be able to backup apps in order to restore it faster
  • REQ400: as a maintainer I want to be sure my connection are not readable in order to improve security

Tasks

tasks from REQ100

  • T100 connect playbook with azure vm

  • T110: install node_exporter on web vm

  • T111: install node_exporter on db vm
  • T112: install node_exporter on monitoring side / testing - optionnal

  • T120: test network between vm and between networks

  • T130: store ssh keys on vault cloud

tasks from REQ200

  • T200: perform database backup command
  • T210: perform database restore command

tasks from REQ300

  • T300: perform docker app backup to tar
  • T301: perform docker app restore from tar (runtime?)
  • T302: perform psql package/config backup/restore
  • T303: perform redis package/config backup/restore
  • T400: certbots the connections

Tricks to backup db

When node is stopping, data might be lost.

Backup

1
docker exec -t your-db-container pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql

Restore

1
cat your_dump.sql | docker exec -i your-db-container psql -U postgres

backup docker

most practical way to save stuff

export process

1
2
docker commit
docker export 265e1f9c08a0 > /tmp/poll.tar

import method

1
2
scp poll.tar
docker import /tmp/poll.tar

REM: check if data timestamp to keep the latest version.

Welcome to MkDocs

This is a test site hosted on GitLab Pages. You can
browse its source code, fork it and start
using it on your projects.

For full documentation visit mkdocs.org.

Commands

  • mkdocs new [dir-name] - Create a new project.
  • mkdocs serve - Start the live-reloading docs server.
  • mkdocs build - Build the documentation site.
  • mkdocs help - Print this help message.

Project layout

mkdocs.yml    # The configuration file.
docs/
    index.md  # The documentation homepage.
    ...       # Other markdown pages, images and other files.