SNS and SQS are AWS services which are often used in an event-driven architecture. A combination of those services offers the possibility to fan-out messages. Often there is a need to filter these messages before replicating them to multiple queues. Now I want to explain a way to achieve this by taking full advantage of these AWS services and without the need to write any custom code.

Let’s start with a quick refresher on SNS-SQS fan-out. SNS is an AWS service that coordinates and manages the delivery or sending of messages to subscribing endpoints. It offers you the possibility to…

SAM is an open-source framework that you can use to build, test and deploy serverless applications on AWS.

The main goal of this article is to familiarize you with AWS SAM so you can test your lambdas on your local machine without the need to (re)deploy them every time on AWS. I’ll discuss different integrations and examples with SAM and lambda.

SAM uses Docker to build and run lambdas locally, so a basic knowledge of Docker is required.

Architecture

Let’s say there’s a company with a lot of documents. These documents have a documentId and a versionId. …

Amazon Textract is a service that automatically extracts text and data from scanned documents. Amazon Textract also identifies the contents of fields in forms and information stored in tables. In this demo project I’ll use Textract to read data and identify tables in PDF documents. These PDF documents represent invoices. The data will be written to an RDS Postgres database. It will make it possible to create a relational database with invoice data from PDF’s. This will give us the opportunity to query the data and collect useful information about the invoices.

Below you can find a high level representation…

In this blog I’ll show how we can create a centralized logging solution where we collect all our Docker logs from different containers. Here we will use the well-known ELK stack (Elasticsearch, Logstash, Kibana).

We will use docker-compose to deploy our ELK stack. Docker-compose offers us a solution to deploy multiple containers at the same time.

Prerequisites:

• Docker
• Docker-Compose

The full template is available on my GitHub. Clone the repository and start the containers by using Docker-compose.

$ git clone https://github.com/lvthillo/docker-elk.git
$ cd docker-elk
$ docker-compose up -d

The first resource which is deployed is Elasticsearch. Elasticsearch is a system…

Amazon EC2 Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. Amazon provides some default CloudWatch Metrics like CPUUtilization or DiskWriteOps but to scale based on Memory Utilization we are forced to create a custom metric.

We will monitor the average RAM (memory) utilization inside our Auto Scaling group and use that metric to make scaling decisions. You can find the full CloudFormation template on my GitHub. It is validated with cfn-lint.

Let’s begin very high level. We need an Auto Scaling Group.

Nothing advanced…

In this tutorial I’ll explain how you can implement a continuous Spring Boot deployment in Kubernetes. A small code change will trigger a new Docker image build and an instant deploy of a new pod in your cluster.
This tutorial is executed on macOs.

Jib is an open-source, fast and simple Java container image builder from Google that handles all the steps of packaging your application into a container image. It does not require you to write a Dockerfile.

We will use the jib-maven-plugin to create a Docker image for our application.
Next we will setup a local minikube cluster and…

In this tutorial I’ll explain how you can setup a secure and high available virtual private cloud architecture and automate the setup using CloudFormation.
We will host a basic high available application inside this VPC.
We will also take the cost into account which means we will not make non-critical resources high available.

High Availability in AWS

In AWS you can deploy resources and use services in different regions like eu-west-1 (EU Ireland), us-east-1 (US N. Virgina), and many more. Every AWS region consists of multiple availability zones. An availability zone (AZ) consists of one ore more data centers which are heavily secured. Availability…

In this tutorial I’ll show how you can safely connect your local or on-premise Python application to AWS services like S3. In many cases the connection is made by just creating an IAM user and attach the correct policy. This will work but is not the recommended approach by AWS. It’s recommended to use IAM roles, even if your application is not running inside AWS.

The Python application will have a web interface in which files can be uploaded to an S3 bucket. The application is running on a server inside the on-premise data center of a company. …