Initial Goal: Microsoft Azure Cloud Experience
The original goal of Competency App – Phase 1 was to gain hands-on experience designing and deploying a backend system using a Microsoft-aligned cloud architecture, suitable for real-world enterprise environments.
Development was carried out using Microsoft-backed tooling, including Visual Studio Code and TypeScript, alongside NestJS, GitHub, and Azure cloud services. This ensured close alignment with a modern Microsoft-centric development workflow from local development through to production deployment.
Over an intensive development period of approximately three weeks, the project focused on building, deploying, and operating a production-style data platform rather than prioritising advanced frontend features.
Technology Stack (Phase 1 Azure)
- Frontend: HTML, EJS, JavaScript, Bootstrap 5, jQuery, DataTables
- Backend API: NestJS (Node.js, TypeScript), TypeORM — deployed to Azure App Service
- Data Platform: Azure SQL Database, Azure Data Factory, Azure Blob Storage
- Cloud & DevOps: Microsoft Azure App Service (Linux), GitHub Actions (CI/CD)
Phase 1 focuses on cloud deployment, data ingestion, and API integration.
Phase 1 Architecture (Azure Implementation)
The Phase 1 architecture followed a clear separation of concerns:
- A NestJS (TypeScript) backend API deployed to Azure App Service (Linux)
- A cloud-hosted Azure SQL Database accessed via TypeORM
- Data ingestion and transformation handled using Azure Data Factory
- A lightweight frontend hosted separately and consuming the API securely
Deployment was automated using GitHub Actions, with a CI/CD workflow responsible for building and deploying the backend to Azure App Service on each push to the main branch. This ensured repeatable, version-controlled deployments consistent with modern cloud engineering practices.
As the frontend and backend were deployed independently, Cross-Origin Resource Sharing (CORS) configuration was required to allow secure API access from the frontend application. This reflected a common production scenario where services are decoupled rather than deployed as a single monolithic application.
Competency App Phase 1
A backend cloud deployment of employee competency data, initially hosted on Microsoft Azure services, and later migrated to a self-managed DigitalOcean environment.
What Worked Well on Azure
From a functional and architectural perspective, the Azure-based implementation worked as intended:
- The backend API deployed cleanly and remained stable under normal usage.
- GitHub Actions provided reliable automated CI/CD deployments.
- Azure SQL integrated seamlessly with the application layer.
- The frontend successfully consumed live data through secure API endpoints.
One operational limitation encountered was Azure App Service’s tendency to scale down or idle under low usage. To mitigate this, a small NestJS module was introduced to periodically ping the Azure SQL database, ensuring the backend remained responsive.
While effective, this approach felt inefficient relative to the ongoing platform cost, and highlighted a mismatch between the project’s scale and the pricing model of the managed cloud environment.
Why a Migration Was Considered
Despite the technical success of the Azure deployment, the platform proved disproportionately expensive relative to the size and purpose of the project.
Additionally, much of the operational behaviour of the system was handled automatically by managed services, limiting visibility into:
- Process lifecycle management.
- Environment configuration.
- Network boundaries and request flow.
Rather than abandoning the project, this created an opportunity to explore an alternative hosting model while preserving the same architectural principles and application behaviour.
Migration to a Self-Managed Cloud Platform
Following the Azure phase, the backend API and database were migrated to a self-managed cloud environment.
- The NestJS backend API and PostgreSQL database are hosted on DigitalOcean.
- The frontend application continues to be hosted independently on Hetzner.
This preserved a distributed, production-style architecture with clear separation between frontend delivery, backend services, and data storage, while significantly reducing operational costs.
Deployment Architecture Overview
Frontend
HTML, EJS, Bootstrap, jQuery
Hosted on Hetzner
Nginx Reverse Proxy
HTTPS Termination
API Routing /api/*
NestJS API
Node.js / TypeScript
DigitalOcean VPS (PM2)
PostgreSQL
Relational Data Store
DigitalOcean Managed DB
Requests flow from the frontend through an Nginx reverse proxy which handles HTTPS termination, forwards API traffic to a private NestJS backend service, and connects securely to PostgreSQL.
Final DigitalOcean Deployment (Post-Migration)
- Frontend: HTML, EJS, JavaScript, Bootstrap 5, jQuery, DataTables – Hetzner hosted
- Reverse Proxy: Nginx – HTTPS termination & API routing
- Backend API: NestJS (Node.js, TypeScript), TypeORM, PM2 – DigitalOcean VPS
- Database: PostgreSQL & PGadmin4 – DigitalOcean Managed Database
- CI/CD: GitHub Actions automated deployments
What Changed — and What Stayed the Same
While the hosting platform changed, the core application architecture remained consistent.
Preserved:
- NestJS backend design and REST API contracts.
- Database schema and access patterns.
- Frontend behaviour and user experience.
Changed:
- Infrastructure ownership and responsibility.
- Deployment and process management.
- Network configuration and security handling.
A key part of this transition involved configuring Nginx as a reverse proxy, responsible for HTTPS termination and routing API requests securely to the backend service.
This ensured encrypted client communication while keeping the backend isolated from direct public access.
Key Technical Lessons Learned
Operating the platform outside a fully managed cloud environment, surfaced several important real-world considerations:
- Environment variable handling across different process managers.
- Database schema resolution and permission configuration.
- Reverse proxy setup and HTTPS request flow.
- The practical differences between managed and self-managed infrastructure.
These challenges became apparent, through hands-on operation of the system, and provided practical insight into the responsibilities typically handled automatically by managed cloud platforms.
Final Outcome and Reflection
This phase of the project successfully achieved two distinct objectives:
- Demonstrating practical experience with Microsoft Azure cloud services and Microsoft-aligned development tooling.
- Applying critical engineering judgement to evaluate cost, control, and long-term suitability.
The final outcome is a more transparent, cost-effective deployment that preserves the original architecture while providing deeper operational understanding.
The project demonstrates not only the ability to deploy a cloud-based system, but also the ability to assess infrastructure decisions critically based on real-world constraints.
| First Name | Last Name | Job | Legal Entity | Discipline | Projects | Qualifications | CPD |
|---|