Project case study
Retail Data Integration Platform
Full-Stack Engineer · Project Technical Lead
Nov 2021 — Jan 2023An enterprise retail data-integration and ETL platform with asynchronous messaging, distributed job scheduling, and an operations dashboard.
Scope
The system processed retail and operational data through scheduled and continuous services. It included Azure WebJobs, Service Bus messaging, Quartz scheduling, SQL Server, integrations, telemetry, and a web application for monitoring and managing ETL operations.
Role & delivery
I was the technical lead for the .NET and backend implementation and also built the operations web application. Working with a database engineer and project manager, I led the service architecture, ingestion pipelines, scheduling mechanics, and production observability.
Selected work
- Reworked the legacy .NET processing layer across Azure WebJobs, Service Bus, Quartz scheduling, retries, resilience, configuration, and Application Insights observability.
- Built high-throughput store telemetry ingestion using System.Threading.Channels producer-consumer architecture, replacing slow batch iteration with streaming parallel processing.
- Implemented and maintained processing services for the ETL flow, including dependency-aware checks before warehouse operations, duplicate-run protection, and deadman timeout monitoring.
- Built an ASP.NET Core and Angular operations application for visibility and controlled intervention: operators could monitor live job states and trigger controlled backload runs.
- Integrated Azure Service Bus and SignalR for asynchronous background work and live dashboard telemetry streaming.
- Standardized scoped dependency injection lifecycles inside Quartz background jobs, eliminating EF Core context concurrency issues.