Job Title: Sr. Data Engineer

Job Type: Permanent role, 4 days on-site located in Calgary, Alberta.

Job Salary: $130k - $145k / Year.

JOB DESCRIPTION

Insight Global is hiring a Sr. Data Engineer in Calgary, AB. This role is hybrid - in-office with 20% flexibility to work from home. This resource will play a key role in architecting and optimizing a modern, scalable, and productized data ecosystem that enables internal teams and external partners to derive maximum value from our data assets. Responsible for designing high-performance data pipelines, storage solutions, and API-driven data access mechanisms that support real-time decision-making and innovation. The ideal candidate has 5+ years of experience in data engineering, with modern data storage and processing, proficiency in ETL frameworks and event-driven architectures. They will also have strong API development skills and proficiency in Go, Java and/or Python.

Duties & Responsibilities:

- Design and build a scalable, cloud-native data platform aligned with microservices

- Develop real-time and batch data pipelines to power data-driven products

- Implement SQL, NoSQL, and hybrid storage strategies Data as a Product

- Enable self-serve data access with secure, well-documented data APIs.

- Collaborate with Product & Business teams to define and optimize data products.

- Ensure data quality, lineage, and governance in all data pipelines and products. Microservices, Integration & Performance

- Build event-driven architectures using Kafka, Azure Event Hub, or Service Bus.

- Develop scalable ETL/ELT processes for ingestion, transformation, and distribution.

- Optimize query performance, indexing, and caching for data-intensive apps. Data Governance, Security, and Compliance

- Enforce data privacy, security, and access controls aligned with compliance standards.

- Implement observability and monitoring for data infrastructure and pipelines.

- Work with DevSecOps teams to integrate security into CI/CD workflows.

REQUIRED SKILLS AND EXPERIENCE

- 5+ years of experience in data engineering, with exposure to Data Mesh and Data as a Product preferred.

- Expertise in modern data storage and processing, including SQL, NoSQL (Cosmos DB, PostgreSQL), Data Lakes (Azure Data Lake, Delta Lake, Apache Iceberg).

- Proficiency in ETL frameworks (e.g. Apache Kafka, Airflow, Flink, Spark, Azure Data Factory, Databricks).

- Experience with Event-driven architectures using Queues, Pub/Sub services (e.g. Azure Service Bus, Azure Event Grid, Amazon Event Bridge) and Containerized Environments (Container Apps, AWS ECS).

- Experience with Apache and/or Azure data platforms or similar, e.g. Fabric, Databricks, Snowflake, and Apache Hudi.

-Strong API development skills using GraphQL, REST, and/or gRPC for enabling data as a product.

- Proficiency in Go, Java, and/or Python.

-Deep understanding of data governance, security, lineage, and compliance using Microsoft Purview, OpenLineage, Apache Ranger, or Azure Key Vault.

- Experience with Infrastructure as Code (IaC) using Bicep, Terraform, or CloudFormation for managing cloud-based data solutions.

- Strong problem-solving and collaboration skills, working across data, engineering, and business teams.

NICE TO HAVE SKILLS AND EXPERIENCE

- Knowledge of ML, AI, and LLMs, including data engineering for model training and inference with Azure Machine Learning, Databricks ML, and MLflow.

- Hands-on experience in Notebooks (e.g. Jupyter, Databricks, Azure Synapse) for data workflows.

- Experience in real-time data streaming architectures.

- Exposure to data monetization strategies and analytics frameworks.

- Familiarity with federated data governance and self-serve data platforms.