Senior Data Engineer - Databrick

Role Title: Senior Data Engineer (Databricks Specialist & AWS)

Location: Hybrid - 3 days onsite in Woodbridge

Contract Duration: 6-Month Contract to start

Our client is accelerating its digital transformation, running a modern data platform built on AWS and actively moving through a major Databricks implementation (supported by an external vendor). The core objective is transforming data into actionable business products for marketing and promotions. The project team needs a seasoned Senior Data Engineer to help lead the charge, stabilize key deliverables, and guide our technical strategy. This is a critical engagement on a high-visibility, fast-paced project where both strong technical hands-on capability and sharp soft skills are essential.

Key Responsibilities

• Solve foundational data engineering and Medallion architecture (Bronze/Silver/Gold) challenges on AWS Databricks, building a robust ingestion framework for diverse, high-volume datasets.
• Apply practical data modeling principles to streamline data layer creation. Resolve internal debates around model structures and determine optimal ways to structure collection and transactional data for business use.
• Standardize, productionize, and build automated, reusable ML Pipelines. Partner with Data Scientists (who come from non-software engineering backgrounds) to convert standalone models into scalable, production-grade assets.
• Cut through complexity and prevent scope creep ("boiling the ocean"). Evaluate why specific data products are being built, establish clear MVP boundaries, and guide the team on what to tackle first, second, and third.
• Provide guidance, architectural direction, and hands-on mentorship to internal team members, elevating overall data engineering standards.
• Use exceptional communication skills and political acumen to navigate strong, diverse internal opinions, aligning business, technical, and consulting stakeholders toward cohesive technical decisions.

Required Skills & Qualifications

• Proven track record running, building, or enterprise-scaling data platforms in AWS environments.
• Deep expertise in the Databricks ecosystem running natively on AWS (PySpark, Delta Lake, MLflow, Delta Live Tables).
• Practical understanding of dimensional modeling, Medallion architecture, and schema design for analytics. Ability to make pragmatic modeling decisions.
• Demonstrated success creating productionized, reusable machine learning pipelines and assisting Data Scientists with code modularization, CI/CD, and pipeline orchestration.
• Ability to look at data engineering through a product lens—evaluating business utility, setting realistic roadmaps, and delivering incremental value.
• Proven capacity to manage conflicting viewpoints, build consensus among strong technical and business voices, and clearly articulate trade-offs.