Description

• Collaborate with business teams and Data Analysts to gather and understand requirements, ensuring alignment with business objectives and data needs.
• Translate business needs into detailed technical requirements in collaboration with subject matter experts (SMEs) to ensure accuracy and feasibility.
• Recommend and design scalable, efficient, and effective data architectures and workflows to support current and future business requirements.
• Design, develop, and maintain data assets to enable seamless extraction, transformation, and loading (ETL/ELT) processes from diverse data sources, making data accessible to client-facing applications, data warehouses, and internal tools.
• Build, operate, and optimize highly scalable and reliable data pipelines and infrastructure to support analytics, reporting and operational use cases.
• Drive end-to-end ownership of projects, including planning, development, testing, and deployment, ensuring timely and successful delivery.
• Collaborate with Quality Assurance (QA) and Support teams to identify, troubleshoot, and resolve issues in production environments, ensuring the stability and reliability of data systems.
• Work with Release Management to plan, coordinate, and implement data pipeline updates, adhering to CI's deployment standards and minimizing disruption to production systems.
• Implement and enforce best practices for observability, data lineage, and governance, ensuring transparency, reliability, and compliance across all data systems.
• Support and mentor junior team members, fostering a collaborative and growth-oriented environment while providing technical guidance and expertise.
• Lead or participate in data migration projects, transitioning legacy systems to modern platforms and architectures while minimizing disruption and data loss.

Experience

• 6+ years of professional experience in data engineering, with a strong focus on designing, developing, and optimizing scalable data pipelines, ETL/ELT workflows, and data integration solutions using modern tools and technologies.

Education/Training

• Post-secondary degree in a quantitative discipline.

• Comprehensive understanding of data pipeline architecture, modern data stack architecture, and cloud-based platforms, including AWS, Snowflake, and other cloud-native solutions.
• In-depth knowledge and experience with the following tools and concept:
  • Data extraction – SQL, Python, API invocation, CDC
  • Database systems – PostgreSQL, Sybase, MySQL, DynamoDB
  • Data storage repositories – SFTP, AWS S3
  • Scheduling of data jobs – CRON, Apache Airflow, AWS Step Functions
  • ETL/ ELT tools and workflow – Snowflake, PySpark, AWS Glue, EMR, AWS Lambda, SCD
  • CI/CD pipelines – Bitbucket, Git, Jenkins, CloudFormation, Terraform, Flyway
• Strong knowledge of observability and data lineage implementation to ensure pipeline transparency and monitoring.
• A strong analytical mindset and sophisticated written and verbal communication skills.
• Experience in leading or contributing to migration projects to modern data platforms like Snowflake and/or AWS.
• Experience in the Financial Services Industry is an asset.
• Ability to work within an organization based upon continuous improvement.