Position - Azure Data Bricks Developer

Location - Toronto, ON

Full Time Role

Required Skills: Strong knowledge of Data Management and Principles At least over 3 years of experience in Azure Enterprise data lake development, with 6- 8 years of total experience in Data Warehousing. Hands-on experience designing and delivering solutions using Azure Data Lake Experience in building ETL/ELT/ data warehouse. Experience of building data pipelines using Azure Data Factory and Apache Spark (preferably PySpark in Databricks). Experience in working with structured and semi-structured data from various sources like blob (json/avro | delimited flat files | parquet... etc.) , rdbms etc. Adept in Structured Query Language. Ability to apply spark Dataframe api to complete data manipulation within Spark Session Good understanding of Spark Architecture including spark core, spark sql, dataframe, driver node, worker node, fault tolerance, collection, executors and tasks Experience in Delta Lake API.Roles and responsibilities: Designing and implementing efficient data ingestion pipelines from multiple sources using Azure Data factory and/or Azure Databricks Integrating the end-to-end data pipeline to take data from different source systems to target data repositories ensuring the quality and consistency of data. Processing performance analysis and code optimization.