- Technical Leadership:
- Provide hands-on technical leadership to the team in designing and implementing data engineering solutions.
- Lead by example in adopting best practices for coding, testing, and deployment.
- ETL Development:
- Design and develop robust ETL pipelines using AWS Glue, Lambda, and other AWS services to process large volumes of data efficiently.
- Implement complex data transformations and integrate data from multiple sources such as APIs, databases, and streaming platforms.
- Distributed Data Processing:
- Develop distributed data processing frameworks to ensure performance and scalability in handling large datasets.
- Optimize the performance of data processing jobs for both batch and real-time workloads.
- Solutioning & Architecture:
- Provide optimal data engineering solutions aligned with client requirements and business objectives.
- Collaborate with architects to design scalable and secure data solutions leveraging AWS cloud services.
- AWS Expertise:
- Utilize AWS services (e.g., S3, Glue, Lambda, Kinesis, DynamoDB) to build efficient and scalable cloud-based solutions.
- Stay updated with the latest AWS services and features to continuously improve system performance and cost efficiency.
- Stakeholder Collaboration:
- Work closely with clients, business analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Communicate progress, challenges, and solutions effectively to both technical and non-technical stakeholders.
- Education:
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- Experience:
- 10+ years of experience in data engineering.
- Strong hands-on expertise in Python, PySpark, and AWS services for data processing and integration.
- Extensive knowledge of distributed data processing frameworks and best practices.
- Skills:
- Strong problem-solving and solution-oriented mindset to deliver optimal results.
- Excellent knowledge of data integration techniques and cloud-based architecture.
- Proficient in implementing complex data transformations and scalable data workflows.
- Exceptional team leadership and mentoring abilities.
- Strong communication skills for effective stakeholder collaboration.
- AWS Certified Solutions Architect certification or equivalent.
- Familiarity with Terraform or CloudFormation for AWS infrastructure as code.
- Wealth Management domain experience.
- Be a key player in building cutting-edge data engineering solutions for large-scale projects.
- Work with a talented and collaborative team in a dynamic environment.