Sr. Python/Pyspark Developer

As a senior Python and PySpark developer, your challenge is to architect and deliver scalable, high-performance data processing solutions that empower data-driven decision-making. You will tackle complex data integration, transformation, and analytics workloads by designing robust applications, optimizing SQL queries, and streamlining workflows. Your expertise will enable the organization to leverage cloud-scale data platforms effectively, maintain high code quality through rigorous testing, and foster collaboration within agile teams to innovate and maintain cutting-edge data solutions.

Responsibilities:

• Design, develop, and maintain robust Python and PySpark applications to support large-scale data processing, transformation, and analytics initiatives.

• Implement and execute unit testing and code quality checks using tools like Pytest and Pylint to ensure reliable and maintainable codebases.

• Design, optimize, and troubleshoot complex SQL queries, primarily in tSQL, to support data extraction, transformation, and loading processes.

• Develop and manage data integration workflows utilizing ETL/ELT frameworks, enterprise schedulers, and scripting in Python and other relevant tools.

• Collaborate with cross-functional teams to gather requirements, create high-level system specifications, and produce detailed design documents for data solutions.

• Implement RESTful API integrations to connect various data sources and applications, ensuring seamless data flow and accessibility.

• Lead and contribute to agile development processes, participating in sprint planning, stand-ups, and code reviews to ensure timely delivery of features.

• Leverage frameworks such as Django, Flask, Pyramid, or Tornado to build scalable web applications or APIs as needed.

• Utilize version control systems like Git, and manage development pipelines to support continuous integration and deployment.

• Apply best practices in software architecture, data structures, and algorithms to optimize application performance and scalability.

• Coordinate with enterprise teams on scheduling and automation tasks using enterprise schedulers and SSIS, enhancing operational efficiency.

• Lead technical discussions and influence stakeholders to adopt best practices and innovative solutions for data processing and application development.

• Maintain thorough documentation of system architecture, design specifications, and coding standards to support ongoing maintenance and knowledge sharing.

• Stay updated on emerging technologies and industry trends to recommend improvements and ensure the solutions remain cutting-edge.

Requirements:

• At least 10 years of Experience in Python development

• Must have hands-on experience in Python and Pyspark

• Experience in Python testing and code analysis tools like Pytest and Pylint

• At least 5 years of fulltime development experience using Python.

• At least 5 years working with tSQL, XML and enterprise schedulers

• Proficient understanding of algorithms, data structures, architectural design patterns and best practices

• Extensive experience in software development and the entire SDLC.

• Solid understanding of a variety of programming tools and development platforms.

• Excellent organizational and leadership abilities.

• Highly analytical mindset, with an ability to see both the big picture and the details.

• Experience in creating high level product specifications and design documents.

• Proven collaboration and influencing skills

• Experience in writing Python applications with the benefit of frameworks like Django, Flask,

• Pyramid, or Tornado.

• Integration experience (ETL, ELT) with Python

• Familiarity with SSIS would be helpful

• General development expertise, use of version control, ticketing, and continuous integration systems.

• Experience in an Agile Development environment

• Experience in consuming API's (i.e. Rest, etc.)
  

Preferred, but not required:

• Detail-oriented, ability to turn deliverables around quickly with a high degree of accuracy

• Strong analytical skills, ability to interpret business requirements and produce functional and technical design documents.

• Good time management skills – Ability to prioritize and multi-task, handling multiple efforts at once.

• Strong desire to understand and learn domain.

• Experience in a financial services/banking industry

• Ability to work in a fast-paced environment; to be flexible and learn quickly.

• Ability to multi-task with attention to detail/ prioritize tasks.