Job Summary We are seeking a highly skilled Sr. Developer with 6 to 10 years of experience to join our dynamic team. The ideal candidate will have extensive experience in Core java Spark in Scala Spark and SparkSQL. This hybrid role offers the flexibility of remote work combined with the benefits of in-office collaboration. The candidate will work during day shifts and will not be required to travel. Responsibilities
- Develop and maintain scalable data processing pipelines using Java Spark in Scala to ensure efficient data handling and processing.
- Collaborate with data scientists and analysts to understand data requirements and translate them into technical specifications.
- Optimize Spark jobs for performance and scalability to handle large datasets effectively.
- Implement data transformation and aggregation logic using SparkSQL to support various business needs.
- Conduct code reviews and provide constructive feedback to ensure code quality and best practices.
- Troubleshoot and resolve issues related to data processing and pipeline failures.
- Work closely with DevOps teams to deploy and monitor Spark applications in production environments.
- Provide technical guidance and mentorship to junior developers to foster their growth and development.
- Stay updated with the latest advancements in Spark and big data technologies to incorporate best practices.
- Collaborate with cross-functional teams to integrate Spark solutions with other data platforms and tools.
- Ensure data security and compliance with company policies and industry regulations.
- Document technical designs processes and procedures to maintain a knowledge base for the team.
- Participate in agile development processes including sprint planning stand-ups and retrospectives. Qualifications
- Possess a Bachelors degree in Computer Science Information Technology or a related field.
- Have a minimum of 6 years of experience working with Spark in Scala Spark and SparkSQL.
- Demonstrate strong problem-solving skills and the ability to troubleshoot complex issues.
- Exhibit excellent communication skills to effectively collaborate with team members and stakeholders.
- Show proficiency in optimizing Spark jobs for performance and scalability.
- Display experience in working with large datasets and data processing pipelines.
- Have a solid understanding of data security and compliance requirements.
- Be familiar with agile development methodologies and practices.
- Demonstrate the ability to mentor and guide junior developers.
- Show a proactive approach to learning and staying updated with the latest technologies.
- Possess strong documentation skills to maintain clear and comprehensive technical records.
- Exhibit the ability to work independently and as part of a team in a hybrid work model.
- Have experience in deploying and monitoring Spark applications in production environments.