Scala Spark Lead - JobRialto : Job Details

Mandatory Skills: Big Data, Scala, Spark, Core Java BigData - 7+ years Scala - 8+ years Spark - 7+ years Core Java - 5+ years Experience: 10+ years Client is a leading provider of platforms; digital innovation; artificial intelligence and end-to-end IT services and solutions for Global 1000 companies. We are transforming corporations through deep domain expertise; knowledge-based ML platforms; as well as profound anthropological efforts to understand the end customer and design products and interactions that create delight. We are deeply committed to developing a comprehensive understanding of our client's problems and developing platforms to address them. We are seeking a highly skilled and motivated Spark Scala Developer to join our dynamic team. As a Spark Scala Developer, you will play a critical role in the design, development, deployment, and optimization of data processing applications. Key Responsibilities: Develop and maintain data processing applications using Spark and Scala. Collaborate with cross-functional teams to understand data requirements and design efficient solutions. Implement test-driven deployment practices to enhance the reliability of applications. Deploy artifacts from lower to higher environments ensuring smooth transition. Troubleshoot and debug Spark performance issues to ensure optimal data processing. Work in an agile environment, contributing to sprint planning, development, and delivering high-quality solutions on time. Provide essential support for production batches, addressing issues and providing fixes to meet critical business needs. Skills/Competencies: Strong knowledge of Scala programming language. Excellent problem-solving and analytical skills. Proficiency in Spark, including the development and optimization of Spark applications. Ability to troubleshoot and debug performance issues in Spark. Understanding of design patterns and data structures for efficient data processing. Familiarity with database concepts and SQL. Java and Snowflake (Good to have). Experience with test-driven deployment practices (Good to have). Familiarity with Python (Good to have). Knowledge of Databricks (Good to have). Understanding of DevOps practices (Good to have). Expectations from this role: Act creatively to develop applications and select appropriate technical options, optimizing application development, maintenance, and performance by employing design patterns and reusing proven solutions. Account for others' developmental activities. Interpret the application/feature/component design to develop the same in accordance with specifications. Code, debug, test, document, and communicate product/component/feature development stages. Validate results with user representatives; integrate and commission the overall solution. Select appropriate technical options for development such as reusing, improving, or reconfiguring existing components or creating own solutions. Optimize efficiency, cost, and quality. Influence and improve customer satisfaction. Set FAST goals for self/team. Typical performance measures: Adherence to engineering process and standards (coding standards). Adherence to project schedule/timelines. Technical issues uncovered during the execution of the project. Defects in the code. Defects post-delivery. Non-compliance issues. On-time completion of mandatory compliance training. Performance Areas: Code as per design. Follow coding standards, templates, and checklists. Review code - for team and peers. Documentation: Create/review templates, checklists, guidelines, and standards for design/process/development. Create/review deliverable documents, design documentation, requirements, test cases/results. Configure: Define and govern configuration management plans. Ensure compliance from the team. Test: Review and create unit test cases, scenarios, and execution. Review test plan created by the testing team. Provide clarifications to the testing team. Domain relevance: Advise Software Developers on the design and development of features and components with a deep understanding of the business problem being addressed for the client. Learn more about the customer domain identifying opportunities to provide valuable additions to customers. Complete relevant domain certifications. Manage Project: Manage delivery of modules and/or manage user stories. Manage Defects: Perform defect RCA and mitigation. Identify defect trends and take proactive measures to improve quality. Estimate: Create and provide input for effort estimation for projects. Manage knowledge: Consume and contribute to project-related documents, share point, libraries, and client universities. Review the reusable documents created by the team. Release: Execute and monitor the release process. Design: Contribute to the creation of design (HLD, LLD, SAD)/architecture for Applications/Features/Business Components/Data Models. Interface with Customer: Clarify requirements and provide guidance to the development team. Present design options to customers. Conduct product demos. Manage Team: Set FAST goals and provide feedback. Understand the aspirations of team members and provide guidance, opportunities, etc. Ensure the team is engaged in the project. Certifications: Take relevant domain/technology certifications. Skill Examples: Explain and communicate the design/development to the customer. Perform and evaluate test results against product specifications. Break down complex problems into logical components. Develop user interfaces, business software components. Use data models. Estimate time and effort required for developing/debugging features/components. Perform and evaluate tests in the customer or target environment. Make quick decisions on technical/project-related challenges. Manage a team. Maintain high motivation levels and positive dynamics in the team. Interface with other teams, designers, and other parallel practices. Set goals for self and team. Provide feedback to team members. Create and articulate impactful technical presentations. Follow high levels of business etiquette in emails and other business communications. Drive conference calls with customers addressing customer questions. Proactively ask for and offer help. Ability to work under pressure, determine dependencies, risks, facilitate planning; handling multiple tasks. Build confidence with customers by meeting the deliverables on time with quality. Estimate time and effort, resources required for developing/debugging features/components. Make appropriate utilization of Software/Hardware. Knowledge Examples: Appropriate software programs/modules. Functional and technical designing. Programming languages - proficient in multiple skill clusters. DBMS. Operating Systems and software platforms. Software Development Life Cycle. Agile - Scrum or Kanban Methods. Integrated development environment (IDE). Rapid application development (RAD). Modelling technology and languages. Interface definition languages (IDL). Knowledge of customer domain and deep understanding of the subdomain where the problem is solved. Education: Bachelors Degree