Senior Engineer - Tata Consultancy Service Limited : Job Details

Technical/Functional Skills DataBricks Engineer/ArchitectExperience Required •Minimum 10+ years of experience inDesign, Development, Architect, implementation, and analytics solutions•Minimum 5+ Years of Experience on DataBricks especially on Architecture side.•Experiencein creating Databricks environment, Dag, access pattern, workflows, Schema setup etc.•Designscalable and efficient data architectures using Databricks.•Hands on- Design and Develop applications using Databricks.•Candidatesmust have hands on in Python core programming and Spark.•Goodexperience/knowledge on AWS•Developand manage ETL (Extract, Transform, Load) pipelines using Databricks to processand transform large volumes of data.•From anAWS perspective, candidates need to be very well versed with Terraform, Glue,Lambda, S3 and their usage and practical issues faced when dealing with them.•At least5+ Experience on AWS (Glue, S3, lambda at the minimum)•Abilityto design and demonstrate system Architecture with different environments.•Optimizethe performance of Databricks clusters, queries, and workloads.•Hands onexperience on Data Warehousing and SQL•Experienceon Pandas•Strongexpertise in SQL is required.•Solidexperience with git software version control and goodunderstanding of code branching strategies and organizationfor code reuse•VeryStrong Data Bricks Architecture/Development knowledge•Experiencewith DevOps & CI/CD tools Jenkins, Cloudbees, Please Build, etc.•Proficiencyworking with large data stores and data sets•Healthcare knowledge is preferred.•Agilemethodology (Jira Preferable)Roles & Responsibilities •Must have capability to understand the current Architecture•Lead theteam and individual contribution as well.•Design,Coding, testing and implementation of cloud-based Data bricks apps•Support existing code thru enhancing, trouble shooting andmaintaining per requirements.•Implementsecurity best practices, including data encryption, access control, and securedata sharing.•Supporton AWS migration•Supportto understand current ETL process Python or Scala•maintain consistent practices for source code managementin github•comply with established CICD practices, and participate inthe creation of new CICD methods•attend daily scrum calls, and actively participate inupdating/creating user stories and tasks in Jira•Workclosely with business analysts.•Leadinnovation by exploring, investigating, recommending, benchmarking, andimplementing data centric technologies for the platform.•Collaboratingwithin a project team to solve complex problems.•Lead byexample coaching and mentoring those of your peers and mentoring lessexperienced team members. Operate as part of the extended management team.Salary Range: - $90,000 to $130,000 a year