Senior Data Engineer (Azure/ Databricks) - Fraser Carver Executive Search Ltd : Job Details

Fraser Carver Executive Search are working with a growing FinTech company who are looking to hire a Senior Data Engineer into the Data Team. This is a permanent role and we are not in a position to offer sponsorship at this time so please only apply if you have unrestrictred right to work in the UK.

Role overview:

We are looking for a Senior Data Engineer reporting into the Data Director to bring our data infrastructure to the next level in terms of reliability and stability, with a view on establishing a larger data mesh set-up managed from a hub of excellence. In the past 3 years we have set up the companies data infrastructure from scratch. We have built a lakehouse with automated reporting and self-serve on top of that. With expanding scope and responsibilities, we need to mature our solution to retain its quality and design how we scale efficiently. The focus of this role will be on DataOps/DevOps, ingestion and data processing within Databricks. The responsibility will be end-to-end due to our small team size, with a real ability to make a difference.

Main responsibilities:

• Mature the data infrastructure to ensure reliability and stability
• Manage ingestion pipelines (currently ADF and python)
• Ensure the quality of raw datasets for the Data Analysts to create a data model on
• Manage and deploy data related tools on Kubernetes (Airflow, Superset, RStudio Connect)
• Support Data Analytics work through management of DBT, deployment rules, DevOps

Skills / Experience required:

• Relevant Data Engineering experience on Azure/ Databricks/ Delta Lake
• Experienced in Kubernetes/ Infrastructure as Code/ Terraform
• Well versed in Azure DevOps
• A strong preference for simplicity and transparency over complexity
• Be innovative, drive forward and make solutions work using best practices

This role is pretty much fully remote but will require visiting the head office near Oxford once a month or if there are important team meetings throughout the year which require your input.