Databricks Data Engineer developing scalable data pipelines within the insurance domain. Building ETL workflows on Azure and Databricks for analytics and business decision-making.
Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Azure technologies
Build and optimise PySpark and SQL-based data processing workflows
Work closely with data analysts, reporting teams, and business stakeholders
Develop and maintain data models aligned to insurance data structures
Implement robust data validation and quality frameworks
Monitor and troubleshoot data pipelines and batch jobs
Identify and resolve performance bottlenecks
Ensure data governance, regulatory compliance, and auditability
Participate in Agile delivery processes
Requirements
7–12 years of experience in Data Engineering
Strong hands-on experience in Databricks and cloud-based data platforms
Proven experience working within the insurance industry
Strong understanding of insurance data domains
Proficiency in PySpark, Python, and SQL
Extensive experience building ETL/ELT pipelines and scalable data architectures
Hands-on experience with Azure cloud technologies
Experience working with Delta Lake architecture and modern data warehousing patterns
Familiarity with Agile methodologies
Benefits
A competitive salary with a generous bonus
Private healthcare
Critical illness life assurance at 4 x your annual salary
Income protection insurance
A rewarding pension
Cash back cards for discounts and promotions
Cycle Scheme for savings on bikes and cycling accessories
Professional and personal development opportunities
Data Platform Expert developing and maintaining data solutions for analysis and reporting at Magna Electronics. Collaborating with various teams to enhance data - driven decision making and insights.
Data Engineer at Mobiz designing, building, and maintaining scalable data solutions for analytics. Collaborating with teams to leverage modern cloud technologies and improve data - driven decision - making.
Head of Data Engineering at Envitia overseeing data architecture services for public sector programs. Leading service mobilization with client stakeholders in a hybrid work environment.
Applied AI Health Data Architect - Senior Manager at PwC designing data architecture for healthcare operations. Contributing to innovative data solutions and mentoring teams for operational excellence.
CRM Functional Data Migration Engineer at Data - Core Systems, Inc. managing Dynamics 365 CE solutions and delivering technical expertise in a consulting role.
Data Engineer role for Alberta Securities Commission, focusing on data pipelines and infrastructure. Involves designing, building, and securing data assets for organizational decision - making.
Data Architect supporting UK MOD programme delivering data and integration capabilities for Defence. Join Amentum's growing IT team focused on critical national infrastructure and security projects.
Data Engineer joining Contour Software's development team to handle various big data projects. Focus on cloud data lake architecture and machine learning implementation.
Enterprise Architect Sr position at PNC collaborating with technology stakeholders to build enterprise data architecture. Focus on data models, governance, and guiding data decisions for business strategy.
Data Engineer architecting and maintaining data products using Databricks and Power BI for Nitro's digital transformation efforts. Collaborating to optimize processes and leverage cloud technologies.