Data Engineer specializing in Azure Databricks and healthcare data pipelines. Responsible for designing high-throughput ingestion pipelines and ensuring compliance with data standards.
Responsibilities
This role requires a highly technical Data Engineer with expert-level proficiency in Azure Databricks, distributed data pipelines, and large-scale healthcare data processing.
This role focuses on designing and implementing high-throughput ingestion pipelines, transactional lakehouse layers, and secure PHI data flows using Azure-native services and Databricks runtime optimizations.
You will build and operate production-grade data pipelines that meet rigorous requirements for security, lineage, compliance (HIPAA), observability, and operational SLAs, supporting analytics, AI, and clinical insights across the organization.
Requirements
5+ years of experience in modern data engineering roles
Expert-level proficiency in: PySpark and Spark SQL
Databricks (Jobs, Workflows, Repos, Delta Live Tables)
Delta Lake architecture and transactional design patterns
Azure Data Factory or Azure Synapse Pipelines
Cloud-native data security (RBAC, ABAC, privilege boundary enforcement)
Strong experience working with healthcare data formats and standards: FHIR (JSON)
HL7 v2/v3
X12 EDI claims data
Deep understanding of distributed systems, data partitioning strategies, concurrency, and cluster resource tuning
Benefits
Comprehensive health, dental, and vision insurance
Health Savings Account with an employer contribution
Senior Data Engineer at Goodwin enhancing data platforms and fostering data - driven culture across teams. Collaborating with IT and Finance on technology solutions and data governance practices.
Director, Data Platform Design and Strategy at MedImpact leading data platform and AI innovations to enhance healthcare services. Overseeing enterprise projects and managing teams to meet strategic goals.
Data Engineer delivering AI - and data - driven solutions for Honeywell’s industrial customers. Architecting and implementing scalable data pipelines and platforms focused on IoT and real - time data processing.
Data Engineering Associate focusing on data quality control and management for distribution platform. Collaborates on large scale data projects to ensure data accuracy and availability for users.
Data Architect managing enterprise data platform built on Microsoft Fabric at Johnstone Supply. Leading architectural standards and collaborating with business and IT leaders for strategic data - driven insights.
Data Engineer at Studyportals responsible for data pipelines and infrastructure. Join a team ensuring accurate and trustworthy data for analytics and business decisions.
AI/ML Engineer designing and refining prompts and workflows using large language models. Responsible for developing data pipelines and delivering scalable AI solutions in a hybrid work environment.
AWS Data Architect at Fractal designing and operationalizing AWS data solutions at enterprise scale. Collaborating with clients and mentoring engineers in best practices.
Senior Data Engineer driving data - driven success at Pacific Life. Collaborating with a team to build scalable and secure data solutions in Newport Beach, CA or Charlotte, NC.
Data Architect managing Commercial Data architecture initiatives for Valmet's sales and service team. Leading AI - driven data integrity and quality efforts in a global context.