SAP Specialist responsible for designing, developing, and executing data migration objects in Hydro’s SAPEX program. Ensuring successful ETL processes and maintaining data quality.
About the role
- Senior Data Engineer focusing on ETL developments in Azure Databricks at OAG, a travel data platform. Collaborating with multi-skilled teams to modernize Flight Status data processing.
Responsibilities
- Design, build, and optimize scalable ETL and Structured Streaming pipelines in Azure Databricks for real-time and batch ingestion of Flight Status data
- Design and implement data ingestion and processing pipelines that consolidate heterogeneous data sources, including APIs, event streams, and file-based feeds, into the OAG lakehouse (Azure Databricks + Delta Lake), ensuring data consistency, reliability, and scalability
- Implement and monitor data quality using automated validation, alerting, and observability practices
- Develop and maintain orchestration workflows in Apache Airflow, coordinating ingestion and transformation processes across multiple data flows
- Build reusable frameworks for schema evolution, error handling, deduplication, and auditing
- Collaborate with data platform, analytics, and product teams to define SLAs, data contracts, and performance targets
- Optimize Spark and Delta Lake performance for scalability, latency, and cost efficiency
- Implement CI/CD pipelines and automation for data workflows using Azure DevOps or equivalent tools
- Mentor engineers, review code, contribute to platform design discussions and planning, and help grow data engineering competencies in the team and across OAG
Requirements
- Proven track record in data engineering, with a strong focus on ETL development and streaming data architectures
- Experience with Azure Databricks, Apache Spark (Structured Streaming), and Delta Lake
- Proficiency in Python (PySpark) and SQL, with experience transforming large-scale, complex datasets
- Hands-on experience in data orchestration and workflow automation (e.g., Apache Airflow or similar)
- Experience working in a cloud data environment (preferably Azure) across storage, compute, and pipeline services
- Familiarity with streaming or messaging technologies (e.g., Kafka, Event Hubs)
- Strong understanding of data quality, validation, and observability practices
- Ability to deliver production-grade solutions with a results-oriented and ownership-driven mindset
- Experience implementing CI/CD and version-control practices using Azure DevOps, GitHub Actions, or similar tools
- Excellent analytical, communication, and collaboration skills
- Strong understanding of modern data engineering patterns and ability to design scalable, modular, and reliable data systems
Benefits
- Company-provided free lunch every day
- Private health insurance
- Company bonus scheme
- Voluntary participation in a company-supported retirement scheme
- A generous annual leave policy, growing with each year of service, and a day off during your birthday month
- Participation in team-building activities, team workshops, and group learning sessions
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Data Engineer building scalable data pipelines and data models within retail at Avaron. Collaborating closely with business and technical teams to ensure reliable data solutions.
Senior Data Engineer building and operating the data platform at bsport. Collaborating with the Data team to optimize data intake and accessibility for analytics and AI.
Data Engineer building and maintaining Azure data platforms for Hultafors Group's analytics and reporting needs. Collaborating across various business functions in a cloud environment.
Lead Data Pipeline Manager at Valpak, overseeing data pipelines for environmental compliance initiatives. Collaborate with teams to ensure data quality and operational performance.
Data Engineer role responsible for building scalable data pipelines and systems at Consort Group in Portugal. Involves data engineering and regulatory reporting across diverse technical environments.
Senior Data Engineer with strong ETL experience using IBM DataStage for data transformations. Collaborating on data quality and design improvements in a hybrid environment.
Data Engineer supporting data capabilities of the fastest - growing family law firm in the UK. Drive data initiatives and collaborate with the BI team in a hybrid working environment.
Data Architect designing AWS - based data solutions in hybrid role with a major financial sector client in LATAM. Involves leading data architecture decisions and optimizing data processing solutions.
Senior Data Engineer developing data solutions for a key client in the financial sector. Collaborating in a hybrid setting located in Quito, Ecuador.