Hybrid Senior Data Engineer

at Kpler

• Job title: Senior Data Engineer
• Job type: Full Time
• Experience level: Senior
• Salary: Not specified
• Degree requirement: Bachelor's Degree
• Tech skills: ElasticSearch, Kafka, Kubernetes, Microservices, Spark
• Location requirements: Hybrid, London, United Kingdom

About the role

• Senior Data Engineer responsible for data pipelines and backend systems at Kpler. Working with a diverse team to deliver insights on global refinery operations and economics.

Responsibilities

• Design, build, and evolve batch and streaming data pipelines that power refinery modeling, analytics, and customer-facing products.
• Own complex data ingestion, transformation, validation, and delivery workflows across multiple data sources.
• Drive improvements in pipeline reliability, scalability, and observability, including retries, backfills, data quality checks, and monitoring.
• Lead schema design, versioning, and evolution strategies to support stable, long-lived data contracts.
• Build and maintain backend components and APIs used to serve data to downstream systems and applications.
• Partner closely with data scientists, product managers, and other engineers to translate domain requirements into robust technical solutions.
• Continuously improve existing systems as data volume, complexity, and product expectations grow.

Requirements

• Deliver high-quality, well-tested, and maintainable code, setting a strong example for engineering best practices.
• Own significant parts of the data platform end-to-end, from ingestion to production delivery.
• Make architectural contributions to data processing, storage, and delivery patterns.
• Contribute to and improve CI/CD pipelines, automation, and operational tooling.
• Instrument services and pipelines with metrics, logs, and alerts, and help define operational standards.
• Play an active role in incident response, root-cause analysis, and long-term system improvements.
• Review code, mentor other engineers, and help reinforce shared coding and architectural standards across the team.
• Exposure to Kafka, Spark, or streaming architectures.
• Experience with Kubernetes.
• Familiarity with event-driven or microservices architectures.
• Exposure to analytical datastores (e.g. Elasticsearch).
• Full-stack awareness (e.g. ability to read, review, and provide feedback on frontend or API-layer pull requests, without being a primary frontend contributor).
• Prior experience working on data products in energy, commodities, or industrial domains.