Senior Big Data Engineer utilizing cutting-edge technologies for real-time data processing. Collaborating in an agile environment to enhance a high-traffic data platform.
Responsibilities
Participate in all agile scrum meetings such as daily standups, refinement sessions, retro sessions, etc
Maintaining the data platform by means of daily reconciliations, data reloads, tackling support tickets, etc.
Being able to come up with new ideas to improve existing products and services.
Implementation of bug fixes and enhancements within the data platform.
Performing code reviews for other engineers.
Taking ownership of releases where necessary
Being able to communicate with the respective stakeholders, making sure that all information is transmitted correctly and to the right audience.
Take ownership of complex, large-scale initiatives, including data migration projects, ensuring successful planning, execution, and delivery.
Requirements
Object oriented programming (Java)
Data modelling using any database technologies
ETL processes (ETLs are oldschool, we transfer in memory now) and experience with Apache Spark or Apache NiFi
Applied understanding of CI\CD in change management
Dockerised applications
Used distributed version control systems
Excellent team player
Meticulous and passionate about your work
Functional oriented programming (experience with Python, Scala)
Analytical mindset
Low latency databases such as ClickHouse
Hold a Bachelor’s Degree in Computer Science or equivalent
Can collaborate in scoping exercises
Mentoring skills
Have worked with both Linux and Windows environments.
Data Engineer constructing and managing data warehouses for travel technology solutions at Onfly. Collaborating across teams to optimize data processing and integration solutions in a hybrid environment.
Data Engineer developing and maintaining efficient data pipelines for strategic client projects. Collaborating with teams to deliver valuable data insights and ensuring data quality and governance.
Data Engineer building and maintaining ETL pipelines for Northstrat's agile development team. Collaborating in a fast - paced environment with a focus on high - quality solutions.
Data Engineering Lead at BlackStone eIT managing a team and driving data solutions aligned with strategic goals. Collaborating with departments to ensure effective data management.
Data engineer designing custom data architectures at Zenika. Developing and maintaining data transformation pipelines, optimizing performance, and collaborating with cross - functional teams.
Senior Data Engineer responsible for implementing and optimizing industrial data infrastructure at AG Solution. Leading a team and collaborating with various stakeholders to ensure data quality and reporting.
Lead Data Engineer designing scalable data architecture on Google Cloud Platform for infas 360. Collaborating with cross - functional teams to develop tailored data products in Bonn.
Data Engineer developing data platforms for Octopus Electric Vehicles. Build and optimise data solutions to transform insights for sustainable transportation.
Senior Data Engineer developing and maintaining data pipelines and solutions for EMC's data analytics products. Collaborating with cross - functional teams to ensure high - quality data management and governance practices.