Design, develop and maintain scalable, reliable data pipelines and APIs in cloud environments.
Collaborate with Data Science and Engineering teams to ensure data quality, governance and cost-effective performance.
Responsibilities
Work on the development, maintenance and evolution of scalable, reliable data solutions
Design, develop and maintain efficient, scalable and reliable data pipelines
Automate extraction, transformation and load (ETL/ELT) routines in cloud environments
Implement solutions for integrating data from multiple sources, including structured and unstructured data
Develop and maintain data APIs and services for consumption by internal and external applications
Ensure data quality, governance, security and integrity at all stages of the data flow
Work on relational and non-relational data modeling to support analytical and operational use cases
Document architectures, pipelines, processes and data models clearly, consistently and in a reusable manner
Collaborate with Data Science, Software Engineering teams and business stakeholders
Support continuous improvement of data processes, performance and cloud cost optimization
Requirements
Proficiency in SQL and database management
Experience with Python for data engineering and process automation
Knowledge of REST APIs, JSON formats and system integration
Version control using Git
Knowledge of data architecture and modeling (relational and non-relational)
Experience with Big Data tools (Spark, Hadoop or similar)
Experience with Cloud Computing, preferably AWS (Azure or GCP as alternatives)
Familiarity with software engineering best practices, including testing, versioning and CI/CD
Technical English for reading documentation and participating in technical discussions
Intern Data Engineer at Flutter Studios learning to create applications in the gaming industry. Join a cross - functional team to implement a basic application in an Agile environment.
AI/Data Engineer at Comcast developing data pipelines and AI solutions for audit processes. Leading team efforts to ensure data quality and compliance with audit objectives across business units.
Senior Data Engineer involved in AWS Cloud and Big Data solutions for Financial Crime. Join CommBank's team to tackle complex data - centric problems with advanced technologies.
Senior Data Engineer leading and mentoring a team in building scalable data pipelines for digital transformation projects in an international software company.
Senior Data Operations Engineer at iKnowHow designing and implementing scalable data - driven applications. Focus on data pipelines, APIs, and collaboration across teams for project success.
Data/AI Engineer supporting innovations in corporate travel with AWS technologies at HRS Group. Collaborating with data teams to develop AI solutions and maintain data pipelines.
Data Engineer developing data solutions for reporting and analytics at Assurant. Implementing and optimizing Data Warehouse solutions in cloud and on - premise environments with Agile methodology.
AI Engineer developing AI - driven analytics for trading at Deloitte. Focusing on scalable data pipelines and collaboration with traders for actionable insights.
Data Engineer involved in delivering scalable enterprise systems and digital transformation platforms for Qualysoft. Collaborate with global teams across various industries and technologies.