Senior Data Engineer at Porto Bank responsible for developing robust data architecture and ensuring data quality across teams. Collaborating with various data solutions for effective data management.
Responsibilities
Develop a robust, scalable Data Architecture that supports data teams (Analytics, BI, Data Science, Machine Learning and other verticals) and the business, democratizing access to data in a secure, simple, structured and high-quality manner;
Design, build and integrate monitoring mechanisms and observability for the data ecosystem;
Design/build development standards and best practices, and document the environment and deliverables;
Document, monitor and ensure data quality;
Design and ensure the application of best practices for pipeline development, focusing on performance and scalability;
Ensure and maintain a vision capable of relating/integrating areas (Data Products and MLOps) and data solutions;
Ensure knowledge is shared across the team;
Identify, manage and address risks related to the activities under your and your team's responsibility;
Ensure and implement the data architecture design;
Ensure correct understanding of business needs and strategic vision;
Ensure that all identified requirements to be developed are delivered with quality, scalability and within the deadline;
Requirements
Degree in Engineering, Computer Science or related fields;
Experience with Azure and/or AWS;
Experience with Databricks;
Experience with Delta Lake / Lakehouse;
Experience with Cosmos DB, Azure Synapse and Analysis Services;
Experience with programming languages Python, SQL and PySpark;
Experience with distributed data processing;
Experience with continuous integration and continuous delivery pipelines (CI/CD) and/or Azure DevOps;
Senior Data Engineer at Keyrus leading the design, development, and delivery of scalable data platforms. Collaborating with teams to translate requirements into production - grade solutions and mentoring engineers.
Senior Data Engineer for global payments platform designing ETL pipelines and data models. Collaborating across teams to tackle complex data challenges in an innovative fintech environment.
Data Warehouse Modelling Engineer designing and maintaining data models using Data Vault 2.0 for iGaming industry. Collaborating with stakeholders and optimizing data models in a hybrid work environment.
Senior Data Engineer driving impactful data solutions for the climate logistics startup HIVED's core data platform. Collaborating with cross - functional squads to enhance analytics and delivery.
Data Engineer developing and maintaining CRE forecasting infrastructure for Cushman & Wakefield. Collaborates with senior economists and technical teams to ensure high - quality data solutions.
Data Engineer at PwC, engaging with Azure cloud services to enhance data handling and integrity. Responsibilities include pipeline optimizations, documentation, and collaboration with stakeholders.