Senior Data Engineer optimizing scalable data pipelines at Matrix for the Brazilian energy market. Involved in ETL processes and data architecture design with a collaborative team environment.
Responsibilities
Develop and optimize high-performance, scalable batch data pipelines for ingesting structured, semi-structured, and unstructured data into our Data Lake (GCS);
Develop and manage ETL/ELT jobs for data processing using Spark/SQL to populate the Trusted (Silver) and Refined (Gold) layers of our Data Warehouse (BigQuery);
Consolidate and model data from internal and external sources to enable Analytics, Business Intelligence, Data Science, GenAI initiatives, and Data Products;
Develop online data integrations (streaming) for data ingestion and processing on GCP;
Design and implement efficient data architectures, optimizing development time, performance, and compute cost;
Reproduce business rules within data transformation processes to generate datasets;
Create and maintain Data Quality, Data Assurance, Lineage, and Data Versioning processes;
Serve as the Technical Reference for Matrix Energia's Data & Analytics team, supporting the most innovative and ambitious initiatives in our day-to-day work.
Requirements
Bachelor's degree in Computer Science, Systems, Technology, Data, or a related field;
Strong experience with Python and Spark/SQL;
Experience with CI/CD, preferably using GitHub;
Experience with Google Cloud Platform (GCP), Google Cloud Storage (GCS), and Google BigQuery;
Experience with data orchestration tools such as Airflow or Prefect;
Experience with data streaming technologies such as Pub/Sub or Azure Functions;
Knowledge of Data Modeling using the Medallion methodology.
Benefits
High-quality health coverage with Omint: access to a top-tier medical and dental network with nationwide coverage.
Alelo Pod card for flexible meal and grocery expenses, accepted at major restaurant chains and supermarkets.
Flexible and sustainable working hours.
Mobility: choose the option that works best for you — transit voucher, fuel allowance, or parking.
Total Pass: an online platform dedicated to promoting quality of life, health, and well-being. Plans provide access to gyms, studios, and online services with exclusive Matrix discounts, covering you and up to 3 dependents.
Group life insurance.
Profit Sharing (PLR), subject to target achievement and performance.
Work with a fantastic team in a collaborative environment focused on building a brighter, more sustainable future.
Data Engineer providing technical expertise on mission - critical NAVSUP OIS program. Work involves data architecture and database management in AWS GovCloud environments.
Senior Data Engineer focusing on data infrastructure for an AI - driven insurtech startup based in Nepal. Collaborating with teams to optimize data models and maintain data quality.
Senior Professional Consultant leading architecture and design for SAP BW and SAC solutions at Freudenberg. Collaborating with stakeholders and optimizing performance of data landscapes.
Senior Data Engineer designing and managing data architectures to transform large - scale data into insights for Humana. Involves leading technical discussions and implementing best data practices.
Data Engineer II at Early Warning Services developing data science tools and infrastructure. Collaborating on software enhancements and mentoring interns in a hybrid work environment.
Senior Data Architect responsible for optimizing data architecture and supporting data - driven business decisions at TruStage. Leading technical guidance for data architecture and cross - functional team collaboration.
Senior Data Architect developing data architecture plans at The Hartford, collaborating with internal teams to align data standards and practices. Leading complex solutions with a focus on operational effectiveness.
Senior Solution Architect defining architecture framework for SA‑CCR in regulatory risk. Collaborating with stakeholders to ensure compliance and efficient data governance.
Senior Data Engineer optimizing and designing data pipelines on AWS for The Rec Hub. Collaborating with the team to enhance data processing and mentorship.