Hybrid Data Engineering Specialist I

at Inmetrics

• Job title: Data Engineering Specialist I
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: Not specified
• Degree requirement: No Education Requirement
• Tech skills: Apache, Azure, Hadoop, HBase, Java, Jenkins, PySpark, Python, Spark
• Location requirements: Hybrid, Sao Paulo, Brazil

About the role

• Specialist in Data Engineering leading pipeline optimization at Inmetrics. Collaborating in innovative data-driven projects within a hybrid work environment.

Responsibilities

• Lead the development and optimization of data pipelines, architectures, and platforms.
• Ensure high performance, governance, and availability of information across the company's ecosystem.
• Work in a collaborative, innovative, and results-oriented environment where data is central to the strategy and the driver of intelligent, sustainable decisions.
• Collaborate with the Card team's Data Engineering group to create data pipelines for ingestion and provisioning of card-domain data into Santander Brazil's corporate data lake.
• Master and promote institution-wide concepts, tools, and technologies related to Data Analysis.
• Technically structure and update the current Guidelines and Policies for the Data Analysis area.
• Participate in defining principles for control, management, and specification of various data methods and models.
• Assist in designing, structuring, and optimizing databases.

Requirements

• Databricks skills: Experience working with Apache Spark on Databricks, including building and optimizing data pipelines.
• Experience with PySpark, Python, and Kedro: Strong programming skills in PySpark, Python, and Kedro to develop, debug, and maintain data transformation code.
• Batch and streaming data processing: Knowledge of batch and streaming (messaging) data processing, with the ability to design, implement, and maintain data processing pipelines.
• DevOps knowledge: Familiarity with using Jenkins for continuous integration and delivery (CI/CD), as well as automation of deployment tasks and pipeline management.
• Git: Proficiency in Git for source code version control and effective collaboration in development teams.
• Agile methods: Understanding of principles and practices of agile methodologies, such as Kanban and Scrum, for effective collaboration and project management.
• Orchestration (e.g., Control-M or others): Knowledge of workflow orchestration tools, important for scheduling and controlling workflows.
• Microsoft Azure knowledge: Experience with key Microsoft Azure data services, including Azure Databricks, Azure Data Factory, and Azure Storage Accounts.
• Experience in on-premises environments (Cloudera): Previous experience with the Cloudera platform or other on-premises big data solutions, including Hadoop, HBase, and Hive, is desirable.
• Object-oriented development knowledge: Familiarity with Java is helpful (not required to code, but to interpret).
• Optional certifications: AZ-900 (Microsoft Azure Fundamentals) and DP-900 (Microsoft Azure Data Fundamentals) certifications are preferred and demonstrate solid knowledge of the Azure platform and data concepts.

Benefits

• Bradesco Health Plan (30% copay)
• Bradesco Dental Plan (no employee contribution)
• Life Insurance
• Wellhub (Gympass)
• Childcare assistance
• Assistance for children with special needs
• Payroll-deductible loan
• Private pension plan
• Pet benefit plan
• SESC membership
• Conexa telemedicine
• Expense allowance
• Meal and food vouchers
• Multi-benefit card
• Medical plan upgrade
• Extended maternity and paternity leave
• Support program for pregnant employees
• Newborn gift basket and the book "Acontecia quando eu nascia"
• Professional development: courses available through the internal university
• 100% remote or hybrid work, depending on project requirements.