Senior Data Engineer developing solutions within the Global Specialty portfolio at an insurance company. Engaging with diverse business partners to ensure high quality data reporting.
About the role
- Data Engineer developing and maintaining the Data Lakehouse platform using Microsoft Azure technology stack at RBC. Collaborating with business and technology teams to enhance data ingestion and modeling processes.
Responsibilities
- Responsible for the development and ongoing maintenance of the Data Lakehouse platform infrastructure using the Microsoft Azure technology stack, including Databricks and Data Factory.
- Manage data pipelines consisting of a series of stages through which data flows (for example, from data sources or endpoints of acquisition to integration to consumption for specific use cases).
- Create new and modify existing Notebooks, Functions and Workflows to support efficient reporting and analytics to the business.
- Create, maintain, and develop Dev, UAT and Production environments ensuring consistency.
- Responsible for using innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks to minimize manual and error-prone processes and improve productivity.
- Champion for the DevOps process to ensure the latest techniques are being used and that implementation methodologies involving new or changes to existing source code or data structures follow the agreed development and release processes and that all productionised code is adequately documented, reviewed and unit tested where appropriate.
- Identify, source, stage, and model internal process improvements to automate manual processes and optimise data delivery for greater scalability, as part of the end-to-end data lifecycle.
- Be curious and knowledgeable about new data initiatives and how to address them. This includes applying their data and/or domain understanding in addressing new data requirements.
Requirements
- Proven experience working within Data Engineering and Data Management architectures like Data Warehouse, Data Lake, Data Hub and the supporting processes like Data Integration, Governance, Metadata Management.
- Proven experience working in cross-functional teams and collaborating with business stakeholders in support of a departmental and/or multi-departmental data management and analytics initiative.
- Strong experience with popular database programming languages for relational databases (SQL, T-SQL).
- Experience working on a cloud data platform such as Databricks or Snowflake.
- Adept in agile methodologies, and capable of applying DevOps and DataOps principles to data pipelines.
- Basic experience in working with data governance, data quality and data security teams.
- Good understanding of datasets, Data Lakehouses, modelling, database design and programming
- Knowledge of Data Lakehousing techniques, solutions and methodologies
- Strong experience supporting and working with cross-functional teams in a dynamic business environment.
- Required to be highly creative and collaborative working closely with business teams and IT teams to define the business problem, refine the requirements, and design and develop data deliverables accordingly.
Benefits
- Leaders who support your development through coaching and managing opportunities
- Opportunities to work with the best in the field
- Ability to make a difference and lasting impact
- Work in a dynamic, collaborative, progressive, and high-performing team
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer at UBDS Group focusing on designing and optimizing modern data platforms. Collaborating in a multidisciplinary team to develop reliable data assets for analytics and operational use cases.
Data Engineer (dbt) at SDG Group involved in all phases of data projects. Collaborate on data ingestion, transformation, and visualization in a hybrid environment.
Data Consultant at SDG Group specializing in Data & Analytics projects. Collaborate on technical - functional definitions, ETL, data modeling, and visualization for cloud solutions.
Senior Data Engineer responsible for growing customer - defined targeting calculations and developing key/value databases for real - time data processing.
Data Engineer focused on creating a data platform for automated cyber insurance. Collaborating with stakeholders to deliver data processing capabilities and governance.
Data Engineer designing and developing data solutions using AI and machine learning for marketing applications. Collaborating in teams to create impactful data - driven solutions for clients across various industries.
Data Engineer building and maintaining data platform solutions for clients at Dignify. Designing, developing, and optimising data models and pipelines with a focus on Google BigQuery.
Senior Data Engineer developing scalable data solutions for electric vehicle market at Kempower. Collaborating with cross - functional teams to enhance data engineering processes.
Data Engineer responsible for developing research analytic data infrastructure at Sutter Health. Involves managing data quality, pipelines, and compliance with healthcare regulations.