Lead Data Engineer providing technical expertise and mentoring at Westpac. Ensuring platform stability and compliance while driving engineering practices within the squad.
About the role
- Azure Data Engineer designing and implementing data pipelines for on-prem to Azure cloud migration. Building ingestion and transformation pipelines using Azure tools.
Responsibilities
- Design and implement end-to-end data pipelines for on-prem to Azure cloud migration
- Build metadata-driven ingestion and transformation pipelines using Azure Data Factory
- Develop and optimize data processing using Azure Databricks, Delta Lake, Spark SQL, PySpark, and Scala
- Create and optimize Azure Synapse stored procedures and analytical models for reporting
- Enable data consumption for Power BI dashboards
- Manage CI/CD deployments using Azure DevOps
- Lead requirement gathering, data modeling, sprint planning, and timely project delivery
Requirements
- 5+ Years of experience
- Design and implement end-to-end data pipelines for on-prem to Azure cloud migration
- Build metadata-driven ingestion and transformation pipelines using Azure Data Factory
- Develop and optimize data processing using Azure Databricks, Delta Lake, Spark SQL, PySpark, and Scala
- Create and optimize Azure Synapse stored procedures and analytical models for reporting
- Enable data consumption for Power BI dashboards
- Manage CI/CD deployments using Azure DevOps
- Lead requirement gathering, data modeling, sprint planning, and timely project delivery
Benefits
- Startup culture with opportunity to grow
- Performance bonuses
- Company stock options
- Health, dental and vision insurance plans
- Pursue growth and learning with career development expenses
- Volunteer and support education for kids & skills development for adults
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer at Uline delivering custom data warehousing solutions for shipping materials distribution. Responsibilities include ETL workflows and data integration management.
MDM Data Engineer managing Profisee MDM platform for Pacific Life. Responsible for data quality, integration and technical support across enterprise systems.
Data Engineer responsible for designing and developing cloud - based data pipelines for ING. Collaborating with international teams focusing on financial risk analytics in the cloud environment.
Data Engineer specializing in SQL and datamigration processes for the municipality of Amsterdam. Collaborating with diverse stakeholders to ensure data integrity and accessibility.
Senior Expert in Data Architecture at OMV Group based in Romania. Leading data governance strategy and managing complex data management responsibilities.
Senior BI and Data Engineer developing innovative BI solutions in the Tech for Good space. Collaborating to enhance STEM education through effective data practices.
Senior Data Engineer at Retail Insight shaping data platforms and mentoring junior engineers. Leading data services design and collaborating with cross - functional teams for client success.
Data Engineer creating scalable, reliable data solutions for Retail Insight's clients and enhancing the data ecosystem. Collaborating with software engineers to ensure data quality and accessibility.
Director of Business Systems and Data Architecture at Ceribell, transforming tech solutions for critical care. Leading enterprise architecture strategy to support business objectives.