Data Engineer at PwC, engaging with Azure cloud services to enhance data handling and integrity. Responsibilities include pipeline optimizations, documentation, and collaboration with stakeholders.
About the role
- Data Engineer focusing on SQL database management and Azure Data Factory pipelines for a consulting firm in Spain. Collaborating on data integration and ensuring data accuracy in cloud environments.
Responsibilities
- SQL Database Management: Design, implement, and maintain SQL databases to ensure efficient data storage and retrieval.
- Optimize and tune SQL queries for maximum performance.
- Elasticsearch Integration: Work with Elasticsearch to index, search, and analyze large volumes of data efficiently.
- Collaborate with cross-functional teams to integrate Elasticsearch into our data ecosystem.
- Azure Data Factory Pipelines: Develop and manage data pipelines using Azure Data Factory.
- ETL Development and Maintenance: Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into our data warehouse.
- Implement data quality checks and ensure the integrity of data throughout the ETL process.
- Ensure the reliability, scalability, and efficiency of data movement within the Azure cloud environment.
- Work in migrating our data sources into Snowflake.
- Proactive Problem Solving: Proactively identify and address data-related issues, ensuring data accuracy and consistency.
- Collaborate with other teams to understand their data requirements and provide effective solutions.
- Clearly communicate complex technical concepts to non-technical stakeholders.
- Collaborate with data scientists, analysts, and other team members to understand data needs and deliver solutions.
- Documentation: Maintain thorough documentation for all data engineering processes, ensuring knowledge transfer and best practices.
Requirements
- Bachelors degree in Computer Science, Information Technology, or related field.
- Proven experience in SQL database design and optimization.
- Experience with github or gitlab.
- Experience with dbt.
- Strong ETL development skills.
- Experience with data modeling.
- Hands-on experience with Snowflake or Databricks.
- Proficiency in creating and managing data pipelines using Azure Data Factory.
- Excellent problem-solving and analytical skills.
- Proactive mindset with the ability to work independently and collaboratively.
- Strong communication and interpersonal skills.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer Manager at PwC focusing on building data infrastructure and solutions. Leading data engineering projects to transform raw data into actionable insights and drive business growth.
Junior Data Engineer at OneMarketData focusing on data quality and integrity in financial datasets. Collaborating with senior analysts and assisting in data management and analysis tasks.
Data Engineer Manager leading robust data solutions for PwC's clients. Transforming raw data into actionable insights and coaching team members.
Senior Data Engineering Analyst developing and implementing data solutions. Collaborating in a diverse environment focused on data processing and analysis for clients' digital transformation.
Principal Software Engineer in Threat Data Platform developing AI - driven tools for threat intelligence automation. Collaborating on robust data pipelines for PANW’s product ecosystem.
Senior Azure Data Engineer maintaining business intelligence solutions for Grupo Gloria, implementing and stabilizing projects in Azure and Databricks with Power BI reporting.
Staff Data Engineer at URBN developing AI - powered digital experiences by integrating algorithmic solutions with creative tools. Collaborating with cross - functional teams for impactful product evolution.
Senior Data Engineer at Anglian Water responsible for scalable data solutions and team collaboration. Leading design, build, and operation of secure data pipelines for critical services.
Data Engineer developing complex data pipelines for Symphony, a global software company. Collaborating with teams and driving data solutions in a hybrid work environment.