Principal Data Engineer designing and implementing data solutions that ensure trust and transparency in supply chains. Collaborating with global teams and mentoring fellow engineers in data practices.
About the role
- Senior Data Engineer leading development and optimization of data solutions. Managing complex projects, ensuring data quality, and mentoring junior engineers in data engineering.
Responsibilities
- Lead design and optimization of complex data solutions.
- Oversee data engineering projects.
- Mentor junior engineers.
- Ensure solutions align with organizational goals.
- Provide strategic guidance.
Requirements
- 10+ years in data engineering/ETL roles, with at least 6+ years in Azure cloud ETL leadership.
- Azure certifications (e.g., Azure Analytics Specialty, Solutions)
- Azure Data Sources: Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics.
- External Sources: APIs, on-prem databases, flat files (CSV, Parquet, JSON).
- Tools: Azure Data Factory (ADF) for orchestration, Databricks connectors.
- Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing.
- Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution.
- Performance Optimization: Partitioning, caching, broadcast joins.
- Delta Lake: Implementing ACID transactions, time travel, and schema enforcement.
- Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks.
- Azure Key Vault: Secure credential management.
- Azure Monitor & Logging: For ETL job monitoring and alerting.
- Networking & Security: VNET integration, private endpoints.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Analyst supporting data - driven decisions through data collection, cleaning, and analysis at an operations team in Hyderabad, India.
Senior Data Engineer role at Dun & Bradstreet focused on data analytics and visualization. Collaborating with teams to optimize data processes and deliver actionable insights.
Senior Data Engineer with AWS expertise leading financial data architecture and scalable solutions. Collaborating in wealth management to enhance data quality and systems.
Data Migration Specialist handling large - scale data migration from legacy to enterprise PLM platform. Analyzing data structures, developing strategies, and ensuring integrity across systems.
Director leading strategy, governance, and delivery of enterprise data platform at Phillips 66. Partnering with AI, Data Science, and business teams to enhance analytics and business systems.
Product Owner driving ERP data migration initiatives for BioNTech’s global landscape. Leading effective data management and ensuring compliance with regulatory standards in a fast - paced environment.
Data Engineer II leading development and delivery of data pipelines for Syneos Health. Collaborating with teams to optimize data processing and integrate solutions into production environments.
Lead Data Engineer overseeing data operations and analytics engineering teams for OneOncology. Focused on operational excellence in data platform and model reliability for cancer care improvement.
Senior AWS Software Data Engineer at Boeing focusing on AWS Data services to support digital analytics capabilities. Collaborating with cross - functional teams to design, develop, and maintain software data solutions.