Senior Lead Data Engineer managing ETL pipelines and architecture design for Novacore data platforms. Collaborating with cross-functional teams to ensure quality standards and business alignment.
Responsibilities
Monitor and manage multiple ETL pipelines supporting Novacore data platforms.
Provide end-to-end troubleshooting, debugging, and resolution of pipeline failures.
Perform root cause analysis for issues and implement permanent fixes.
Proactively communicate pipeline failures, resolutions, and status updates to internal stakeholders and clients.
Design and develop new ETL pipelines and data processing workflows as per business requirements.
Assist clients in defining data architecture and proposing optimal technical solutions.
Recommend and implement scalable, efficient, and cost-effective ETL and data architecture approaches.
Support enhancements and optimizations to existing pipelines and data models.
Manage code deployments to Production environments following established release processes.
Utilize Azure DevOps for source control, CI/CD pipelines, release management, and work item tracking.
Ensure deployment best practices, version control, and rollback strategies are followed.
Provide ongoing operational support and maintenance for data pipelines and related systems.
Create and maintain technical documentation, including pipeline workflows, architecture diagrams, and runbooks.
Collaborate with cross-functional teams (engineering, analytics, and business stakeholders) to ensure smooth delivery and alignment with business goals.
Adhere to agreed SLAs, timelines, and quality standards.
Requirements
Key Responsibilities:- ETL Pipeline Monitoring & Support o\tMonitor and manage multiple ETL pipelines supporting Novacore data platforms. o\tProvide end-to-end troubleshooting, debugging, and resolution of pipeline failures. o\tPerform root cause analysis for issues and implement permanent fixes. o\tProactively communicate pipeline failures, resolutions, and status updates to internal stakeholders and clients. 1.\tETL Development & Architecture Design o\tDesign and develop new ETL pipelines and data processing workflows as per business requirements. o\tAssist clients in defining data architecture and proposing optimal technical solutions. o\tRecommend and implement scalable, efficient, and cost-effective ETL and data architecture approaches. o\tSupport enhancements and optimizations to existing pipelines and data models. 2.\tProduction Deployment & DevOps Support o\tManage code deployments to Production environments following established release processes. o\tUtilize Azure DevOps for source control, CI/CD pipelines, release management, and work item tracking. o\tEnsure deployment best practices, version control, and rollback strategies are followed. 3.\tOperational Support, Documentation & Collaboration o\tProvide ongoing operational support and maintenance for data pipelines and related systems. o\tCreate and maintain technical documentation, including pipeline workflows, architecture diagrams, and runbooks. o\tCollaborate with cross-functional teams (engineering, analytics, and business stakeholders) to ensure smooth delivery and alignment with business goals. o\tAdhere to agreed SLAs, timelines, and quality standards.
Technology Consulting Senior Consultant role at Protiviti, mentoring interns while leading data engineering and analytics initiatives for clients. Collaborating across diverse industries to deliver meaningful insights.
Senior Data Architect designing scalable AI - ready data architectures for Novo Nordisk. Building data platforms, ensuring compliance standards, and collaborating with cross - functional teams.
Data Engineering Coordinator at AlmapBBDO overseeing data pipeline architecture and implementation. Leading team development and ensuring data quality across platforms in a hybrid environment.
AWS Data Engineer designing data models and supporting data architecture for various clients at EXL. Collaborating to deliver data solutions for improved business outcomes in a hybrid work environment.
Senior Data Engineer at Noda creating scalable data solutions for smarter, sustainable buildings. Collaborating with teams to optimize data for high - performance analytics.
Senior Consultant position at Protiviti mentoring teams on data analytics and engineering solutions using Microsoft technologies, enhancing efficiency and client relationship management.
Leading Technology Consulting as Associate Director at Protiviti, focusing on Microsoft Fabric and Databricks. Strengthening client relationships through analytics and mentoring teams in consulting engagements.
GCP Data Engineer specializing in data governance, architecture, and quality. Collaborates in a hybrid environment across multiple locations in Mexico.
Director of Data Engineering leading data architecture and analytics at Petfolk. Overseeing data infrastructure and managing a data team to drive AI and business intelligence solutions.