Senior Manager leading a team of database engineers to manage CCC's data platform. Overseeing mission - critical applications and collaborating with cross - functional teams in a hybrid environment.
About the role
- Senior Technical Lead/Architect focusing on designing and implementing AWS data ingestion pipelines. Collaborating with teams to enhance operational SLAs and data quality workflows.
Responsibilities
- Provide technical solution discovery effort on any new capabilities or new functionality.
- Assist PO with technical user stories to ensure healthy backlog features
- Lead the development of real-time data pipelines using AWS DMS, MSK, Kafka or Glue Streaming and for CDC ingestion from multiple SQL Server sources (RDS/on-prem).
- Build and optimize streaming and batch data pipelines using AWS Glue (PySpark) to validate, transform, and normalize data to Iceberg and DynamoDB.
- Define and enforce data quality, lineage, and reconciliation logic with support for both streaming and batch use cases.
- Integrate with S3 Bronze/Silver layers and implement efficient schema evolution and partitioning strategies using Iceberg.
- Collaborate with architects, analysts, and downstream application teams to design API and file-based egress layers.
- Implement monitoring, logging, and event-based alerting using CloudWatch, SNS, and EventBridge.
- Mentor junior developers and enforce best practices for modular, secure, and scalable data pipeline development.
Requirements
- 6+ years of hands-on expert level data engineering experience in cloud-based environments (AWS preferred) with event driven implementation
- Strong experience with Apache Kafka / AWS MSK including topic design, partitioning, and Kafka Connect/Debezium
- Proficiency in AWS Glue (PySpark) and for both batch and streaming ETL
- Working knowledge of AWS DMS, S3, Lake Formation, DynamoDB, and Iceberg
- Solid grasp of schema evolution, CDC patterns, and data reconciliation frameworks
- Experience with infrastructure-as-code (CDK/Terraform) and DevOps practices (CI/CD,Git
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
As a Principal Data Architect at Solstice, lead the design and implementation of data architecture solutions. Ensure data integrity, security, and accessibility to meet strategic organizational goals.
Data Platform Specialist overseeing data workflows and enhancing data quality for Stackgini's AI - driven IT solutions. Collaborating with teams to drive improvements and stakeholder support.
Data Engineer designing data pipelines in Python for a major railway industry client. Collaborate with Data Scientists and ensure code quality with agile methodologies.
Senior Data Engineer responsible for building and optimizing data pipelines for banking analytics initiatives. Collaborating with data teams to ensure data quality and readiness for enterprise use.
Senior Data Engineer developing scalable data solutions on Databricks for analytics and operational workloads. Collaborating with cross - functional teams to modernize the data ecosystem.
Data Engineer focused on analytics and data pipeline development for network optimisation. Collaborating with teams to deliver high - quality data solutions with Python and SQL.
Senior Product Manager defining platform capabilities for Data Cloud in Salesforce. Collaborating with R&D teams while shaping product strategy for Data 360 integration.
Senior Data Engineer at Goodwin enhancing data platforms and fostering data - driven culture across teams. Collaborating with IT and Finance on technology solutions and data governance practices.
Director, Data Platform Design and Strategy at MedImpact leading data platform and AI innovations to enhance healthcare services. Overseeing enterprise projects and managing teams to meet strategic goals.