Senior Data Engineer responsible for building and optimizing data pipelines for banking analytics initiatives. Collaborating with data teams to ensure data quality and readiness for enterprise use.
About the role
- Principal Data Pipeline Lead at SS&C overseeing development of scalable data pipelines. Leading a small team and providing technical guidance for modern data platform integration.
Responsibilities
- Lead the development of batch and real-time data pipelines on top of a modern data platform
- Design and build scalable ingestion and transformation pipelines
- Mentor a small team of engineers
- Collaborate with platform engineering team to build pipelines
- Implement CDC pipelines using Debezium and Kafka
- Build streaming pipelines using Kafka and Apache Flink
- Develop transformation workflows using Python, Spark / PySpark, and Airflow
- Ingest data from DB2 replication streams
- Process legacy fixed-width and CSV data feeds
- Integrate API-based data sources
- Store and manage data using Apache Iceberg and Parquet
- Enable analytics through Trino and StarRocks
Requirements
- 8+ years building data platforms or large-scale data pipelines
- Strong programming experience in Python
- Experience with Spark / PySpark
- Experience building pipelines with Apache Airflow
- Experience with Kafka-based streaming architectures
- Experience implementing CDC pipelines (Debezium or similar)
- Experience with Apache Flink or other streaming frameworks
- Experience with Parquet and modern table formats such as Apache Iceberg
- Experience with distributed query engines such as Trino, Presto, or StarRocks
- Experience integrating data from heterogeneous or legacy systems
- Experience leading or mentoring engineers
Benefits
- Competitive salary
- Opportunities for increased leadership scope as the team expands
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Senior Data Engineer developing scalable data solutions on Databricks for analytics and operational workloads. Collaborating with cross - functional teams to modernize the data ecosystem.
Data Engineer focused on analytics and data pipeline development for network optimisation. Collaborating with teams to deliver high - quality data solutions with Python and SQL.
Senior Product Manager defining platform capabilities for Data Cloud in Salesforce. Collaborating with R&D teams while shaping product strategy for Data 360 integration.
Senior Data Engineer at Goodwin enhancing data platforms and fostering data - driven culture across teams. Collaborating with IT and Finance on technology solutions and data governance practices.
Director, Data Platform Design and Strategy at MedImpact leading data platform and AI innovations to enhance healthcare services. Overseeing enterprise projects and managing teams to meet strategic goals.
Data Engineer delivering AI - and data - driven solutions for Honeywell’s industrial customers. Architecting and implementing scalable data pipelines and platforms focused on IoT and real - time data processing.
Data Engineering Associate focusing on data quality control and management for distribution platform. Collaborates on large scale data projects to ensure data accuracy and availability for users.
Data Architect managing enterprise data platform built on Microsoft Fabric at Johnstone Supply. Leading architectural standards and collaborating with business and IT leaders for strategic data - driven insights.
Data Engineer at Studyportals responsible for data pipelines and infrastructure. Join a team ensuring accurate and trustworthy data for analytics and business decisions.