Data Architect leading design and implementation of cloud data platforms for digital transformation. Collaborating with stakeholders to define data strategies and governance models.
About the role
- Senior Data Engineer at SS&C building and optimizing data pipelines in a lakehouse environment. Collaborating with data architects and stakeholders in the financial services sector.
Responsibilities
- Implement and maintain end-to-end data pipelines for data acquisition from diverse sources, including databases, APIs, files, and messaging systems such as Kafka.
- Build robust data validation, enrichment, and transformation workflows using Python and pySpark.
- Develop and optimize data storage and querying layers using technologies such as Apache Iceberg, Trino, StarRocks, and Snowflake.
- Implement and maintain dimensional data models, including Star and Snowflake schemas, as defined by data architecture standards.
- Integrate and manage streaming data flows using Kafka for both ingestion and real-time data distribution.
- Design and implement data quality checks, monitoring, and alerting to ensure high data reliability.
- Contribute to metadata management, data governance, and security practices, including access controls and data masking.
- Enable data distribution and consumption through files, APIs, Kafka, Snowflake data sharing, and analytics tools.
- Optimize pipeline performance, cost, and scalability while troubleshooting and resolving production issues.
- Collaborate closely with data architects, analysts, data scientists, and stakeholders to deliver high-quality data products.
- Mentor junior engineers and promote best practices in code quality, testing, and CI/CD for data pipelines.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- 5+ years of hands-on experience in data engineering roles, including at least 2 years working with big data or lakehouse platforms.
- Strong proficiency in Python and pySpark for building scalable data processing pipelines.
- Hands-on experience with analytical and query platforms such as Trino, StarRocks, and Snowflake.
- Experience working with open table formats, particularly Apache Iceberg.
- Proven experience with streaming technologies, especially Apache Kafka.
- Solid understanding of dimensional modeling and data warehousing concepts.
- Familiarity with data quality frameworks, metadata management, governance tools, and security best practices.
- Experience with cloud platforms such as AWS, Azure, or GCP, and infrastructure-as-code tools.
- Strong problem-solving skills with experience debugging and tuning complex data pipelines.
- Excellent communication and collaboration skills.
Benefits
- Health insurance
- Retirement plans
- Paid time off
- Flexible work arrangements
- Professional development
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer Consultant designing and optimizing data infrastructure for clients' business needs. Working with SQL and data visualization tools in a mainly remote role with some onsite responsibilities in Denver.
Data Engineer creating Real - Time Data Processing applications for a leading iGaming operator. Work involves stream data manipulation and collaboration in an Agile environment.
Data Engineer at Voodoo optimizing real - time data pipelines for gaming and consumer apps to support growth. Joining a top - tier data team dedicated to monetizing via advertising partners in a competitive landscape.
Cloud Data Engineer designing data architectures for cloud platforms at fifty - five. Collaborating with local and global teams to optimize marketing ROI and customer experience.
SAP Specialist responsible for designing, developing, and executing data migration objects in Hydro’s SAPEX program. Ensuring successful ETL processes and maintaining data quality.
Senior Data Engineer building scalable data pipelines and data models within retail at Avaron. Collaborating closely with business and technical teams to ensure reliable data solutions.
Senior Data Engineer building and operating the data platform at bsport. Collaborating with the Data team to optimize data intake and accessibility for analytics and AI.
Data Engineer building and maintaining Azure data platforms for Hultafors Group's analytics and reporting needs. Collaborating across various business functions in a cloud environment.
Lead Data Pipeline Manager at Valpak, overseeing data pipelines for environmental compliance initiatives. Collaborate with teams to ensure data quality and operational performance.