Data Engineer II leading development and delivery of data pipelines for Syneos Health. Collaborating with teams to optimize data processing and integrate solutions into production environments.
About the role
- Senior Data Engineer leading the design and optimization of scalable data architectures on AWS. Collaborating on complex data pipelines and mentoring junior engineers.
Responsibilities
- Lead the design and development of scalable, high-performance data architectures on AWS, leveraging services such as S3, EMR, Glue, Redshift, Lambda, and Kinesis.
- Architect and manage Data Lakes for handling structured, semi-structured, and unstructured data.
- Design and build complex data pipelines using Apache Spark (Scala & PySpark), Kafka Streams (Java), and cloud-native technologies for batch and real-time data processing.
- Optimize these pipelines for high performance, scalability, and cost-effectiveness.
- Develop and optimize real-time data streaming applications using Kafka Streams in Java.
- Build reliable, low-latency streaming solutions to handle high-throughput data, ensuring smooth data flow from sources to sinks in real time.
- Manage Snowflake for cloud data warehousing, ensuring seamless data integration, optimization of queries, and advanced analytics.
- Implement Apache Iceberg in Data Lakes for managing large-scale datasets with ACID compliance, schema evolution, and versioning.
- Design and maintain highly scalable Data Lakes on AWS using S3, Glue, and Apache Iceberg.
- Work with business stakeholders to create actionable insights using Tableau.
- Build data models and dashboards that drive key business decisions, ensuring that data is easily accessible and interpretable.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent work experience).
- 5+ years of experience in Data Engineering or a related field, with a proven track record of designing, implementing, and maintaining large-scale distributed data systems.
- Proficiency in Apache Spark (Scala & PySpark) for distributed data processing and real-time analytics.
- Hands-on experience with Kafka Streams using Java for real-time data streaming applications.
- Strong experience in Data Lake architectures on AWS, using services like S3, Glue, EMR, and data management platforms like Apache Iceberg.
- Proficiency in Snowflake for cloud-based data warehousing, data modeling, and query optimization.
- Expertise in SQL for querying relational and NoSQL databases, and experience with database design and optimization.
Benefits
- Lead and mentor junior engineers, fostering a culture of collaboration, continuous learning, and technical excellence.
- Ensure high-quality code delivery, adherence to best practices, and optimal use of resources.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Lead Data Engineer overseeing data operations and analytics engineering teams for OneOncology. Focused on operational excellence in data platform and model reliability for cancer care improvement.
Senior AWS Software Data Engineer at Boeing focusing on AWS Data services to support digital analytics capabilities. Collaborating with cross - functional teams to design, develop, and maintain software data solutions.
Senior Data Engineer designing and improving software for business capabilities at Barclays. Collaborating with teams to build a data and intelligence platform for Equity Derivatives.
Senior AI & Data Engineer developing and implementing AI solutions in collaboration with clients and teams. Working on projects involving generative AI, predictive analytics, and data mastery.
Consultant driving IA business growth in Deloitte's Artificial Intelligence & Data team. Delivering innovative solutions using data analytics and automation technologies.
Data Engineer responsible for managing data architecture and pipelines at Snappi, a neobank. Collaborating with teams to enable data processing and analysis in innovative banking solutions.
Data Engineer at Destinus developing the data platform to support production and analytics needs. Involves migrating Excel sources to Lakehouse and integrating ERP systems in a hybrid role.
Senior Data Engineer developing solutions within the Global Specialty portfolio at an insurance company. Engaging with diverse business partners to ensure high quality data reporting.
Data Engineer at UBDS Group focusing on designing and optimizing modern data platforms. Collaborating in a multidisciplinary team to develop reliable data assets for analytics and operational use cases.