Data Engineer II at Dun & Bradstreet collaborating with teams to enhance data quality standards and practices. Driving best - in - class data management across diverse disciplines.
About the role
- Senior Data Engineer building data pipelines using emerging technologies for Wavicle Data Solutions. Joining a flexible, digitally connected team focused on cloud, data, and AI solutions.
Responsibilities
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of sources like Hadoop, Spark, AWS Lambda, etc.
- Experience with AWS Cloud on Data Integration with Apache Spark, EMR, Glue, Kafka, Kinesis and Lambda in S3, Redshift, RDS, and MongoDB/DynamoDB ecosystems.
- Strong real-life experience in Python development, especially in PySpark in AWS Cloud environment.
- Design, develop, test, deploy, maintain and improve data integration pipeline.
- Develop pipeline objects using Apache Spark / Pyspark / Python or Scala.
- Design and develop data pipeline architectures using Hadoop, Spark and related AWS Services.
- Load and performance test data pipelines built using the above-mentioned technologies.
Requirements
- Bachelor or Master's degree in Computer Science, or related field is required.
- 8+ years of hands-on professional work experience with AWS and Python programming, experience with Python frameworks is required
- Hands-on expertise with cloud platforms including AWS and GCP
- Expert level knowledge of using SQL to write complex, highly-optimized queries across large volumes of data.
- Working experience on ETL pipeline implementation using AWS services such as Glue, Lambda, EMR, Shell, S3, SNS, Pyspark, etc. is required.
- Strong knowledge of data warehousing solutions, particularly Amazon Redshift
- Hands-on professional work experience using emerging technologies (Snowflake, Talend, and/or Databricks) is highly desirable.
- Proficiency in DBT for data transformation and modeling
- Experience with automation of data workflows and processes
- Strong problem solving and troubleshooting skills with the ability to exercise mature judgement.
Benefits
- Competitive compensation and bonuses
- Unlimited paid time off
- Health, retirement, and life insurance plans
- Long-term incentive programs
- Meaningful work that blends innovation and purpose
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Data Engineer at NVIDIA developing and optimizing database architectures while collaborating across software and hardware teams. Focus on data - driven systems in data center environments for complex networking verification.
Senior Data Engineer building reliable data infrastructure for AI - powered health experiences. Collaborating on data pipelines and ensuring data quality in a hybrid work environment.
Manager overseeing HR technology and data architecture operations enabling HR strategy effectiveness at Healthfirst. Leading a team of analysts for data integrity and operational support across systems.
Distinguished Data Engineer in data pipelines driving innovation in AWS. A hands - on technical leader delivering next - gen data solutions across Capital One's lines of business.
Product Manager for Capital One enhancing financial experiences and optimizing product development with technology. Collaborating with cross - functional teams and leading initiatives for the Core Data Platform Services.
Sr. Lead Data Engineer at Capital One responsible for delivering data solutions and mentoring teams. Engage in collaborative Agile practices and drive powerful experiences for financial empowerment.
Senior Data Engineer at EDO building data pipelines for advertising effectiveness measurement. Collaborating with Data Scientists using AWS, Airflow, DBT, and Snowflake on large data sets.
Data Engineer transforming data into actionable insights at InnoWave. Designing data pipelines and collaborating between technical teams and business stakeholders in Lisbon.
Data Engineer developing and maintaining infrastructure for analytics at Waitwhile. Responsible for data lifecycle management, API availability, and AI operationalization.