Data Engineer Consultant building and maintaining enriched data infrastructure for analytical thinking at Northwest Permanente. Involves data collection, cleansing, and transformation for business intelligence.
About the role
- Senior Data Engineer developing and maintaining data pipelines and solutions for EMC's data analytics products. Collaborating with cross-functional teams to ensure high-quality data management and governance practices.
Responsibilities
- Develops, test, and maintains prioritized engineering (design/build/test) work as a member of a data analytics product pipeline architecture and infrastructure team
- Ensures delivery of solutions for organization of data, such as transformation, conformation, integration, and aggregation of data
- Partners with the Data Platform Operations (DPOs), Information Security, Network, and Operations teams to design and implement new data pipelines for new business initiatives as related to data and analytics
- Serves as a technical expert for the Data Engineering team
- Ensures best practices and internal processes are followed when solutions are designed and implemented
- Ensures the delivery of solutions for acquisition of data, such as landing, staging, rapid ingestion, and historical data extraction
- Creates processes framework, design and implementation of large-scale data conversion and data ingestion on various cloud and on-premise platforms using most optimal processing and programming techniques
- Investigates and analyzes feasibility of data integration prototypes and program requirements
- Develops specifications and plans the full range of programming actions to produce data integration components
- Develops and maintains highly complex automated ETL/ELT pipeline architecture that cleans, transforms, and aggregates data from disparate sources using architecturally approved tools, programming languages, and services to deliver a data mesh approach
- Develops and maintains highly complex automated ETL/ELT monitoring and alarming solutions using architecturally approved languages and services
- Monitors, troubleshoots, and optimizes data pipelines for quality, performance, and reliability creating a single source of truth providing business customers with highly reliable and quality data
- Builds highly complex unit tests to enable test driven development and automation to drive code reliability
- Develops, maintains, tests, and deploys code with architecturally approved toolsets using Dev Ops practices, ETL/ELT pipelines, data catalogs and data queries
- Serves as a technical resource and supports team members on EMC’s data integration projects and helps them learn new technologies
- Leads evaluation of new technologies including performing POCs and presenting results to others, with a goal of providing technical recommendations
- Balances and designs data analytics product solutions based on cost and performance to meet business requirements
- Guides business customers to develop, troubleshoot, and optimize complex SQL and ETL/ELT solutions to solve reporting, metrics, and analytics problems.
Requirements
- Bachelor’s degree, preferably in a computer related field or equivalent relevant experience
- Eight years of data integration development experience, or related experience
- Excellent SQL Knowledge and experience working with relational databases
- Excellent analytics skills to work with unstructured datasets preferred
- Excellent knowledge of applicable programming languages such as Python
- Strong knowledge of cloud services such as RDS, S3, SNS, CloudWatch and Lambda
- Advanced knowledge of DBT/Informatica or other ETL/ELT tools preferred
- Excellent knowledge of Snowflake
- Excellent knowledge of data modeling
- Excellent ability to build processes supporting data transformation, data structures, metadata, dependency and workload management
- Excellent knowledge of orchestration or scheduling tools
- Excellent ability to manipulate, process and extract value from large, disconnected datasets
- Advanced knowledge of DevOps and/or DataOps practices, CI/CD pipelines, and containerization technologies preferred
- Excellent verbal and written communication skills
- Strong attention to detail, organizational and multi-tasking skills required with the ability to adapt to changing priorities
- Ability to maintain confidentiality.
Benefits
- Health insurance
- 401(k) matching
- Paid time off
- Volunteer time off annually
- Tuition reimbursement
- Wellness initiatives to improve team member well-being and reduce health insurance costs
- Flexibility to dress for your day and opportunities for alternative work arrangements
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Vice President - Business, Data Architect role at TD Securities focusing on business data architecture and analytics capabilities. Collaborate with stakeholders to define and govern data models and ensure alignment with strategy.
Staff Data Engineer at Headspace building privacy - first data platforms for mental health support. Leading data engineering strategies and mentoring team members to enhance data - driven decision making.
Senior Data Engineer building and implementing data pipelines at Headspace. Collaborating with analytics and data science teams to enhance personalized mental health support.
Data Engineering Intern working on data pipelines and infrastructure in fast - growing fintech. Collaborating with data engineers, learning best practices and developing data solutions.
Senior Software Engineer building and maintaining data infrastructure for Gusto. Collaborating with Data Science and Business Intelligence teams to achieve their goals.
Data Engineer building and maintaining scalable data pipelines for AI Search Infrastructure at You.com. Collaborating across teams to ensure data quality and enable AI capabilities.
Data Engineer developing and managing technology - based data solutions for clients in different industries in Greece. Participating in software development lifecycle within Agile team setting.
Data Architect leading design and governance of high - quality data architectures for clients. Collaborating with engineering teams and stakeholders to transform business challenges into scalable data solutions.
Data Engineer supporting vehicle buying and selling solutions through integration pipelines. Collaborating with teams to build digital vehicle platforms and optimize data processes in São Paulo.