Senior Data Engineer leading data engineering initiatives for automotive client. Building and optimizing data pipelines using Python and PySpark in a hybrid work model.
Responsibilities
Design, develop, and optimize scalable data pipelines using Python and PySpark for batch and streaming workloads
Build, schedule, and monitor complex workflows using Airflow, ensuring reliability and maintainability
Architect and implement CI/CD pipelines for data engineering projects using GitHub, Docker, and cloud-native solutions (GCP preferred)
Apply test-driven development (TDD) practices and automate unit/integration tests for data pipelines
Implement secure coding best practices and design patterns throughout the development lifecycle
Work closely with Data Architects, QA teams, and business stakeholders to translate requirements into technical solutions
Create and maintain technical documentation, including process/data flow diagrams and system design artifacts
Lead and mentor junior engineers, providing guidance on coding, testing, and deployment best practices
Analyze and resolve technical issues across the data stack, including pipeline failures and performance bottlenecks
Cross-train team members outside the project team (e.g., operations support) for full knowledge coverage
Requirements
7+ years of Data Engineering experience building production-grade data pipelines using Python and PySpark
Experience designing, deploying, and managing Airflow DAGs in enterprise environments
Experience maintaining CI/CD pipelines for data engineering workflows, including automated testing and deployment
Experience with cloud workflows and containerization, using Docker and cloud platforms (GCP preferred) for data engineering workloads
Knowledge and ability to follow twelve-factor design principles
Experience and ability to write object-oriented Python code, manage dependencies, and follow industry best practices
Proficiency with Git for source code management and collaboration (commits, branching, merging, GitHub/GitLab workflows).
Experience working with command lines in Unix/Linux-like environments
Solid understanding of SQL for data ingestion and analysis
Engineering mindset. Able to write code with an eye for maintainability and testability
Collaborative mindset. Comfortable with code reviews, paired programming, and using remote collaboration tools effectively
Detroit Labs is not currently able to hire candidates who will reside outside of the United States during their term of employment
Benefits
Full medical, dental, vision benefits
401K contribution options
Quarterly outings and events
Paid holidays and vacation time
Parental leave program
Monthly budgets for “team fun” bonding events
Free lunch for various company meetings and Lunch & Learns
Access to our mentorship program and employee resource groups (ERGs)
Senior Data Engineer at Goodwin enhancing data platforms and fostering data - driven culture across teams. Collaborating with IT and Finance on technology solutions and data governance practices.
Director, Data Platform Design and Strategy at MedImpact leading data platform and AI innovations to enhance healthcare services. Overseeing enterprise projects and managing teams to meet strategic goals.
Data Engineer delivering AI - and data - driven solutions for Honeywell’s industrial customers. Architecting and implementing scalable data pipelines and platforms focused on IoT and real - time data processing.
Data Engineering Associate focusing on data quality control and management for distribution platform. Collaborates on large scale data projects to ensure data accuracy and availability for users.
Data Architect managing enterprise data platform built on Microsoft Fabric at Johnstone Supply. Leading architectural standards and collaborating with business and IT leaders for strategic data - driven insights.
Data Engineer at Studyportals responsible for data pipelines and infrastructure. Join a team ensuring accurate and trustworthy data for analytics and business decisions.
AI/ML Engineer designing and refining prompts and workflows using large language models. Responsible for developing data pipelines and delivering scalable AI solutions in a hybrid work environment.
AWS Data Architect at Fractal designing and operationalizing AWS data solutions at enterprise scale. Collaborating with clients and mentoring engineers in best practices.
Senior Data Engineer driving data - driven success at Pacific Life. Collaborating with a team to build scalable and secure data solutions in Newport Beach, CA or Charlotte, NC.
Data Architect managing Commercial Data architecture initiatives for Valmet's sales and service team. Leading AI - driven data integrity and quality efforts in a global context.