Data Engineer building scalable data systems for e-commerce use cases. Focus on automation, pipeline orchestration, and leveraging cloud services to improve data architecture.
Responsibilities
Build and maintain scalable data pipelines to process and integrate e-commerce data from multiple sources
Develop production-ready Python scripts for API consumption, web scraping, and data transformation
Design and implement cloud-based data architecture using AWS services such as S3, EC2, and Lambda
Orchestrate and monitor scheduled data collection workflows using Apache Airflow
Evaluate and implement tools for high-performance data transformation and analysis (e.g., Polars, DuckDB, PySpark)
Collaborate with data analysts to deliver accessible, well-structured datasets for reporting and advanced analytics
Identify opportunities to improve data quality, reliability, and performance across all data collection endpoints
Requirements
Proficiency in programming languages such as Java and Python
Hands-on experience with SQL and database design
Previous experience as a Data Engineer or in a similar role
Strong technical knowledge of web data collection architectures, including proxy and header management
Excellent numerical and analytical skills
Willingness to learn and adapt to new tools and technologies
Proactive, curious, and highly tenacious, with a strong drive to grow and stay ahead of industry trends
Comfortable navigating a fast-paced, ambiguous environment with a high degree of independence
A Bachelor’s degree in a quantitative field (e.g., Computer Science, Engineering, Information Systems) is preferred
Software Engineer at Warner Music Group developing an innovative Data Platform for the music industry. Collaborating with dynamic teams to enhance music data processing and delivery.
Data Engineer role specializing in Azure & Snowflake at InfoCentric. Leading design and delivery of enterprise - scale data platforms for large organizations.
Principal Data Architect at PointClickCare ensuring coherent and scalable data architecture. Driving unified data direction while collaborating with Engineering Architecture team for AI enablement.
Data Engineer Tech Lead developing data solutions at Carelon. Leading a cross - functional team to optimize data workflows and maintain data integrity.
Lead Data Engineer responsible for evolving Manna’s data infrastructure for drone delivery. Overseeing data architecture and analytics while building scalable data pipelines.
Data Engineer designing, implementing, and optimizing data pipelines for DeepLight AI. Collaborating closely with a multidisciplinary team to analyze large - scale data.
Data Engineer designing and maintaining scalable ETL pipelines at Satori Analytics. Collaborating with teams to deliver high - quality analytics solutions across various industries.
Data Architect responsible for defining enterprise data architecture on AWS and Databricks Lakehouse platforms. Enabling scalable data lakes and enterprise analytics for financial services organizations.
Data Platform Operations Support leading data engineering strategy across projects for EXL. Driving innovation and optimization while collaborating with various teams in the organization.
Manager II leading data engineering projects at Navy Federal Credit Union. Overseeing data governance and quality initiatives while managing engineering teams in a hybrid work environment.