Senior Data Engineer building and maintaining robust data pipelines for various data products at Beep Saúde. Collaborating within the team and leading data governance practices.
About the role
- Senior Data Engineer developing data infrastructure and pipelines for fintech platform mylo in Egypt. Collaborating on data solutions using Python, SQL, and AWS services in a hybrid work environment.
Responsibilities
- Design, develop, and maintain large-scale, reliable data pipelines using Python, SQL, and big data technologies such as Apache Spark and Kafka.
- Build and optimize ETL/ELT processes for data transformation, loading, and integration from multiple sources.
- Develop and maintain data storage solutions using both relational and NoSQL databases, including SQL Server, PostgreSQL, MySQL, and MongoDB.
- Implement and manage CI/CD pipelines for data workflows, enabling automated deployments and version control.
- Work with AWS services to build, deploy, and monitor cloud-based, scalable data solutions.
- Leverage Apache Airflow for orchestrating workflows and PostHog for analytics tracking and event data.
- Manage and enhance data warehousing solutions to support business intelligence and analytics needs.
- Ensure data accuracy, consistency, and security across diverse systems and sources.
- Troubleshoot and optimize data systems for performance, scalability, and cost efficiency.
- Actively promote and contribute to a collaborative, innovative, and agile team
Requirements
- 5+ years of experience in data engineering, building and maintaining production-grade data pipelines and architectures.
- Proficient in Python and SQL.
- Hands-on with relational databases (SQL Server, PostgreSQL, MySQL) and NoSQL (MongoDB).
- Experience with big data and stream processing tools (e.g., Apache Spark, Kafka).
- Skilled in implementing CI/CD pipelines for data workflows.
- Strong understanding of AWS services (S3, Redshift, Lambda, Glue).
- Experience with Apache Airflow for workflow orchestration.
- Familiarity with PostHog or Amplitude for analytics tracking and event management.
- Comfortable with Docker, Kubernetes, and Linux shell scripting.
- Solid grasp of data modeling, warehousing, scalability, and reliability best practices.
- Proven ability to ensure data quality, governance, and security.
- Strong communication skills and a collaborative mindset.
- Passion for continuous learning and staying updated on emerging technologies.
Benefits
- Office environment: When you come to our b_labs office, you'll find creative workspaces and an open design to foster collaboration between teams.
- Flexibility: You know best whether you want to work from home or in the office.
- Equipment: From "Day 1" you will receive all the equipment you need be successful at work.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Software Developer in Test working on cloud - based data platform at Tecsys. Ensuring quality and reliability of data pipelines and transformations using automation frameworks.
Data Engineer responsible for designing, building, and optimizing data pipelines and architectures in a tech environment. Requires extensive experience with modern data warehousing and cloud platforms.
Lead Data Engineer role at Brillio focusing on AI & Data Engineering with expertise in Azure and MS Fabric. Collaborate within the Data Engineering team in Pune, Maharashtra, India.
Data Architect at Whiteshield designing scalable, secure data architectures for national and enterprise transformation programs. Architecting modern data platforms to support analytics, AI and operational use cases.
Data Engineer managing scalable data ecosystems for actionable business intelligence and cross - functional stakeholder collaboration. Optimizing ETL/ELT pipelines and ensuring data integrity and security.
Data Engineer specializing in data architecture and solutions for a banking environment, driving value for customers through innovative engineering practices and technologies in data management.
Technical Lead for data engineering and reporting in healthcare technology at Dedalus. Shaping innovative software solutions and leading cross - functional technical teams in Australia.
Senior ML Data Engineer working on data pipeline curation for Mobileye's autonomous vehicle dataset. Collaborating across teams to enhance ML engineering and vision model applications.
Data Engineer managing customer datasets to enhance industrial research and development. Responsible for ETL pipelines and data ingestion for the Uncountable Web Platform.