Software Engineer at Warner Music Group developing an innovative Data Platform for the music industry. Collaborating with dynamic teams to enhance music data processing and delivery.
About the role
- Senior Data Engineer (AWS) with expertise in Python and data services. Working on enterprise-scale data processing and analytics initiatives in a hybrid model.
Responsibilities
- Design, develop, and maintain scalable data processing pipelines using Python, PySpark, and Spark SQL
- Build and optimize distributed data processing workflows on AWS platforms.
- Leverage AWS data services such as EMR, Glue, Lambda, and S3 for batch and real-time data processing.
- Design and manage data storage solutions using RDS/MySQL, Redshift , and other AWS-native databases.
- Implement effective data modeling, schema design, and schema evolution strategies.
- Perform performance tuning and optimization of Spark jobs and SQL queries.
- Monitor and troubleshoot data pipelines using AWS CloudWatch and logging frameworks.
- Manage secrets and credentials securely using AWS Secrets Manager.
- Collaborate with data architects, analysts, and stakeholders to translate business requirements into technical solutions.
- Debug complex data issues and provide root cause analysis with long-term fixes.
- Ensure data quality, reliability, and scalability across platforms
Requirements
- 10–13 years of overall experience in Data Engineering
- Strong proficiency in Python and SQL
- Extensive hands-on experience with PySpark and Spark SQL
- Strong experience with AWS data services , including: EMR Glue Lambda S3 RDS / MySQL Redshift CloudWatch Secrets Manager
- Solid understanding of distributed computing concepts
- Strong experience in data modeling, schema handling, and performance tuning
- Excellent debugging, analytical, and problem-solving skills.
- Ability to work effectively in a hybrid and collaborative environment
Benefits
- Immediate joiners only
- Hybrid work mode
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer role specializing in Azure & Snowflake at InfoCentric. Leading design and delivery of enterprise - scale data platforms for large organizations.
Principal Data Architect at PointClickCare ensuring coherent and scalable data architecture. Driving unified data direction while collaborating with Engineering Architecture team for AI enablement.
Data Engineer Tech Lead developing data solutions at Carelon. Leading a cross - functional team to optimize data workflows and maintain data integrity.
Lead Data Engineer responsible for evolving Manna’s data infrastructure for drone delivery. Overseeing data architecture and analytics while building scalable data pipelines.
Data Engineer designing, implementing, and optimizing data pipelines for DeepLight AI. Collaborating closely with a multidisciplinary team to analyze large - scale data.
Data Engineer designing and maintaining scalable ETL pipelines at Satori Analytics. Collaborating with teams to deliver high - quality analytics solutions across various industries.
Data Architect responsible for defining enterprise data architecture on AWS and Databricks Lakehouse platforms. Enabling scalable data lakes and enterprise analytics for financial services organizations.
Data Platform Operations Support leading data engineering strategy across projects for EXL. Driving innovation and optimization while collaborating with various teams in the organization.
Manager II leading data engineering projects at Navy Federal Credit Union. Overseeing data governance and quality initiatives while managing engineering teams in a hybrid work environment.