Junior Data Engineer at OneMarketData focusing on data quality and integrity in financial datasets. Collaborating with senior analysts and assisting in data management and analysis tasks.
About the role
- Sr. Lead Data Engineer implementing data platforms with expertise in Databricks/AWS EMR. Collaborating with cross-functional teams to enhance data-driven initiatives.
Responsibilities
- Lead the design and implementation of distributed data processing workloads using Python, including data access from relational databases and cloud storage technologies.
- Ensure performance, security, and reliability; coordinate with application teams to define database design.
- Collaborate with cross-functional teams to support data-driven initiatives.
- Mentor junior team members and promote best practices.
- Oversee maintenance and troubleshooting of distributed data processing systems.
- Drive innovation by evaluating and integrating new technologies.
- Produce system design documents and participate in technical walkthroughs.
- Perform application and system performance tuning and troubleshoot performance issues.
- Effectively interact with global customers, business users, and IT staff.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent work experience.
- 10+ years of IT experience in application support or development.
- Design & build ingestion pipelines using ETL/ELT and schema-on-read in data lake technologies.
- Experience in designing transactional systems, data warehouses, data lakes, and data integrations within a big data ecosystem, particularly leveraging cloud technologies.
- Provide technical expertise in the design and implementation of data ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and large-scale data analytics tools.
- Strong experience in managing distributed data processing workloads, including configurations.
- Experience in working with multi-threaded, high-performance, low-latency messaging systems.
- Experience in AWS cloud-based technologies.
- Experience using system tools, source control systems, utilities, and third-party products.
- Excellent communication skills, with strong verbal and writing proficiencies.
Benefits
- Health & Wellness: Health care coverage designed for the mind and body.
- Flexible Downtime: Generous time off helps keep you energized for your time on.
- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
- Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Data Engineer Manager leading robust data solutions for PwC's clients. Transforming raw data into actionable insights and coaching team members.
Senior Data Engineering Analyst developing and implementing data solutions. Collaborating in a diverse environment focused on data processing and analysis for clients' digital transformation.
Principal Software Engineer in Threat Data Platform developing AI - driven tools for threat intelligence automation. Collaborating on robust data pipelines for PANW’s product ecosystem.
Senior Azure Data Engineer maintaining business intelligence solutions for Grupo Gloria, implementing and stabilizing projects in Azure and Databricks with Power BI reporting.
Staff Data Engineer at URBN developing AI - powered digital experiences by integrating algorithmic solutions with creative tools. Collaborating with cross - functional teams for impactful product evolution.
Senior Data Engineer at Anglian Water responsible for scalable data solutions and team collaboration. Leading design, build, and operation of secure data pipelines for critical services.
Data Engineer developing complex data pipelines for Symphony, a global software company. Collaborating with teams and driving data solutions in a hybrid work environment.
Data Engineer focused on building and maintaining data pipelines using SQL Server and T - SQL. Designing data solutions for reporting and analytics from various internal and third - party systems.
Data Engineer responsible for building scalable data solutions and collaborating with various teams. Focused on data extraction, transformation, and maintaining optimal architecture.