Data Solutions Architect leading business intelligence solutions and analytics at Crowe. Overseeing data pipelines and analytics frameworks to drive decision - making and compliance.
About the role
- Data Engineer Tech Lead developing data solutions at Carelon. Leading a cross-functional team to optimize data workflows and maintain data integrity.
Responsibilities
- Enforces coding standards, reviews code, and ensures maintainability.
- Coaches developers on technical skills and problem-solving.
- Helps team members grow in their technical careers.
- Develop ETL/ELT processes to ingest and transform data from various sources including MongoDB, APIs, Snowflake and flat files (JSON, ORC, Avro, Parquet, CSV).
- Implement data loading strategies into Snowflake and other data warehouses.
- Write and optimize complex SQL queries for data extraction, transformation, and analysis across multiple database platforms.
- Works with Product Managers, Architects, and other teams to align technical goals with business needs.
- Leverage AWS services to manage and orchestrate data workflows, ensuring high availability and scalability.
- Implementing the Job in AWS EMR, AWS Glue , AWS Lambda using pyspark.
- Perform performance tuning on Spark jobs and SQL queries to ensure efficient data processing.
- Participate in Agile ceremonies and contribute to sprint planning, story grooming, and retrospectives.
- Collaborate with cross-functional teams including data scientists, analysts, and DevOps engineers to deliver data solutions.
- Ensure data accuracy, consistency, and integrity through validation and quality checks.
- Maintain documentation for data pipelines, schemas, and data flow processes.
- Work with Kafka or similar technologies to build and maintain real-time data streaming solutions.
- Develop and maintain data ingestion processes from external/internal APIs, ensuring secure and reliable data flow.
Requirements
- 5+ years’ experience in Spark ecosystem, Python/Scala programming, MongoDB data loads, Snowflake and AWS platform (EMR, Glue, S3)
- 6+ years’ IT experience and good expertise in SDLC/Agile
- 6+ years’ experience in SQL, complex queries, and optimization
- Coaches developers on technical skills and problem-solving.
- Hands on experience in writing advanced SQL queries, familiarity with variety of databases.
- Experience in coding solutions using Python/Spark and performing performance tuning/optimization.
- Experience in building and optimizing ‘Big-Data’ pipelines in Cloud.
- Experience in handling different file formats like JSON, ORC, Avro, Parquet, CSV.
- Hands on experience in data processing with NoSQL databases like MongoDB.
- Familiarity and understanding of jobs scheduling.
- Hands on experience on working with APIs to process data.
- Understanding of data streaming, such as Kafka services.
- Certification on Snowflake (Snow PRO certification), and AWS (Cloud Practitioner/Solution Architect).
- Hands-on experience on Kafka streaming pipelines implementation.
Benefits
- Extensive focus on learning and development
- An inspiring culture built on innovation, creativity, and freedom.
- Holistic well-being
- Comprehensive range of rewards and recognitions
- Competitive health and medical insurance coverage
- Best-in-class amenities and workspaces
- Policies designed with associates at the center.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Lead Data Engineer designing and building scalable data solutions utilizing AI technology for a globally recognized financial institution. Serving sophisticated clients across the globe.
Data Engineer Consultant building and maintaining enriched data infrastructure for analytical thinking at Northwest Permanente. Involves data collection, cleansing, and transformation for business intelligence.
Vice President - Business, Data Architect role at TD Securities focusing on business data architecture and analytics capabilities. Collaborate with stakeholders to define and govern data models and ensure alignment with strategy.
Staff Data Engineer at Headspace building privacy - first data platforms for mental health support. Leading data engineering strategies and mentoring team members to enhance data - driven decision making.
Senior Data Engineer building and implementing data pipelines at Headspace. Collaborating with analytics and data science teams to enhance personalized mental health support.
Data Engineering Intern working on data pipelines and infrastructure in fast - growing fintech. Collaborating with data engineers, learning best practices and developing data solutions.
Senior Software Engineer building and maintaining data infrastructure for Gusto. Collaborating with Data Science and Business Intelligence teams to achieve their goals.
Data Engineer building and maintaining scalable data pipelines for AI Search Infrastructure at You.com. Collaborating across teams to ensure data quality and enable AI capabilities.
Data Engineer developing and managing technology - based data solutions for clients in different industries in Greece. Participating in software development lifecycle within Agile team setting.