Senior Data Engineer focusing on data infrastructure for an AI - driven insurtech startup based in Nepal. Collaborating with teams to optimize data models and maintain data quality.
About the role
- Senior Data Engineer at Skillfield designing distributed data processing solutions using Apache Spark. Collaborates on cloud and on-prem solutions across enterprise levels in a hybrid work environment.
Responsibilities
- Design, develop, and maintain ETL and ELT pipelines using Apache Spark (batch and streaming)
- Build Spark applications in Scala, applying distributed processing best practices
- Optimise Spark workloads to improve performance, scalability, and reliability
- Work within the Hadoop ecosystem, including HDFS, Hive, and HBase
- Design and support high-performance analytical solutions using ClickHouse
- Build and manage data ingestion workflows using Apache NiFi
- Operate and troubleshoot data platforms in Linux-based environments
- Partner with data architects on solution design, schemas, and data models
- Investigate and resolve data pipeline failures and performance issues
- Maintain technical documentation and delivery artefacts in Confluence
- Track delivery progress and work items using Jira
Requirements
- Hands-on experience building solutions with Apache Spark
- Strong Scala development capability
- Experience working in Linux environments
- Practical knowledge of HDFS, Hive, and HBase
- Experience with ClickHouse or comparable analytical databases
- A solid understanding of distributed systems and performance optimisation
- Experience working across cloud, hybrid, or on-prem platforms
- Familiarity with Git-based workflows and CI/CD pipelines
- Nice to Have
- Experience with Kafka or other streaming platforms
- Exposure to Databricks, EMR, or managed Spark services
- Experience with orchestration tools such as Airflow
- Awareness of data security, identity management, and governance practices
Benefits
- Enjoy flexibility, support, and a focus on sustainable delivery
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Senior Professional Consultant leading architecture and design for SAP BW and SAC solutions at Freudenberg. Collaborating with stakeholders and optimizing performance of data landscapes.
Senior Data Engineer designing and managing data architectures to transform large - scale data into insights for Humana. Involves leading technical discussions and implementing best data practices.
ウーブン・バイ・トヨタのデータプラットフォーム実現に情熱を持つエンジニアを募集中。スケーラブルなデータアーキテクチャの設計・実装を担当します。
Data Engineer II at Early Warning Services developing data science tools and infrastructure. Collaborating on software enhancements and mentoring interns in a hybrid work environment.
Senior Data Architect responsible for optimizing data architecture and supporting data - driven business decisions at TruStage. Leading technical guidance for data architecture and cross - functional team collaboration.
Senior Data Architect developing data architecture plans at The Hartford, collaborating with internal teams to align data standards and practices. Leading complex solutions with a focus on operational effectiveness.
Senior Solution Architect defining architecture framework for SA‑CCR in regulatory risk. Collaborating with stakeholders to ensure compliance and efficient data governance.
Senior Data Engineer optimizing and designing data pipelines on AWS for The Rec Hub. Collaborating with the team to enhance data processing and mentorship.
Data Engineer at TeCreation focusing on data analysis and innovative system development in the well - being industry. Collaborating on data integration and business intelligence reporting.