Hybrid Software Engineer, Data Infrastructure

at CrowdStrike

• Job title: Software Engineer, Data Infrastructure
• Job type: Full Time
• Experience level: Mid level, Senior
• Salary: $120,000 - $180,000 per year
• Degree requirement: Bachelor's Degree
• Tech skills: Apache, Chef, Cloud, Distributed Systems, HDFS, Java, Kafka, Kubernetes, Python, Spark, Yarn, Zookeeper, Go
• Location requirements: Hybrid, New York City, United States

About the role

• Software Engineer in CrowdStrike's Data Engineering team, focused on building Spark ecosystems and scalable infrastructure. Collaborating with engineers to create solutions for cybersecurity challenges.

Responsibilities

• Design and implement extremely scalable large scale distributed Spark cluster on Kubernetes platform optimized for Data center environment.
• Perform and oversee a variety of functions to ensure that our Spark infrastructure on Kubernetes is available, reliable, stable and secure
• Applies judgment in analyzing and selecting technologies, installation and maintenance of software and hardware systems that allow multiple Engineering and DataScience teams to interact with this system
• Design, writes and own APIs, automate and generate reports to visualize cluster health.
• Has responsibility for building and maintaining the Spark infrastructure such that software engineers, data analysts and data scientists can run jobs to gather data and insights.
• Work on Query systems, like Apache Presto or Trino, Jupyter Notebooks

Requirements

• Strong in one or both of field i.e., Spark ecosystem or Kubernetes ecosystem
• Strong analytical skills and with deep understanding of Distributed Systems
• Strong programming skills in languages as Go, Java or Python
• Understanding of Apache Spark ecosystem technologies (Deltalake/Iceberg, YARN, HDFS, Kafka, Zookeeper, Hive, Jupyter notebooks)
• Experience with large-scale business critical platforms with Spark on Kubernetes on data center or cloud.
• Experience with continuous deployment on K8 with Helm, fluxCD etc
• Solid understanding of Spark(data) and K8 storage systems (Object stores S3/Minio, Spark ephemeral storage, Persistent volumes mapping and claims)
• Understanding or spark memory management and experience with spark internals.
• Familiarity with Chef is preferred.
• Proven ability to work with both local and remote teams
• Strong communication skills both verbal and written.

Benefits

• Market leader in compensation and equity awards
• Comprehensive physical and mental wellness programs
• Competitive vacation and holidays for recharge
• Paid parental and adoption leaves
• Professional development opportunities for all employees regardless of level or role
• Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
• Vibrant office culture with world class amenities
• Great Place to Work Certified™ across the globe