Platform Engineer responsible for building and managing Kubernetes platforms for Spark workloads at Comcast. Collaborating with data teams to ensure efficient data processing and cost optimization.
Responsibilities
Architecting and managing platforms where Spark runs, such as Kubernetes clusters or cloud services
Packaging Spark workloads and integrating with orchestration systems like Apache Flyte
Deploying Infrastructure via Terraform/Ansible
Troubleshooting job failures and execution anomalies
Optimizing Spark configurations to reduce costs
Building automation and tools in languages like Python, Java, or Scala
Writing medium to complex SQL Queries
Implementing monitoring systems for platform stability
Working closely with data engineers and scientists
Engaging with open-source communities to discuss technical challenges
Creating and maintaining documentation for Kubernetes infrastructure
Providing training and support to team members
Requirements
Bachelor's degree in computer science or a related field, or equivalent experience
Expertise in Apache Spark
Proficiency in Python, Pyspark, and Scala/Java
Proficient in Linux Scripting (Bash)
Proficient in writing SQL
Experience in CI/CD tools, Github
Experience in using observability tools like Prometheus, Grafana
Knowledge of networking concepts like TCP/IP, DNS, Load Balancer
Automation via Terraform/Ansible
Hands-on experience with cloud providers (AWS, Azure, GCP)
Familiarity with technologies like Delta Lake, Apache Iceberg, Apache Kafka, Hadoop
Strong analytical skills for debugging complex issues
Senior Full Stack Engineer responsible for developing and integrating AI tools in a HealthTech startup, providing impactful solutions for therapists' workflows.
Full Stack Engineer developing AI systems for a proactive chat app. Building features across frontend, backend, and AI integrations for reliable workflows.
Full - Stack Developer creating innovative digital solutions at Dreamlight Labs. Develop modern web interfaces and APIs while collaborating with a dynamic team in the creative industry.
Software Tech Lead responsible for Full Stack architecture and development at SPiNE Energy startup. Engage in innovative energy applications and scalable solutions for energy management.
Product Engineer responsible for full - stack development and feature ownership using AI tools. Join JustDice in Hamburg to build mobile and apps in a supportive environment.
Senior Staff Engineer at mylo responsible for large - scale technical impact across backend systems. Collaborating with teams to ensure architectural excellence and improve system reliability.
Full Stack Developer at Centah managing integrations and API features for a scalable SaaS platform. Contributing to system reliability and mentoring less experienced engineers.
Director of Software Engineering providing strategic and technical leadership across Ensemble’s software delivery organization. Drive technology strategy and lead engineering teams to deliver scalable software solutions.
Developing and maintaining state of the art solutions for McKesson. Senior Associate Software Engineer role requires collaboration with teams and participation in Agile methodology.
Lead Software Engineer responsible for influencing and implementing technology strategy for software delivery teams at Ensemble. Focused on developing reusable components and mentoring technical deliverables.