Senior Databricks Engineer at Jade Global responsible for designing and optimizing large-scale data pipelines. Collaborating with teams to transform raw data into actionable insights.
Responsibilities
Design and implement scalable ETL/ELT pipelines using Apache Spark on Databricks.
Build and optimize data lakes and data warehouses using Delta Lake and related tools.
Develop, schedule, and monitor workflows using Databricks Workflows, Airflow, or other orchestration tools.
Collaborate with data scientists, analysts, and stakeholders to understand data needs and deliver reliable solutions.
Tune Spark jobs for performance, scalability, and cost-efficiency.
Implement CI/CD pipelines and DevOps best practices for data engineering projects.
Ensure data security, compliance, and governance using tools like Unity Catalog.
Troubleshoot and resolve issues related to data ingestion, transformation, and integration.
Participate in data platform migration projects (e.g., from on-premise to Databricks or from legacy platforms to modern cloud-native architectures).
Mentor junior team members and contribute to a culture of technical excellence and innovation.
Requirements
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
6+ years of hands-on experience in data engineering or big data development
3+ years of experience working with Databricks, including Spark, Delta Lake, and MLflow
Strong programming skills in Python and SQL
Solid understanding of distributed systems and performance tuning in Spark
Experience with cloud platforms such as Azure, AWS, or GCP (Azure Databricks preferred)
Experience working with structured, semi-structured, and unstructured data
Familiarity with DevOps practices, including version control (Git), CI/CD, and infrastructure as code (Terraform, ARM templates)
Excellent problem-solving skills and ability to communicate complex technical ideas to non-technical stakeholders
Experience in Databricks migration projects (e.g., from Hadoop, on-premise, or legacy ETL tools to Databricks)
Ability to thrive in a fast-paced, dynamic, client-facing role where delivering solid work products to exceed high expectations is a measure of success
Excellent leadership and interpersonal skills
Eager to contribute to a team-oriented environment
Strong prioritization and multi-tasking skills with a track record of meeting deadlines
Ability to be creative and analytical in a problem-solving environment
Effective verbal and written communication skills
Adaptable to new environments, people, technologies, and processes
Ability to manage ambiguity and solve undefined problems.
Power Studies Engineer at EVS supporting advanced electrical system modeling and analysis across utility projects. Collaborating with teams to deliver high quality power system studies.
Project Engineer at Honeywell designing and delivering turbomachinery control systems. Engage in full project lifecycle, collaborating across teams to ensure operational efficiency and client satisfaction.
Memory Systems Firmware Development Engineer tasked with designing and implementing firmware for Intel's memory subsystems. Collaborating with cross - functional teams to enhance memory technology and ensure product innovation.
Engineer III responsible for designing control systems for high voltage substation projects. Collaborating with engineering teams to deliver technical solutions and project support.
Senior Environmental Engineer leading hazardous building materials assessments and Phase I/II Environmental Site Assessments in environmental consulting. Collaborating with local teams in Nova Scotia and Atlantic Canada.
Care Engineer delivering technical support for Nokia's NPC and CSD systems, ensuring reliability and performance. Collaborating with customers and global teams for problem resolution.
Sr. Principal PIC Development Engineer overseeing the design and development of Infinera’s Photonic Integrated Circuits products. Collaborating cross - functionally to manage project budgets and schedules.
Business Engineer developing client relationships for ABGi Technology while overseeing a team of consultants. Focused on sales growth and client engagement.
Commercial Refrigeration Engineer troubleshooting and resolving refrigeration systems for clients. Providing service and maintenance while developing knowledge in industrial systems with a mobile work structure.
Wintel Engineer responsible for maintaining the reliability and performance of hybrid Windows Server platforms. Ensuring consistency and security for business - critical applications in Glasgow.