Senior Cloud Systems Engineer providing Databricks administration for government/financial services. Responsible for platform management, security, and automation in a hybrid setup.
Responsibilities
Administer Databricks accounts and workspaces across SDLC environments.
Standardize configuration, naming conventions, and operational practices.
Configure and maintain clusters, compute policies, SQL warehouses, runtime versions, libraries, jobs, repositories, and workspace settings.
Monitor platform health through operational dashboards, alerts, and monitoring tools.
Maintain operational documentation, runbooks, and platform procedures.
Implement and enforce least-privilege access controls across platform resources.
Manage identity integrations including SSO, SCIM provisioning, and role-based access control.
Administer service principals and group-based access permissions.
Enable audit logging and support security monitoring and compliance reviews.
Implement secure secrets management and connectivity patterns.
Administer Unity Catalog including metastores, catalogs, schemas, and tables.
Manage data ownership, permission grants, and governance policies.
Configure and maintain external locations and storage credentials.
Support data classification, tagging, and lineage integrations with governance teams.
Coordinate with cloud and network teams to establish secure connectivity patterns.
Implement storage access controls and secure object storage integrations.
Support cloud logging, monitoring, and security integration with enterprise platforms.
Automate platform configuration and administration using APIs, CLI tools, and Infrastructure-as-Code frameworks.
Implement CI/CD pipelines for deploying jobs, notebooks, and configurations across environments.
Implement Databricks Asset Bundles (DABs) for standardized deployment workflows.
Reduce configuration drift through automated deployment processes.
Implement cost control policies such as cluster policies and auto-termination rules.
Analyze usage metrics and provide recommendations to improve cost efficiency.
Monitor and optimize SQL warehouse performance and cluster autoscaling.
Implement Delta Lake optimization strategies including OPTIMIZE, VACUUM, and Z-ordering.
Administer Delta Live Tables pipelines and support data engineering teams.
Monitor pipeline health and address job failures or performance issues.
Support integrations with business intelligence tools and metadata catalog systems.
Assist with troubleshooting data pipeline and query performance issues.
Maintain platform configuration documentation and governance standards.
Develop onboarding materials and self-service guides for platform user.
Support user onboarding and workspace access provisioning.
Provide guidance to platform users and development teams on best practices.
Conduct capacity planning and forecast resource usage based on platform growth.
Monitor concurrent workloads and resource allocation.
Recommend scaling strategies to support increased platform usage.
Ensure platform stability during peak usage periods.
Requirements
Bachelor’s Degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.
7+ years of experience in cloud infrastructure, data platform administration, or enterprise platform operations.
3+ years of hands-on experience administering Databricks environments.
Structural Systems Engineer specializing in structural analysis of aerospace vehicle pressurized systems. Involving design, development, and execution of test programs for launch and space structures.
Systems Engineer at Quevera collaborating with experts to deliver innovative solutions. Join our dynamic team recognized as a top employer in the Baltimore/DC area.
Staff Systems Engineer working on delivering complex software applications into operations with a talented team at CACI. Supporting development and verification of mission capabilities while ensuring operational efficiency.
Senior Systems Engineer supporting mission - critical software and AI/ML product development. Collaborating within an Agile team to transition complex systems to operational use.
IT Support Specialist ensuring installation, support, and maintenance of IT systems in healthcare settings. Focusing on efficiency, stability, and customer service with a team - oriented approach.
RF Systems Engineer III developing spacecraft communication systems for civil, commercial, and National Security Space programs. Collaborating with cross - functional teams to enhance RF communications technology.
Systems Engineer supporting deployment and operational reliability in cloud - based healthcare platform. Collaborate with engineering and QA teams to manage cloud environments and troubleshoot issues.
Business Systems Analyst participating in daily support and enhancement of systems for health care. Involved in development and configuration to support Cambia's mission in health care.
Epic Systems Analyst supporting pharmacy IT systems for Connecticut Children’s. Utilizing expertise in complex application and systems enhancements or replacements.
Systems Analyst for Connecticut Children’s health improving computer systems and supporting colleagues. Utilizing data gathering techniques for effective solutions in a healthcare environment.