Staff Software Engineer - Backend

San Francisco, California At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the world’s best data and AI infrastructure platform, so our customers can focus on the high value challenges that are central to their own missions. Founded in 2013 by the original creators of Apache Spark™, Databricks has grown from a tiny corner office in Berkeley, California to a global organization with over 1000 employees. Thousands of organizations, from small to Fortune 100, trust Databricks with their mission-critical workloads, making us one of the fastest growing SaaS companies in the world. Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data and AI technology, while simultaneously operating with the resilience, security and scale that is critical to making customers successful on our platform. We develop and operate one of the largest scale software platforms. The fleet consists of millions of virtual machines, generating terabytes of logs and processing exabytes of data per day. At our scale, we regularly observe cloud hardware, network, and operating system faults, and our software must gracefully shield our customers from any of the above. As a software engineer with a backend focus, you will work closely with your team and product management to prioritize, design, implement, test, and operate micro-services for the Databricks platform and product. This implies, among others, writing software in Scala/Java, building data pipelines (Apache Spark™, Apache Kafka), integrating with third-party applications, and interacting with cloud APIs (AWS, Azure, CloudFormation, Terraform). Below are some example teams you can join:

• Data Science and Machine Learning Infrastructure:

Build services and infrastructure at the intersection of machine learning and distributed systems. Our technology empowers the flagship collaborative workspace, notebooks, IDE integrations, and project management products. We also enable machine learning at scale with tools for environment management, distributed training, and managing the Machine Learning lifecycle through MLflow.

• Compute Fabric:

Build the resource management infrastructure powering all big data and machine learning workloads on Databricks platform in a robust, flexible, secure, and cloud-agnostic way. The software manages millions of virtual machines.

• Data Plane Storage:

Deliver reliable high performance services & client libraries for storing & accessing humongous amount of data on cloud storage backends e.g., AWS S3, Azure Blob Store.

• Enterprise Platform:

Offer simple powerful experience for onboarding & managing all their data teams across 10ks users on Databricks platform by building reliable scalable services & infrastructure with intuitive UIs & delivering high-impact cross-cutting projects driving "land & expand" strategy for enterprise customers.

• Observability:

Provide world class platform for Databricks engineers to comprehensively observe & introspect their applications & services by building scalable data-intensive infrastructure processing huge amounts logs & telemetry enabling teams to become more data-driven & build robust services.

• Service Platform:

Build high-quality services & manage services in all environments unified way providing engineers libraries tools services guidance to develop reliable scalable secure services; build unified platform for engineers to deploy/update services across clouds/environments.

• Core Infra:

Build core infrastructure powering Databricks available across geographic regions & Cloud providers; build highly available distributed systems heavily utilizing cloud native projects contributing back whenever possible; run thousands Kubernetes clusters across regions orchestrate millions VMs daily. Competencies:

• BS/MS/PhD in Computer Science or related field
• 10+ years production level experience in Java Scala C++ or similar language
• Comfortable working towards multi-year vision incremental deliverables
• Experience architecting developing deploying operating large scale distributed systems
• Experience SaaS platforms or Service-Oriented Architectures
• Good knowledge SQL
• Experience software security sensitive data handling
• Experience cloud technologies AWS Azure GCP Docker Kubernetes

Pay Range: $192000—$260000 USD local pay range About Databricks: Founded in 2013 by original creators Apache Spark™ global organization over 1000 employees thousands organizations small Fortune 100 trust mission-critical workloads fastest growing SaaS companies world headquarters San Francisco Benefits: Comprehensive benefits details available at https://www.mybenefitsnow.com/databricks Diversity & Inclusion: Committed fostering diverse inclusive culture hiring practices meet equal employment opportunity standards compliance employer discretion applying U.S government license export-controlled technology access application process includes fields personal information resume/CV upload cover letter upload legal work authorization questions voluntary self-identification surveys