Senior Site Reliability Engineer, Infrastructure Foundations The Wikimedia Foundation is looking for a Senior Site Reliability Engineer to support and develop the platform serving the world’s favorite encyclopedia, Wikipedia, to millions of people around the globe. Wikimedia’s Site Reliability Engineering (SRE) team is principally responsible for ensuring our global top-10 website and its underlying infrastructure is healthy and developing further in support of Wikimedia’s mission: to help everyone share in the sum of all knowledge. The SRE team at Wikimedia is a globally distributed and diverse team of engineers with a drive to explore, experiment, and embrace new technologies. We work in the open by publishing all of our documentation and code as open source, and all our production systems are powered by open source software. We invite you to go through our documentation and code -- no login required. If you find what we do interesting, if you are up to the challenge of improving the reliability and delivery of one of the Internet’s top websites, and you enjoy the idea of working in a remote-first role, we may just be the right place for you. You are responsible for:

Performing day-to-day operational/DevOps tasks on Wikimedia’s public facing infrastructure (deployment, maintenance, configuration, troubleshooting)
Implementing and utilizing configuration management and deployment tools (Puppet, Kubernetes)
Leading continuous improvement, by automating the installation, configuration and maintenance of services on our platform
Work closely with product teams helping them bring scalable functionality to our users by assisting in the architectural design of new services and making them operate at scale
Participating in a 24/7 on-call rotation shared across the broader SRE team. This includes taking part in incident response, diagnosis and follow-up on system outages or alerts across Wikimedia’s production infrastructure.
Collaborating with a global, cross-functional team in an asynchronous communication environment
Mentoring peers in your areas of technical and operational strength
Ability and willingness to travel 1-2 times a year for in-person events and team meetings
Most importantly, share our values and work in accordance with them

Skills and Experience:

6+ years of experience in an SRE/Operations/DevOps role as part of a team
Experience with shell and any scripting languages used in an SRE context (Python, Go, Bash, Ruby; we primarily use Python) and configuration management tools (Puppet, Ansible; we use Puppet)
Experience designing and managing infrastructure security for large fleets of diverse services
Experience with technical response during security incidents
Experience with package management on Linux systems (we use Debian)
Strong Linux system-level troubleshooting skills
History of automating tasks and processes, identifying process gaps, and finding automation opportunities
Strong English language skills (verbal and written) and ability to work independently as an effective part of a globally distributed team working across multiple time zones
Experience leading and participating in incident response and post-incident review rituals with root cause analysis implementation

Additionally:

Experience setting fleet-wide security policies
Software supply chain security knowledge
Awareness of open source infrastructure security landscape
Experience working with software security teams
Credential management systems experience
Immutable logging/auditing implementation experience
Use/maintenance/configuration of monitoring infrastructure like Prometheus/Grafana
Developing/contributing to Free/Open Source software or communities preferred
LAMP stack technologies experience including PHP/HHVM/memcached/Redis; MediaWiki experience is a plus
Defining cross-team SLOs implementation experience

About the Wikimedia Foundation