If you are a devops engineer working remotely or on a distributed team, you have a couple of challenges that can make or break your success working with agile developers, quality assurance engineers, site reliability engineers, and other system operators.
The first challenge is the ops responsibilities: making sure the systems and services operate reliably. According to a recent survey on the future of monitoring and AIops, 61 percent of respondents state that both the network operations center and devops engineers are responsible for responding to system and application incidents. So, most devops engineers need to help resolve operational issues, such as scaling infrastructure, dealing with blocks in build pipelines, or providing subject matter expertise on security issues.
The second challenge is the dev responsibilities, especially developing and supporting CI/CD (continuous integration/continuous delivery) pipelines, infrastructure as code, and other automations. Devops engineers may be members of agile development teams and take on user stories to build or enhance automation. Other times, devops engineers can be part of a shared services team that supports these automations for multiple development teams. Regardless of the approach, creating CI/CD and infrastructure automation requires collaboration to understand functional requirements, operating environments, compliance factors, security posture, and performance considerations.
Although devops practitioners have dev and ops responsibilities, devops more specifically refers to the collaboration between application development and IT operations. My definition of devops focuses on collaboration. “Devops is about the culture, collaborative practices, and automation that aligns development and operations teams, so they have a single mindset on improving customer experiences, responding faster to business needs, and ensuring that innovation is balanced with security and operational needs.”
Devops requires establishing working principles to drive collaboration and foster the culture. If you consider yourself a devops engineer, study these best practices, especially if you work remotely or on a distributed team.
Meet your teammates where they are
The first important consideration is to understand who your teammates are and what tools they use to collaborate. This sounds simple, but it may not be in large organizations where departments and teams have some autonomy to select their tools and collaboration practices. It’s further complicated because, as a devops engineer, you may have to use one set of tools when working on dev responsibilities and a second set when responding to ops issues.
Remote engineers and distributed devops teams should not grumble over these complexities or try to force people into a common way of working. If devops collaboration and culture are to be successful, remote engineers must start by joining the tools and remote collaboration practices the teams already use.
This could mean updating user stories in Jira when working on CI/CD pipelines, responding to tickets in Cherwell when there are application alerts, and then having a dialog with business stakeholders in Slack. Working with the different tools your teams are accustomed to using is critical for remote engineers.
Invest in devops reporting capabilities and documentation
Much of devops focuses on automation, including continuous integration, testing, deployment, infrastructure, and application monitoring. Responsible software developers aim for robust and reliable applications; devops engineers should do the same for the automations that they develop.
These automations have customers, and they will have operational incidents. For example, devops engineers should consider the application developers as their customers for CI/CD pipelines, and system IT operations for the infrastructure as code scripting. Both groups will escalate issues to devops engineers if builds fail or when infrastructure doesn't deploy.
These escalations can be a particularly bad scenario for remote devops engineers or distributed teams. If an issue blocks teammates from doing their jobs well, it impacts everyone’s productivity and collaboration. And working remotely means you can’t simply go tap a teammate on the back to get problems resolved.
Seeing as devops engineers provide services to other engineers, there’s an opportunity to make devops automations and code transparent and supportable by others. Design reports, dashboards, and documentation so that automations can be reviewed, diagnosed, and improved on by others. Automations should be centralized in version control and have their own documentation and change controls.
The implementations also need strong alerts, reporting, and logging. Remote teams should create detailed alerts so teammates understand them without needing to ask for help. Build pipelines should have enough reporting to help users know what failed, where, and why. When infrastructure setup and configurations are automated with code, when each task is surrounded by strong error checking and other information-level logging, then end-users (who are engineers) can diagnose and resolve issues on their own.
Lead the organization on quality and security
One other risk of distributed teams is wrongly assuming that a critical capability that you can’t easily see or experience is someone else’s responsibility. Many in devops focus on the highly visible key performance indicators, such as deployment frequency and mean time to resolve issues, while other factors, such as automated testing and proactive security, fall behind.
In the 2020 DevSecOps Community Survey, 55 percent of respondents deploy at least once per week, and 18 percent claim multiple daily deployments. But when it comes to security, 45 percent of the survey’s respondents know it’s important but don’t have time to work on it.
Security isn’t the only gap. In the 2019 state of testing report, only 25 percent of respondents claimed they have more than 50 percent of their functional tests automated.
Although these gaps are prevalent across a large number of devops organizations, there’s even less of a chance that they'll be championed when devops engineers are remote or working on distributed teams. In many organizations, it’s hard to step up and take responsibility for an underinvested technical function. It’s even more challenging for distributed teams.
If there is a sizable gap in quality or security, devops engineers on agile teams should add spikes on the backlog to experiment with shift-left testing and CI/CD security integrations. That’s one way to draw attention to the gap and take responsibility to address it.
Lastly, many teams are working remotely because of COVID-19. These seven best practices for remote agile teams also apply to devops engineers.