Why Developers Will Take Charge of Security, Tests in Prod
Developers will “eat” the IT security function, predicted Larry Maccherone, who works as the Dev[Sec]Ops transformation architect for security automation platform Contrast Security and made headlines as the author of the DevSecOps Manifesto. It’s inevitable, he argued. And as if that’s not shocking enough, testing in production should become the norm, he added.
It may sound crazy, and the InfoBip audience of primarily new programmers seemed hesitant to embrace the idea. Nonetheless, Maccherone laid out his case at an InfoBip Shift workshop in Miami last month.
Agile Drives Development To Eat Other Functions
It’ll be one facet in a list of functions that developers have taken over. Development has already “devoured” IT/Ops, thanks to cloud native environments because developers no longer need IT/Ops to spin up and maintain a server, he said; they can go online and do that themselves. For a long time, this “Shadow IT” was fought by IT/Ops; now it’s just how things work, he contended.
“If you think back to the early days of cloud native and the DevOps movements, the IT department was the first department you went to when you needed a new server to stand up your new application. You had to fill out a ticket and wait six months till the machine came in, they configured it, and they gave it to you,” he said. ”Now, you can go on AWS and, in three clicks, you can have a new server.”
Larry Maccherone built his case for development owning security at the spring InfoBip conference.
The same phenomenon is going to happen with security, thanks to automation and the speed at which development now happens, he predicted. And it will happen whether developers want responsibility for security or not.
“The long arc of history for software engineering has been that the development team itself takes over more and more responsibility for all of these things that used to be done by security, by specialists,” he said. “You would think that you, as the development team, should own responsibility for everything that’s happening around the application that could affect your ability to deliver value to your customers. When somebody else arbitrarily controls one part of it as a silo, then it’s not very effective.”
The trend started with agile teams owning the quality of the products they’re producing. Before that, quality assurance was responsible for that, but within 10 years, QA departments started to disappear.
“There’s very few dedicated QA departments at most organizations now,” he said. “Basically, cross-functional teams in Agile, Scrum caused development to subsume most of the responsibility … for the quality of the products they’re producing.”
While it is not an absolute — there are still true DevOps departments — today when people at many organizations say IT, they mean the help desk, he said.
Why Development Should Own Security
Security as a standalone unit is a bit of a drag. Essentially, they function as gatekeepers who slow down developers, he explained. They set policy and maybe have pages of checklists for developers to follow, but the function has no teeth. There aren’t consequences when development ignores security, but often there are consequences for the chief information security officer (CISO) when they try to play hardball.
“The reason there’s no consequences, though, is because the power dynamic is that you, as developers … [have] more power than the security folks, so that CISO has been beat up in meetings with his boss and the VP of engineering, where they said security just threw this crap on us, and we couldn’t ship on time,” he said.
Development is shipping to production multiple times a day, and that’s created a pace that blows the minds of security staff, he added. They can’t keep up.
Not only that, but it’s demoralizing work, because you basically call somebody’s application flawed, and that has led to high turnover in security, he noted.
“Nobody wants to do it. Nobody wants to be on the receiving end of it,” he said. “But here’s the really scary part. What about the people that stay? They’re the ones that like calling someone’s baby ugly all day long. They’re not going to be very good at building relationships and moving people, they’re just going to get angry and mad and try to enforce things and the developer is going to keep ignoring them.”
Often when security workers do try to enforce the policies, they find themselves on the losing end with no consequences because the cost of enforcing security is a trade-off in speed.
One Path to a Shift Security Left
Shift left means moving security responsibilities to the development team. That’s also what DevSecOps meant, but the meaning has drifted since he wrote his book, Maccherone said. He now prefers the phrase developer-centric application security.
“This is why I drifted away from the phrase DevSecOps. A lot of people think DevSecOps is slapping DevOps lipstick on traditional security,” he said. “That’s not my definition.”
Platform engineering is critical to this shift because it empowers software engineers to take ownership and do the work themselves. DevOps and security teams need to shift their thinking to supporting developers rather than trying to control them.
“What they need [is] to shift to being tool smiths for the development team to own that function,” he said.
He foresees platform engineering playing a role in this next evolution of application security. Basically, platform engineers will provide the “easy button self-service capability that development teams can just consume without human interaction,” he said. It’s an approach he used when he worked at Comcast. He also had coaches who supported the developers in following best practices for security.
Accountability also shifts to the development team, he added. After the talk, I asked him how that might happen, and he suggested a system where developers are actually evaluated on their security posture and eventually penalized if they don’t take action on securing their applications.
”If the developers write the vulnerabilities into the software, yet the security people are the ones who get called out and maybe even go to jail, potentially, if they get hacked… it’s not a very effective thing,” he told the audience. “You have to have the feedback come to the people who are responsible for causing the pain.”
Testing in Production
One way to get developers to take security more seriously is to move testing and fixing vulnerabilities to correspond to the pull request, he added. That will mean testing in production, which he contended is more effective and efficient anyway.
The best way to get developers to take security seriously is to move testing and fixing vulnerabilities to the pull request, argued Maccherone.
“I’m basically just arguing that the pull request is the ideal place to put it,” he told the audience of developers. “Why? The pull request is very immediate and it’s very important to the developer who submits it. You’re nodding! You’re passionate about the code you wrote that morning.”
Yet he estimated that 90% of the clients he works with push this security testing downstream to a point that doesn’t really make sense when you consider developer workflow. Most security people don’t even know what a pull request is, he added.
Automated testing tools will be key to making this possible, added Maccherone.
”If an automated tool gives you a message that says, ‘This has this vulnerability problem, this has this linting problem,’ then you’re going to fix it immediately because you definitely don’t want the guy on your team or the gal on your team who’s responsible for reviewing that to say, ‘Hey, you don’t pass that automated test, I’m not going to approve your code to get merged in,’” Maccherone said. “So developers are highly motivated to do this at this point.”
