Google is now paying developers more money to work on securing their Linux kernels this year. The gesture may well be the start of the company's bid to enforce a tighter grip on open source.
Google's action comes on the heels of rising threats to Linux that unfolded in the last year, as hackers pivot to new strategies like writing malware strains in the Go programming language.
The spread rate of malware is staggering. Infected code incidents made a 500 percent spike in the last year. That represents a 2,000 percent increase since 2017, according to Google.
This spike is no doubt because Go allows hackers to be versatile and target Windows, Mac, and Linux from the same codebase. Adding to this, 2020 alone saw a 40 percent increase in Linux-related malware families.
We can speculate all we want on the number of factors driving this shift, such as accelerated cloud adoption. But there is no denying the existence of a significant marketplace gap.
Capsule8 is a firm that specializes in securing Linux-based production environments. It's chief scientist, vice president of product strategy Brandon Edwards, does not expect this trend in Linux vulnerabilities to die down.
"I know that they have been dedicated towards Linux kernel security at the developer level. And, you know that can obviously be a challenging effort because getting review and approval for code to go into the kernel always brings on a fight it seems," Edwards told LinuxInsider.
The second is confidentiality mode where you cannot even do that. The idea is that in integrity mode you cannot modify any kernel code. But you can still observe behaviors in the kernel.
Code visibility is a crucial stem for reviewing open-source coding. How Google plans to carry out the intervention is a major concern for Edwards.
"My fear is that when Google draws that sort of line. If its software coders remove visibility, they are actually going to hurt security more," warned Edwards.
That is especially if it is under the pretense or assumption that whatever preventative measure they are putting in place means they no longer need visibility, he reasoned.
Security mitigation can almost always be bypassed. Usually, it is bypassed in a way that can be generalized. The mitigation is never an obstacle that an adversary has to bypass at once, according to Edwards.
"But if you have solid visibility or observability, it means that the attacker must take into consideration how every artifact or side effect of the movement might reveal their presence," he detailed.
So, visibility should enable observing the same accident mitigation as intended to prevent an accident. At the same time, coders must be mindful of the biases that go into creating the mitigation, he cautioned.
"You will definitely have biases when you create observability. But those biases still lend themselves towards observing things that you did not know were going to happen. That can be used for security's sake," Edwards said.
This type of support is not surprising for the Google conglomerate. Google has joined efforts to fund Linux security developers for open-source projects, according to Linux officials. For instance, in early February Google donated $350,000 to the Python Software Foundation to support projects focused on improving supply-chain security.
That organization is also a founding member of the Open Source Security Foundation. OSSF is a Linux Foundation project dedicated to improving open-source software security.
According to Dan Lorenc, the engineering lead with Google's open-source security team, Google's interest in improving Linux security stems from its role as a producer of open-source software.
"Our team is responsible for making it easy for Google to securely consume open-source software that we take in as our dependencies to power pretty much all of our infrastructures," he said. "Also, to make it easy for Google engineers to securely produce and deliver open-source software in things like Chrome, Android and cloud -- all over Google."
The two Linux security developers that Google is funding are seasoned Linux kernel maintainers. This is par for the course, Lorenc said, because Google's approach to starting security initiatives within existing software projects is to work with people who are already on board as active maintainers.
PHP programming language developer and maintainer Nikita Popov said that two malicious commits were added to the php-src repository in both his name and that of PHP creator Rasmus Lerdorf. Officials said they do not yet know how exactly this happened.
But it appears to be a compromise of the git.php.net server rather than a compromise of an individual git account, according to Nikita Popov, senior software engineer at JetBrains and member of PHP Group. As a result, git.php.net server officials decided that maintaining their own git infrastructure is an unnecessary security risk and will discontinue the git.php.net server.
Instead, the repositories on GitHub, which were previously only mirrors, will become Canonical. This means that changes should be pushed directly to GitHub rather than to git.php.net.
"The backdoor hack into the PHP source code is nothing new. But the mitigation at play underscores how open source and the open-source community can detect and prevent chaos. Not stop it, but perhaps lessen its impact," Lorenc told LinuxInsider.
The PHP example is a good one given that PHP runs 80 percent of the Internet, he added, noting that Google is built on Linux, and security starts at the kernel.
"Any improvements we make here directly benefit our users," he said.
"The reality is this incident could have had far-reaching consequences, but the PHP team did a great job in detecting this early and preventing the compromised code from reaching end users," credited Lorenc.
His hope is that this incident reminds everyone that the security of our source control and build systems are as important as our production environments. Numbers do not lie.
"Supporting the maintainers who are already working to improve the security of open source is a great first step," agreed Lorenc. "But we need to get on the offensive and start building technology that can prevent entire classes of issues."
A great example here is programming with memory-safe languages and tooling, he added. Fuzzing has gotten great at finding bugs, but coders cannot fix them all. New modes of development can prevent those bugs from ever being introduced, he continued.
The biggest challenge Big Tech faces in achieving these security goals is not losing the security war with open-source software.
That has rightfully garnered the industry's attention. But solutions require consensus about the challenges and cooperation in the execution.
"The problem is complex, and there are many facets to cover: supply chain, dependency management, identity, and build pipelines for starters, added Lorenc.
"The security of open-source software has rightfully garnered the industry's attention, but solutions require consensus about the challenges and cooperation in the execution," he urged.
Google's action comes on the heels of rising threats to Linux that unfolded in the last year, as hackers pivot to new strategies like writing malware strains in the Go programming language.
The spread rate of malware is staggering. Infected code incidents made a 500 percent spike in the last year. That represents a 2,000 percent increase since 2017, according to Google.
This spike is no doubt because Go allows hackers to be versatile and target Windows, Mac, and Linux from the same codebase. Adding to this, 2020 alone saw a 40 percent increase in Linux-related malware families.
We can speculate all we want on the number of factors driving this shift, such as accelerated cloud adoption. But there is no denying the existence of a significant marketplace gap.
Capsule8 is a firm that specializes in securing Linux-based production environments. It's chief scientist, vice president of product strategy Brandon Edwards, does not expect this trend in Linux vulnerabilities to die down.
"I know that they have been dedicated towards Linux kernel security at the developer level. And, you know that can obviously be a challenging effort because getting review and approval for code to go into the kernel always brings on a fight it seems," Edwards told LinuxInsider.
All About Kernel Security
Code writers have two modes of operation, explained Edwards. One is integrity mode where you can still collect certain data points from the kernel that might be useful for visibility.The second is confidentiality mode where you cannot even do that. The idea is that in integrity mode you cannot modify any kernel code. But you can still observe behaviors in the kernel.
Code visibility is a crucial stem for reviewing open-source coding. How Google plans to carry out the intervention is a major concern for Edwards.
"My fear is that when Google draws that sort of line. If its software coders remove visibility, they are actually going to hurt security more," warned Edwards.
That is especially if it is under the pretense or assumption that whatever preventative measure they are putting in place means they no longer need visibility, he reasoned.
Security mitigation can almost always be bypassed. Usually, it is bypassed in a way that can be generalized. The mitigation is never an obstacle that an adversary has to bypass at once, according to Edwards.
"But if you have solid visibility or observability, it means that the attacker must take into consideration how every artifact or side effect of the movement might reveal their presence," he detailed.
So, visibility should enable observing the same accident mitigation as intended to prevent an accident. At the same time, coders must be mindful of the biases that go into creating the mitigation, he cautioned.
"You will definitely have biases when you create observability. But those biases still lend themselves towards observing things that you did not know were going to happen. That can be used for security's sake," Edwards said.
Google's Connection to Open Source
Google and The Linux Foundation recently announced that Google will fund two full-time Linux security developers to focus entirely on securing the kernel.This type of support is not surprising for the Google conglomerate. Google has joined efforts to fund Linux security developers for open-source projects, according to Linux officials. For instance, in early February Google donated $350,000 to the Python Software Foundation to support projects focused on improving supply-chain security.
That organization is also a founding member of the Open Source Security Foundation. OSSF is a Linux Foundation project dedicated to improving open-source software security.
According to Dan Lorenc, the engineering lead with Google's open-source security team, Google's interest in improving Linux security stems from its role as a producer of open-source software.
"Our team is responsible for making it easy for Google to securely consume open-source software that we take in as our dependencies to power pretty much all of our infrastructures," he said. "Also, to make it easy for Google engineers to securely produce and deliver open-source software in things like Chrome, Android and cloud -- all over Google."
The two Linux security developers that Google is funding are seasoned Linux kernel maintainers. This is par for the course, Lorenc said, because Google's approach to starting security initiatives within existing software projects is to work with people who are already on board as active maintainers.
What Happened
The official PHP Git server was compromised by a seasoned attacker trying to plant malware in the code base of the PHP project. The two malicious commit attempts were pushed by a hacker using the names and credentials of the repository.PHP programming language developer and maintainer Nikita Popov said that two malicious commits were added to the php-src repository in both his name and that of PHP creator Rasmus Lerdorf. Officials said they do not yet know how exactly this happened.
But it appears to be a compromise of the git.php.net server rather than a compromise of an individual git account, according to Nikita Popov, senior software engineer at JetBrains and member of PHP Group. As a result, git.php.net server officials decided that maintaining their own git infrastructure is an unnecessary security risk and will discontinue the git.php.net server.
Instead, the repositories on GitHub, which were previously only mirrors, will become Canonical. This means that changes should be pushed directly to GitHub rather than to git.php.net.
"The backdoor hack into the PHP source code is nothing new. But the mitigation at play underscores how open source and the open-source community can detect and prevent chaos. Not stop it, but perhaps lessen its impact," Lorenc told LinuxInsider.
The PHP example is a good one given that PHP runs 80 percent of the Internet, he added, noting that Google is built on Linux, and security starts at the kernel.
"Any improvements we make here directly benefit our users," he said.
Potential for Critical Consequences
Compromises like this show that the attack surface of our software supply chain is much broader than it appears. Components like build systems, source code management tools, and artifact repositories all need to be treated as critical production environments, because they are, urged Lorenc."The reality is this incident could have had far-reaching consequences, but the PHP team did a great job in detecting this early and preventing the compromised code from reaching end users," credited Lorenc.
His hope is that this incident reminds everyone that the security of our source control and build systems are as important as our production environments. Numbers do not lie.
No Easy Fix
Google and other big tech companies can make or break the open-source communities in terms of their support, sometimes financial. The goal is to make the Linux environment more secure."Supporting the maintainers who are already working to improve the security of open source is a great first step," agreed Lorenc. "But we need to get on the offensive and start building technology that can prevent entire classes of issues."
A great example here is programming with memory-safe languages and tooling, he added. Fuzzing has gotten great at finding bugs, but coders cannot fix them all. New modes of development can prevent those bugs from ever being introduced, he continued.
The biggest challenge Big Tech faces in achieving these security goals is not losing the security war with open-source software.
That has rightfully garnered the industry's attention. But solutions require consensus about the challenges and cooperation in the execution.
"The problem is complex, and there are many facets to cover: supply chain, dependency management, identity, and build pipelines for starters, added Lorenc.
Attention Now Where It Belongs
DevOps automation for open-source governance firm Sonatype estimated that at least 80 percent of the code in modern systems is open source. But it has not gotten anywhere close to that level of attention or resources from a security perspective, noted Lorenc. The biggest challenge we face as an industry is prioritizing the security of open-source software."The security of open-source software has rightfully garnered the industry's attention, but solutions require consensus about the challenges and cooperation in the execution," he urged.