Facebook open-sources a static analyzer for Python code

Need a tool to check your Python-based applications for security issues? Facebook has open-sourced Pysa (Python Static Analyzer), a tool that looks at how data flows through the code and helps developers prevent data flowing into places it shouldn’t.

Python Static Analyzer

How the Python Static Analyzer works

Pysa is a security-focused tool built on top of Pyre, Facebook’s performant type checker for Python.

“Pysa tracks flows of data through a program. The user defines sources (places where important data originates) as well as sinks (places where the data from the source shouldn’t end up),” Facebook security engineer Graham Bleaney and software engineer Sinan Cepel explained.

“Pysa performs iterative rounds of analysis to build summaries to determine which functions return data from a source and which functions have parameters that eventually reach a sink. If Pysa finds that a source eventually connects to a sink, it reports an issue.”

It’s used internally by Facebook to check the (Python) code that powers Instagram’s servers, and do so quickly. It’s used to check developer’s proposed code change for security and privacy issues and to prevent them being introduced in the codebase, as well as to detect existing issues in a codebase.

The found issues are flagged and, depending on their type, the report is send either to the developer or to security engineers to check it out.

You can get Pysa from here and you can use a number of already developed definitions to help it find security issues.

“Because we use open source Python server frameworks such as Django and Tornado for our own products, Pysa can start finding security issues in projects using these frameworks from the first run. Using Pysa for frameworks we don’t already have coverage for is generally as simple as adding a few lines of configuration to tell Pysa where data enters the server,” the two engineers added.

The tool’s limitations and stumbling blocks

Pysa can’t detect all security or privacy issues, just data flow–related security issues. What’s more, it can’t detect all data flow–related issues because the Python programming language is very flexible and dynamic (allows code imports, change function call actions, etc.)

Finally, those who use it have make a choice about how many false positives and negatives they will tolerate.

“Because of the importance of catching security issues, we built Pysa to avoid false negatives and catch as many issues as possible. Reducing false negatives, however, may require trade-offs that increase false positives. Too many false positives could in turn cause alert fatigue and risk real issues being missed in the noise,” the engineers explained.

The number of false positives can reduced by using sanitizers and manually added and automatic features.

Automate manual security, risk, and compliance processes in software development

The future of business relies on being digital – but all software deployed needs to be secure and protect privacy. Yet, responsible cybersecurity gets in the way of what any company really wants to do: innovate fast, stay ahead of the competition, and wow customers!

SD Elements

In this podcast recorded at RSA Conference 2020, we’re joined by Ehsan Foroughi, Vice President of Products from Security Compass, an application security expert with 13+ years of management and technical experience in security research. He talks about a way of building software so that cybersecurity issues all but disappear, letting companies focus on what they do best.

Good morning. Today we have with us Ehsan Foroughi, Vice President of Products from Security Compass. We’ll be focusing on what Security Compass calls the Development Devil’s Choice and what’s being done about it. Ehsan tell me a little about yourself.

A brief introduction: I started my career in cybersecurity around 15 years ago as a researcher doing malware analysis and reverse engineering. Around eight years ago I joined an up and coming company named Security Compass. Security Compass has been around for 14 years or so, and it started as a boutique consulting firm focusing on helping developers code securely and push out the products.

When I joined SD Elements, which is the software platform and the flagship of the product was under development. I’ve worn many hats during that time. I’ve been a product manager, I’ve been a researcher, and now I own the R&D umbrella effort for the company.

Thank you. Can you tell me a little bit about Security Compass’ mission and vision?

The company’s vision is a world where people can trust technology and the way to get there is to help companies develop secure software without slowing down the business.

Here’s our first big question. The primary goals of most companies are to innovate fast, stay ahead of the competition and wow customers. Does responsible cybersecurity get in the way of that?

It certainly feels that way. Every industry nowadays relies on software to be competitive and generate revenue. Software is becoming a competitive advantage and it drives the enterprise value. As digital products are becoming critical, you’re seeing a lot of companies consider security as a first-class citizen in their DevOps effort, and they are calling it DevSecOps these days.

The problem is that when you dig into the detail, they’re mostly relying on reactive processes such as scanning and testing, which find the problems too late. By that time, they face a hard choice of whether to stop everything and go back to fix, or accept a lot of risk and move forward. We call this fast and risky development. It gets the software out to production fast, by eliminating the upfront processes, but it’s a ticking time bomb for the company and the brand. I wouldn’t want to be sitting on that.

Most companies know that they need proactive security like threat modeling, risk assessments, security training. That’s a responsible thing to do, but it’s slow and it gets in the way of the speed to the market. We call this slow and safe development. It might be safe by the way of security compliance, but it opens up to competitive risk. This is what we call the Development Devil’s Choice. Every company that relies on it has two bad choices, fast and risky or slow and safe.

Interesting. Do you believe the situation will improve over time as companies get more experienced in dealing with this dilemma?

I think it’s going to get worse over time. There are more regulations coming. A couple of years ago GDPR came up, and then it’s California Consumer Privacy Act, and then the new PCI regulations.

The technology is also getting more complex every day. We have Dockers and Kubernetes, there’s cloud identity management and the shelf life of the technology is reducing. We no longer have the 10 years end of life Linux systems that we can rely on.

SD Elements

So, how are companies dealing with this problem in the age of agile development?

I’m tempted to say that rather than dealing with it, they’re struggling with it. Most agile teams define their work by the way of user stories. On rare occasions, the teams take the time to compile the requirements and bake for security, and bake it into their stories. But in the majority of the cases, the security requirements are unknown and implicit. This means that they rely on people’s good judgment, and they rely on expertise. This expertise is hard to find and we do have a skill shortage in the security space. When you find them, they’re also very expensive.

How do these teams integrate security compliance into their workflow?

In our experience, most agile teams have been relying on testing and scanning to find the issues, and then that means that they have a challenge. When they uncover the issue, they have to figure out if they should go back and fix or they take the risk and move forward. Either way, it’s a lot of patchwork. When the software gets shipped, everybody crosses their fingers and hopes that everything went well. This usually leads to a lot of silos. Security becomes oppositional to development.

What happens when the silos occur? Are teams wasting their effort? Reworking software?

It adds a lot of time and anxiety. The work ends up being manual, expensive and painfully deliberate. The security compliance side of the business gets frustrated with the development, they find inconsistencies against each other and it just becomes a challenge.

No matter how companies develop software, their steps for security and compliance are likely not very accurate. That means that the management also has no visibility into what’s going on. There are lots of tools and processes today to check on the software that is being built, but usually they don’t help make it secure from the start. They usually point out to the problems and they show how it was built wrong.

Finding that out is a challenge because it exacerbates this dilemma of development versus security. It’s like being told that you didn’t need heart surgery if you ate healthy food for the past 10 years. It’s a bit too late and not particularly helpful.

I’m hearing you describe a serious problem that’s haunting company leaders. It seems they have two pretty bad options for development, fast and risky or slow and safe. Is that it? Are companies doom to choose between these two?

Well, there’s hope. There is a third option emerging. You don’t need to be fast and risky or slow and safe. The option is to be nearly as fast, without slowing down and being secure at the same time. We call it the balanced development. It’s similar to how the Waze app knows where you’re driving and tells you specifically at each step where you should be going and where you should be turning.

The key is to bring security left in the cycle, circle rapid around the development and make sure that it’s done in tandem. Testing and scanning should not find anything by the end of the cycle if this is done right. These systems mostly leverage automation to balance the development effort between the fast and risky and the slow and safe.

SD Elements

Ehsan, can you tell us more about these systems? How do they work and how do they support the jobs of security teams?

Well, automation is the key. It starts by capturing the knowledge of the experts into a knowledge base, and automating so that the system understands what you’re working on, what you’re doing, and delivering the actions that you need to take to bake security in right at the time you need it.

It constantly also updates the knowledge base to stay on top of the regulation changes, technology changes, and during development the teams are advised of the latest changes. When the project is finished, the system is almost done with the security and compliance actions and activities, and all of it is also documented so that the management can see what risk they are taking on.

Thank you very much for the insight and for the thoughtful discussion. What advice would you give company leaders as they start to tackle these issues?

Well, I have a couple of advice, mostly based on the companies we have been working with. I would say, stay pragmatic and balanced. Focus on getting 80% fast and 80% secure. Don’t get bogged down. Number two, I would say educate your organization, especially the executives. Executive buy-in is very important. Without that you can’t change the process and you can’t do it in silos from within one small team. You have to get people’s buy-in and support.

The next one is investing in automating the balanced approach. This investment is sometimes hard, but the earlier you do it, the better. I see a lot of companies bugged down by investing in the smaller, easier projects like updating and refreshing their scanning practice. It usually pays off to go to the heart of the problem and invest in that, because all of your future investments are more optimized.

I find it also useful when working with the developers, to always start with why? Why are you doing this? Why are you asking them to follow a certain process? If they understand the business value of it, they’ll be more cooperative with you.

And finally, try our system. We have a platform called SD Elements that enables you to automate your balanced development.

If anyone’s listening and interested in connecting with you or Security Compass, how can they find you?

Well, you should check out our website at www.securitycompass.com. We’d love to prove our motto to you: Go fast and stay safe. Thanks for joining us.

Microsoft Application Inspector: Check open source components for unwanted features

Want to know what’s in an open source software component before you use it? Microsoft Application Inspector will tell you what it does and spots potentially unwanted features – or backdoors.

Microsoft Application Inspector

About Microsoft Application Inspector

“At Microsoft, our software engineers use open source software to provide our customers high-quality software and services. Recognizing the inherent risks in trusting open source software, we created a source code analyzer called Microsoft Application Inspector to identify ‘interesting’ features and metadata, like the use of cryptography, connecting to a remote entity, and the platforms it runs on,” Guy Acosta and Michael Scovetta, security program managers at Customer Security and Trust, Microsoft, explained the Inspector’s genesis.

The Microsoft Application Inspector:

  • Is a client .NET Core-based tool that can be run from a command line in Windows, Linux or MacOS
  • Uses static analysis and a customizable JSON-based rules engine to analyze the target code. Users can add/edit/remove default rule patterns (there are over 500) as well as add their own rules
  • Is able to analyze code written in a variety of programming languages

Once the tool does its work, it generates a HTML “report” that shows the features, project summary and meta-data detected (see image above). JSON and TEXT output format options are supported for those who prefer them.

Each discovered feature can be broken down into more specific categories and receives a confidence indicator. Users can see for themselves the source code snippets that produced each “discovery”.

Microsoft Application Inspector

Use cases

“Basically, we created Application Inspector to help us identify risky third party software components based on their specific features, but the tool is helpful in many non-security contexts as well,” the developers explained.

“For instance, it can also help identify feature deltas or changes between versions which can be critical for detecting injection of backdoors. Well constructed and hidden backdoors can go undetected by a tool that is only looking for poor security programming practices because it doesn’t look at context at a feature level.”

Nevertheless, the tool is not meant to replace security static analyzers or security code reviews, but to be used alongside them.

“Knowing what is in your software is the first step to making key choices about what actions are appropriate before allowing it to be deployed in your own or to customer environments. Our tool includes hundreds of default identifying patterns for detecting general features like frameworks used, file I/O, OS API’s as well as the ability to detect key security and privacy features of a component,” the developers concluded.

Microsoft Application Inspector is open source and available for download from GitHub.