API3:2019 Excessive Data Exposure

In the previous post we covered API2:2019 Broken User Authentication, which was the second post in this series. If you want to start from the beginning, go to the first post, API1:2019 Broken Object Level Authorization.

#RSAC 2022

Excessive data exposure is something that web applications can face, not just APIs. That said, because web-based APIs are basically services on the web, they can be abused even more easily to exfiltrate sensitive data than a regular web app. It’s easy for an attacker to find APIs (just connect to any web app or mobile app using a web proxy and see the API calls for yourself!), to call them, and then look at the responses to see if anything being sent looks potentially sensitive. For instance, if the data in a field passed back is named “password”, “sin” or “secret”, you’re most likely onto something.

Using a web proxy to watch the API calls go back and forth is sometimes called “sniffing”, but no matter what you call it, it’s easy to do! Anyone with the tiniest amount of web-app hacking training can do this on day one. This means this threat is prevalent (happens all the time) and very dangerous (because unsophisticated attackers can easily execute it).

Some APIs are *supposed* to return sensitive data. This vulnerability is when sensitive data is exposed to someone it should not be (for instance, someone who is not a valid user, seeing another user’s sensitive data, or for whom that specific data should not be shown due to their role within the system). Since whether data is sensitive in nature is not obvious to automated testing tools, it can be a bit more difficult to identify than other types of vulnerabilities.

* Note: occasionally the vulnerability rears its head via poorly-generated and/or overly-populated responses. For instance, the API delivers the entire table worth of data, which includes sensitive information, but then the client-side front-end sifts through it and only reveals the non-sensitive/appropriate data to the end user. Unfortunately, if the API call is not encrypted in transit, this means a malicious actor could see all of the data if they were sniffing the API at the time.

How do we avoid this?

Let’s look at some great advice from the project team (I may have added a bit onto their list):

  • Never rely on the client side to filter sensitive data. By this we mean, only return the data you need to return! Don’t send a ton of stuff you do not need to, then let the GUI/front end decide what to show the user. Make these important decisions on the server.
  • Classifying then label all your data. If you know immediately when you look at something that it is sensitive, it’s automatic to treat it in a certain way. Educate your developers and other areas of IT on how to classify data, and to ask the security team if they aren’t sure.
  • In the design of your API, add user stories and/or threat models around this potential vulnerability. Making protecting sensitive data part of your design.
  • Review the responses from the API to make sure they contain only legitimate data, data that the specific user (or users with that role inside your system) are allowed to access.
  • Back-end engineers should always ask themselves “who is the consumer of the data?” before exposing a new API endpoint. Or better yet, perform threat modelling on your data flows, THEN design.
  • Avoid using generic methods such as to_json() and to_string(). Instead, cherry-pick specific properties you really want to return. You do not need to return everything. In fact, it’s better for your cloud bills to return only what you need, even if it requires a bit more programming.
  • Classify sensitive and personally identifiable information (PII) that your application stores and works with, reviewing all API calls returning such information to see if these responses pose a security issue.
  • Implement a schema-based response validation mechanism as an extra layer of security. As part of this mechanism define and enforce data returned by all API methods, including errors.
  • Perform strict linting on your API definition file, to ensure you have input validation built-in, by default, for every variable.

In the next blog post we will be talking about API4:2019 Lack of Resources & Rate Limiting!

The OWASP API Security Top Ten: API1:2019 Broken Object Level Authorization

The following blog series will be on the OWASP API Security Top Ten, 2019 version. The new version is coming out shortly, and we will add even more to this series when the time comes.

What is “The OWASP Top Ten”?

OWASP is an international non-profit, with a huge community, over 100 active open source projects, and over 300 in-person chapters around the world. The thing they are most well known for is “The OWASP Top Ten”, an awareness document created to teach the world about the most dangerous risk to web applications. Over the years, various project teams have created more “Top Ten” documents for Serverless, IoT, and APIs!

You can visit the project page here, and see the PDF from the project team here. Thank you very much to the project team who worked so hard on this, especially the leaders: Erez Yalon, Inon Shkedy and Paulo Silva. We Hack Purple applaud your efforts to help make the internet a safer place!

Now down to business! Let’s dive into #1 on this infamous list.

"Shifting Left is not enough." Tanya Janca presenting at OWASP Global AppSec, Dublin, Ireland, Feb 2023
“Shifting Left is not enough.” Tanya Janca, presenting at OWASP Global AppSec, Dublin, Ireland, Feb 2023

API1:2019 Broken Object Level Authorization

Back in the day, PenTesters used to be able to “minus 1” from any USER ID located in the URL parameters and they could quite often see the previous user’s account data. The URL parameters would have “userid=622”, the tester switches it to “userid=621”, and suddenly they were reporting a vulnerability. Fast forward to today, and although most Web Apps don’t fall for this trick anymore, unfortunately APIs are often in a state where changing the parameters a bit can fetch all sorts of data that they should not be able to access. It’s very easy using a intercept web proxy to sniff the API calls, see the parameters, and change the value of one of them. This vulnerability is the most-often exploited and most-damaging of everything listed in the API Security Top Ten document, which is why it’s #1.

But how bad is it really?

It’s pretty bad! It can result in sensitive data exposure (confidentiality broken), changed data (integrity ruined), deleted data (no availability) and even complete account takeover. No one wants this for their users.

How does this happen?

When the IT world moved from monolithic applications into the world of micro service architecture, a few things were lost in the translation. Monolithic applications have been performing session management for a very long time, meaning it knows what state it is in, and thus they can see when an attacker tries to access things it should not. Session management (keeping track of which user is logged in, and that access they are allowed) isn’t native to RESTful APIs. REST APIs are *supposed* to be stateless. And that creates a problem for us since we need to keep track of this in order to avoid this vulnerability.

Every time any user asks for access to anything, we should validate that they are 1) still the user we think they are (we validate the session) and 2) that they are allowed access or have permission to see what they want to see or do what they want to do. Every single time we must check. That one time we forget, is when PenTesters and malicious actors alike do a happy dance, because we have left open a hole in our armour.

How do we avoid this?

As you might have guessed, the OWASP API Security Top Ten Project Team has some thoughts on the matter! We’ve added some thoughts to each item below, please see the original document for a more-succinct description.

  • Implement a proper authorization mechanism that relies on the user policies and hierarchy. There are tools/products that you can buy that perform these functions for you. This is one of those programming things that is difficult to get right, and most security folks recommend you buy a well-trusted solution over attempting to build your own, due to complexity and cost.
  • Use an authorization mechanism to check if the logged-in user has access to perform the requested action on the record in every function that uses an input from the client to access a record in the database. Check every single time.
  • Use random and unpredictable values as GUIDs for records’ IDs. Do not use incremental, guessable, numbers. If a GUID or other record ID is sent to your system that your never issued, trigger an alert and block that IP address immediately, because your API is under attack.
  • Write tests to evaluate the authorization mechanism. Do not deploy vulnerable changes that fail the tests. Ensure the entire project team understands that failing any of these tests blocks all releases to production until they are fixed. You must pass these tests to get to prod.

Helpful links from the OWASP API Security Top Ten Project Team!

In the next article we are going to discuss API2:2019 Broken User Authentication!

From the OWASP API Security Top Ten!