Web application security training in 2015?
Our response is yes. Our application assessment course constantly changes. We look at the thousands of assessments that we perform for our customers and take those vulnerabilities discovered, new architectures and designs and try and build practical exploitation scenarios for our students. We love breaking the web, the cloud, 'the box that's hosted somewhere you can't recall but just works', as there's always new approaches and methods one can take to own the application layer.
Last month I discovered a vulnerability in Redhat's OpenStack Platform. What was cool about this vulnerability is that it's not a new class of vulnerability but when deployed in an organisation, it allows an authenticated user the ability to read files on the filesystem with the permissions of the web server. Owning organisations is all about exploiting flaws and chaining them together to achieve the end goal.
We want to teach you the same process: from learning how to own the application layer whilst having fun doing it at BlackHat Asia - Singapore.
During the course we will have a view on:
Come and join us! It will be fun :)
A few days ago, during one of those nights with the baby crying at 2:00 am and the only thing you can do is to read emails, I realised that Gmail shows the content of compressed files when reading them in Google Docs. As often is the case at SensePost, the "think evil (tm)" came to me and I started to ponder the possibilities of injecting HTML inside the file listing. The idea is actually rather simple. Looking at the file format of a .zip file we see the following:
Every file in the compressed file must have two entries; ZipFileRecord and ZipDirEntry. Both of these entries contain the filename, but only the first one contains the length of filename (it must match the actual length). Our first test case is obvious; if we could modify this name once the file was compressed, would Google sanitise it? Thankfully, the answer is, yes! (go Google!)
As you can see, Google shows the file name inside the compressed file but the tag is displayed with HTML entities. If we then try to see the contents of the file, Google responds by telling us it's not possible to read the content of the file (it's empty) and shows you the file "without formatting" after a few seconds:
Finally, the filename is shown but not sanitised:
Why this is possible?
Remember that the zip format has the name of the compressed files twice. Google uses the first one (ZipFileRecord) for displaying the file names, but in the vulnerable page it uses the second one (ZipDirEntry).
Possible attack vectors
Going back to the 'thinking evil (tm)' mindset, it is now possible to leave a "comprehensive" name in the first entry and inject the malicious payload in the second one. When I first discovered the possibility of doing this, I contacted Google, however, the XSS is in the googleusercontent.com domain, which Google's security team described as a "sandbox" domain (i.e. we aren't injecting into the DOM of google.com) and therefore not worthy of a bounty. Which I accept, if I had to prove usefulness this could be used as part of a simple social engineering attack, for example:
Leading the victim to my phishing site:
Which then proceeds to steals their Google session, or allows the attacker to use BeEF:
Granted, there are simpler ways of achieving the same result. I just wanted to demonstrate how you can use file meta-information for such an attack.
While doing some thinking on threat modelling I started examining what the usual drivers of security spend and controls are in an organisation. I've spent some time on multiple fronts, security management (been audited, had CIOs push for priorities), security auditing (followed workpapers and audit plans), pentesting (broke in however we could) and security consulting (tried to help people fix stuff) and even dabbled with trying to sell some security hardware. This has given me some insight (or at least an opinion) into how people have tried to justify security budgets, changes, and findings or how I tried to. This is a write up of what I believe these to be (caveat: this is my opinion). This is certainly not universalisable, i.e. it's possible to find unbiased highly experienced people, but they will still have to fight the tendencies their position puts on them. What I'd want you to take away from this is that we need to move away from using these drivers in isolation, and towards more holistic risk management techniques, of which I feel threat modelling is one (although this entry isn't about threat modelling).
The tick box monkeys themselves, they provide a useful function, and are so universally legislated and embedded in best practise, that everyone has a few decades of experience being on the giving or receiving end of a financial audit. The priorities audit reports seem to drive are:
But security vendors prioritisation of controls are driven by:
Every year around Black Hat Vegas/Pwn2Own/AddYourConfHere time a flurry of media reports hit the public and some people go into panic mode. I remember The DNS bug, where all that was needed was for people to apply a patch, but which, due to the publicity around it, garnered a significant amount of interest from people who it usually wouldn't, and probably shouldn't have cared so much. But many pentesters trade on this publicity; and some pentesting companies use this instead of a marketing budget. That's not their only, or primary, motivation, and in the end things get fixed, new techniques shared and the world a better place. The cynical view then is that some of the motivations for vulnerability researchers, and what they end up prioritising are:
Unfortunately, as human beings, our decisions are coloured by a bunch of things, which cause us to make decisions either influenced or defined by factors other than the reality we are faced with. A couple of those lead us to prioritising different security motives if decision making rests solely with one person:
The result of all of this is that different companies and people push vastly different agendas. To figure out a strategic approach to security in your organisation, you need some objective risk based measurement that will help you secure stuff in an order that mirrors the actual risk to your environment. While it's still a black art, I believe that Threat Modelling helps a lot here, a sufficiently comprehensive methodology that takes into account all of your infrastructure (or at least admits the existence of risk contributed by systems outside of a “most critical” list) and includes valid perspectives from above tries to provide an objective version of reality that isn't as vulnerable to the single biases described above.
A longish post, but this wasn't going to fit into 140 characters. This is an argument pertaining to security metrics, with a statement that using pure vulnerability count-based metrics to talk about an organisation's application (in)security is insufficient, and suggests an alternative approach. Comments welcome.
Apart from the two bookends (SOSS and DBIR), other metrics are also published.
From a testing perspective, WhiteHat releases perhaps the most well-known set of metrics for appsec bugs, and in years gone by, Corsaire released statistics covering their customers. Also in 2008, WASC undertook a project to provide metrics with data sourced from a number of companies, however this too has not seen recent activity (last edit on the site was over a year ago). WhiteHat's metrics measure the number of serious vulnerabilities in each site (High, Critical, Urgent) and then slice and dice this based on the vulnerability's classification, the organisation's size, and the vertical within which they lie. WhiteHat is also in the fairly unique position of being able to record remediation times with a higher granularity than appsec firms that engage with customers through projects rather than service contracts. Corsaire's approach was slightly different; they recorded metrics in terms of the classification of the vulnerability, its impact and the year within which the issue was found. Their report contained similar metrics to the WhiteHat report (e.g. % of apps with XSS), but the inclusion of data from multiple years permitted them to extract trends from their data. (No doubt WhiteHat have trending data, however in the last report it was absent). Lastly, WASC's approach is very similar to WhiteHat's, in that a point in time is selected and vulnerability counts according to impact and classification are provided for that point.
Essentially, each of these approaches uses a base metric of vulnerability tallies, which are then viewed from different angles (classification, time-series, impact). While the metrics are collected per-application, they are easily aggregated into organisations.
In the extreme edges of ideal metrics, the ability to factor in chains of vulnerabilities that individually present little risk, but combined is greater than the sum of the parts, would be fantastic. This aspect is ignored by most (including us), as a fruitful path isn't clear.
One could just as easily claim that absolute bug counts are irrelevant and that they need to be relative to some other scale; commonly the number of applications an organisation has. However in this case, if the metrics don't provide enough granularity to accurately position your organisation with respect to others that you actually care about, then they're worthless to you in decision making. What drives many of our customers is not where they stand in relation to every other organisation, but specifically their peers and competitors. It's slightly ironic that oftentimes the more metrics released, the less applicable they are to individual companies. As a bank, knowing you're in the top 10% of a sample of banking organisations means something; when you're in the highest 10% of a survey that includes WebGoat clones, the results are much less clear.
In Seven Myths About Information Security Metrics, Dr Hinson raises a number of interesting points about security metrics. They're mostly applicable to security awareness, however they also carry across into other security activities. At least two serve my selfish needs, so I'll quote them here:
Myth 1: Metrics must be “objective” and “tangible”There is a subtle but important distinction between measuring subjective factors and measuring subjectively. It is relatively easy to measure “tangible” or objective things (the number of virus incidents, or the number of people trained). This normally gives a huge bias towards such metrics in most measurement systems, and a bias against measuring intangible things (such as level of security awareness). In fact, “intangible” or subjective things can be measured objectively, but we need to be reasonably smart about it (e.g., by using interviews,surveys and audits). Given the intangible nature of security awareness, it is definitely worth putting effort into the measurement of subjective factors, rather than relying entirely on easy-to-measure but largely irrelevant objective factors. [G Hinson]
Myth 3: We need absolute measurementsFor some unfathomable reason, people often assume we need “absolute measures”—height in meters, weight in pounds, etc. This is nonsense!
If I line up the people in your department against a wall, I can easily tell who is tallest, with no rulers in sight. This yet again leads to an unnecessary bias in many measurement systems. In fact, relative values are often more useful than absolute scales, especially to drive improvement. Consider this for instance: “Tell me, on an (arbitrary) scale from one to ten, how security aware are the people in your department are? OK, I'll be back next month to ask you the same question!” We need not define the scale formally, as long as the person being asked (a) has his own mental model of the processes and (b) appreciates the need to improve them. We needn't even worry about minor variations in the scoring scale from month to month, as long as our objective of promoting improvement is met. Benchmarking and best practice transfer are good examples of this kind of thinking. “I don't expect us to be perfect, but I'd like us to be at least as good as standard X or company Y. [G Hinson]
While he writes from the view of an organisation trying to decide whether their security awareness program is yielding dividends, the core statements are applicable for organisations seeking to determine the efficacy of their software security program. I'm particularly drawn by two points: the first is that intangibles are as useful as concrete metrics, and the second is that absolute measurements aren't necessary, comparative ordering is sometimes enough.
Measuring effort, or attacker cost, is not new to security but it's mostly done indirectly through the sale of exploits (e.g. iDefence, ZDI). Even here, effort is not directly related to the purchase price, which is also influenced by other factors such as the number of deployed targets etc. In any case, for custom applications that testers are mostly presented with, such public sources should be of little help (if your testers are submitting findings to ZDI, you have bigger problems). Every now and then, an exploit dev team will mention how long it took them to write an exploit for some weird Windows bug; these are always interesting data points, but are not specific enough for customers and the sample size is low.
Ideally, any measure of an attacker's cost can take into account both time and their exclusivity (or experience), however in practice this will be tough to gather from your testers. One could base it on their hourly rate, if your testing company differentiates between resources. In cases where they don't, or you're seeking to keep the metric simple, then another estimate for effort is the number of days spent on testing.
Returning to our sample companies, if the 5 vulnerabilities exposed in the Visigoth's each required, on average, a single day to find, while the Ostrogoth's 20 bugs average 5 days each, then the effort required by an attacker is minimised by choosing to target the Visigoths. In other words, one might argue that the Visigoths are more at risk than the Ostrogoths.
With this base metric, it's then possible to capture historical assessment data and provide both internal-looking metrics for an organisation as well as comparative metrics, if the testing company is also employed by your competitors. Internal metrics are the usual kinds (impact, classification, time-series), but the comparison option is very interesting. We're in the fortunate position of working with many top companies locally, and are able to compare competitors using this metric as a base. The actual ranking formulae is largely unimportant here. Naturally, data must be anonymised so as to protect names; one could provide the customer with their rank only. In this way, the customer has an independent notion of how their security activities rate against their peers without embarrassing the peers.
Inverting the findings-per-day metric provide the average number of days to find a particular class of vulnerability, or impact level. That is, if a client averages 0.7 High or Critical findings per testing day, then on average it takes us 1.4 days of testing to find an issue of great concern, which is an easy way of expressing the base metric.
As mentioned above, a minimum number of assessments would be needed before the metric is reliable; this is a hint at the deeper problems that randomly selected project days are not independent. An analyst stuck on a 4 week project is focused on a very small part of the broader organisation's application landscape. We counter this bias by including as many projects of the same type as possible.
This metric would also be very useful to include in each subsequent report for the customer, with every report containing an evaluation against their longterm vulnerability averages.
As mentioned above, a key test for metrics is where they support decision making, and the feedback from the client was positive in this regard.
This idea is still being fleshed out. If you're aware of previous work in this regard or have suggestions on how to improve it (even abandon it) please get in contact.
Oh, and if you've read this far and are looking for training, we're at BH in August.
Since joining SensePost I've had a chance to get down and dirty with the threat modeling tool. The original principle behind the tool, first released in 2007 at CSI NetSec, was to throw out existing threat modeling techniques (it's really attack-focused risk) and start from scratch. It's a good idea and the SensePost approach fits nicely between the heavily formalised models like Octave and the quick-n-dirty's like attack trees. It allows fairly simple modeling of the organisation/system to quickly produce an exponentially larger list of possible risks and rank them.
We've had some time and a few bits of practical work to enhance the tool and our thinking about it. At first, I thought it would need an overhaul, mostly because I didn't like the terminology (hat tip to Mr Bejtlich). But, in testament to Charl's original thinking & the flexibility of the tool, no significant changes to the code were required. We're happy to announce version 2.1 is now available at our new tools page. In addition, much of our exploration of other threat modeling techniques was converted into a workshop of which the slides are available (approx 30MB).
The majority of the changes were in the equation. The discussion below will give you a good idea of how you can play with the equation to fundamentally change how the tool works.
There are 5 values you can play with in the equation:
In English that translates to: The risk is equal to; the average of the impact of the attack and it's likelihood, combined with the value of the asset (exposed through a particular interface), and reduced by the trust of the user performing the attack and the location they are performing it from.
We felt there were two problems with this equation:
Once again in English: The risk of an attack is; the likelihood of the attack reduced by the average of both the trust in the user & location, combined with, the value of the asset reduced by the potential impact of the attack (value at risk). (The 0.2 & 2.5 are just to make it fit the scales. Specifically, the 0.2 is because the scale of the entities is 1-5 and we're looking to make a percentage, and the 2.5 is to fit the 0-25 scale on the final graph.)
The key change which breaks backward compatibility here is that impact now becomes a moderator on value. i.e. the impact of an attack determines how much of the asset's value is exposed.
The way things are now modeled, interfaces represent the value of a system. For the most part, all a system's interfaces should have the same value, because as we often see, even minor interfaces that expose limited functionality can often be abused for a full compromise. However, the actual attack (called threats in the tool) determined how much of that value is exposed. For example, a worst-case XSS is (depending on the system of course) probably going to expose less of the system's value than a malicious sysadmin publicly pwning it (once again, dependent on the system and controls in place).
Unfortunately, there's still no provable way to perform threat modeling, but we feel we can go quite far in providing a quick and useful way of enumerating and prioritising attacks (and hence defenses) across complex system.
In a future blog post, I hope to cover some of the really cool scenario planning the tool can let you do, and the pretty graphs it gave us an excuse to justify budgets with.
[ Credit to the Online LaTeX Equation Editor for the formulas, although if you'd like to copy paste the formula described above into the tool, here's an ascii version:
( ( ( lik * ( ( ( (6 - usr) + (6 - loc) ) / 2 ) * 0.2 ) ) + ( int * ( imp * 0.2 ) ) ) * 2.5 )