From the team that won the world's first Soccer Hack Cup, we bring you the latest and the greatest in computer hacking training - SensePost Hacking By Numbers Extended Edition - a local course that combines two of the brand new courses we just finished presenting at Black Hat Las Vegas.

The training will be offered in Brooklyn Pretoria from 14 - 17 September 2010. Here's how it will work:

• 14 - 15 September: Cadet Edition
• 16 - 17 September: Bootcamp Edition

Ok ok ok, so Pretoria is not exactly Vegas, but the courses are fresh and updated and packed full of exciting new content, tools and techniques.

For more information visit our Extended Edition page, or drop us a note via training[at]sensepost[dot]com.

[Update: Disclosure and other points discussed in a little more detail here.]

Why memcached?

At BlackHat USA last year we spoke about attacking cloud systems, while the thinking was broadly applicable, we focused on specific providers (overview). This year, we continued in the same vein except we focused on a particular piece of software used in numerous large-scale application including many cloud services. In the realm of "software that enables cloud services", there appears to be a handful of "go to" applications that are consistently re-used, and it's curious that a security practitioner's perspective has not as yet been applied to them (disclaimer: I'm not aware of parallel work).

We choose to look at memcached, a "Free & open source, high-performance, distributed memory object caching system" ¹. It's not outwardly sexy from a security standpoint and it doesn't have a large and exposed codebase (total LOC is a smidge over 11k). However, what's of interest is the type of applications in which memcached is deployed. Memcached is most often used in web application to speed up page loads. Sites are almost² always dynamic and either have many clients (i.e. require horizontal scaling) or process piles of data (look to reduce processing time), or oftentimes both. This implies that the sites that use memcached contain more interesting info than simple static sites, and are an indicator of a potentially interesting site. Prominent users of memcached include LiveJournal (memcached was originally written by Brad Fitzpatrick for LJ), Wikipedia, Flickr, YouTube and Twitter.

I won't go into how memcached works, suffice it to say that since data tends to be read more often than written in common use cases the idea is to pre-render and store the finalised content inside the in-memory cache. When future requests ask for the page or data, it doesn't need to be regenerated but can be simply regurgitated from the cache. Their Wiki contains more background.

go-derper

We released go-derper, a tool for playing with memcached instances. It supports three basic modes of operations:

1. Fingerprinting memcacheds to determine interesting servers
2. Extracting a (user-limited) copy of the cache
3. Writing data into the cache

The tool has minor requirements: a recent Ruby and the memcache-client gem. What follows are basic use cases.

Fingerprinting

Let's assume you've scanned a hosting provider and found 239 potential targets using a basic .nse that hunts down open memcached instances³. You need to separate the wheat from the chaff and figure out which servers are potentially interesting; one way to do that is by extracting a bunch of metrics from each cache. Start small against one cache: insurrection:demo marco$ ruby go-derper.rb -f x.x.x.x [i] Scanning x.x.x.x x.x.x.x:11211 ============================== memcached 1.4.5 (1064) up 54:10:01:27, sys time Wed Aug 04 10:34:36 +0200 2010, utime=369388.17, stime=520925.98 Mem: Max 1024.00 MB, max item size = 1024.00 KB Network: curr conn 18, bytes read 44.69 TB, bytes written 695.93 GB Cache: get 514, set 93.41b, bytes stored 825.73 MB, curr item count 1.54m, total items 1.54m, total slabs 3 Stats capabilities: (stat) slabs settings items (set) (get)

44 terabytes read from the cache in 54 days with 1.5 million items stored? This cache is used quite frequently. There's an anomaly here in that the cache reports only 514 reads with 93 billion writes; however it's still worth exploring if only for the size.

We can run the same fingerprint scan against multiple hosts using

ruby go-derper.rb -f host1,host2,host3,...,hostn

or, if the hosts are in a file (one per line):

ruby go-derper.rb -F file_with_target_hosts

Output is either human-readable multiline (the default), or CSV. The latter helps for quickly rearranging and sorting the output to determine potential targets, and is enabled with the "-c" switch:

ruby go-derper.rb -c csv -f host1,host2,host3,...,hostn

Lastly, the monitor mode (-m) will loop forever while retrieving certain statistics and keep track of differences between iterations, in order to determine whether the cache appears to be in active use.

Mining

Once you've identified a potentially interesting target, it's time to mine that cache. The basic leach switch is "-l":

insurrection:demo marco$ ruby go-derper.rb -l -s x.x.x.x [w] No output directory specified, defaulting to ./output [w] No prefix supplied, using "run1"

This will extract data from the cache in the form of a key and its value, and save the value in a file under the "./output" directory by default (if this directory doesn't exist then the tool will exit so make sure it's present.) This means a separate file is created for every retrieved value. Output directories and file prefixes are adjustable with "-o" and "-r" respectively, however it's usually safe to leave these alone.

By default, go-derper fetches 10 keys per slab (see the memcached docs for a discussion on slabs; basically similar-sized entries are grouped together.) This default is intentionally low; on an actual assessment this could run into six figures. Use the "-K" switch to adjust:

ruby go-derper.rb -l -K 100 -s x.x.x.x

As mentioned, retrieved data is stored in the "./ouput" directory (or elsewhere if "-o" is used). Within this directory, each new run of the tool produces a set of files prefixed with "runN" in order to keep multiple runs separate. The files produced are:

• runN-index, an index file containing metadata about each entry retrieved
• runN-<md5>, a file containing the bytestream from a retrieved value

The mapping between key and file in which the value is stored occurs in the index file, which is useful in that potentially malicious data (keynames) aren't used when interacting with your local filesystem APIs.

At this point, there will (hopefully) be a large number of files in your output directory, which may contain useful info. Start grepping.

What we found with a bit of field experience was that mining large caches can take some time, and repeating grep gets quite boring. The tool permits you to supply your own set of regular expressions which will be applied to each retrieved value; matches are printed to the screen and this provides a scroll-by view of bits of data that may pique your interest (things like URLs, email addresses, session IDs, strings starting with "user", "pass" or "auth", cookies, IP addresses etc). The "-R" switch enables this feature and takes a file containing regexes as its sole argument:

ruby go-derper.rb -l -K 100 -R regexs.txt -s x.x.x.x

Over-writing

In this blog entry I don't cover the kinds of data we discovered (it'll be subject to a separate entry), however it may come to pass that you discover an interesting cache entry that you'd like to overwrite. Recall entries were stored in "./output" by default, with a prefix of "runN". If the interesting entry was stored in "output/run1-e94aae85bd3469d929727bee5009dddd", edit the file in whatever manner you see fit and save it to your local disk. Then, tell go-derper to write the entry back into the cache with:

ruby go-derper.rb -w output/run1-e94aae85bd3469d929727bee5009dddd

This syntax is simple since go-derper will figure out the target server and key from the run's index file.

And so?

Go-derper permits basic manipulations of a memcached instance. We haven't covered finding open instances or the kinds of data one may come across; these will be the subject of followup posts. Below are the slides from the talk, click through to SlideShare for the downloadable PDF. ¹ http://www.memcached.org

² We're hedging here, but we've not come across a static memcached site.

³ If so, you may be as surprised as we were in finding this many open instances.

Today at BlackHat USA 2010 we released a tool for manipulating memcached instances; we still need to write it up properly but here's a link to the tool for the moment.

tl;dr: if you find a memcached, you can dump the cache and manipulate entries in the cache.

As the need for online anonymity / privacy grew, the proxy industry flourished with many proxy owners generating passive incomes from their proxy networks.

Although 'proxy' is normally thought to imply some sort of daemonized application, such as Squid (or a SOCKS) daemon, the last couple of years have heralded in the age of CGI proxies and more commonly, their PHP variants.

These PHP proxies are extremely trivial to deploy and configure, especially since most hosting environments have PHP installed by default. When development of PHProxy (a popular PHP proxy) ceased, many devoted fans starting releasing their own customised PHProxy fixes and variants. In recent years, however, many proxy owners have gravitated towards Glype since it seemed to be well maintained (though the current status may be questionable).

While there have been tools created to portscan targets through SOCKS and HTTP proxies, I am not aware of any which reliably performed it through the current range of PHP proxies.

By default, Glype has few restrictions on what hosts / ports can be accessed through it and normally displays its cURL error messages as well. Using these apparent weaknesses, GlypeAhead is able to perform portscans of targets with reasonable accuracy.

It must be mentioned that even if a port is open on the target, it will be regarded as closed if the Glype proxy in use is unable to successfully connect to the service.

Due to GlypeAhead being a proof-of-concept tool, its logic has been purposefully limited to only work against Glype installations which display the default cURL error messages.

Below are screenshots of a Nmap scan compared to a GlypeAhead scan.

GlypeAhead is written in PHP (as is the Glype proxy), requires the cURL extension to be compiled with PHP, and is available from either our Pentesting Tools section, or directly from here.

Since joining SensePost I've had a chance to get down and dirty with the threat modeling tool. The original principle behind the tool, first released in 2007 at CSI NetSec, was to throw out existing threat modeling techniques (it's really attack-focused risk) and start from scratch. It's a good idea and the SensePost approach fits nicely between the heavily formalised models like Octave and the quick-n-dirty's like attack trees. It allows fairly simple modeling of the organisation/system to quickly produce an exponentially larger list of possible risks and rank them.

We've had some time and a few bits of practical work to enhance the tool and our thinking about it. At first, I thought it would need an overhaul, mostly because I didn't like the terminology (hat tip to Mr Bejtlich). But, in testament to Charl's original thinking & the flexibility of the tool, no significant changes to the code were required. We're happy to announce version 2.1 is now available at our new tools page. In addition, much of our exploration of other threat modeling techniques was converted into a workshop of which the slides are available (approx 30MB).

The majority of the changes were in the equation. The discussion below will give you a good idea of how you can play with the equation to fundamentally change how the tool works.

There are 5 values you can play with in the equation:

1. imp - the impact of a risk being realised
2. lik - the likelihood of the risk occurring
3. int - the value of an asset (represented by an interface to that asset)
4. usr & loc - the measurable trust placed in a user & location respectively

The current default formula is:

In English that translates to: The risk is equal to; the average of the impact of the attack and it's likelihood, combined with the value of the asset (exposed through a particular interface), and reduced by the trust of the user performing the attack and the location they are performing it from.

We felt there were two problems with this equation:

1. It doesn't acknowledge impact as linked to value. e.g. You can't have a huge impact on something of low value.
2. It doesn't see trust as linked to likelihood. e.g. a trusted user in a trusted location is less likely to commit an attack.
3. It double weights trust with location and user trust counting at full weight.
4. It's maybe a little far from semi-consensual views on the subject

After much internal wrangling, and some actual work on modeling fairly complex stuff, we came up with a new equation. While we feel this works better, it does mean the way things are modeled changes, and hence backwards compatibility with existing models is broken (but you don't need to use this equation). The new equation (consider the risk= implied) is:

Once again in English: The risk of an attack is; the likelihood of the attack reduced by the average of both the trust in the user & location, combined with, the value of the asset reduced by the potential impact of the attack (value at risk). (The 0.2 & 2.5 are just to make it fit the scales. Specifically, the 0.2 is because the scale of the entities is 1-5 and we're looking to make a percentage, and the 2.5 is to fit the 0-25 scale on the final graph.)

The key change which breaks backward compatibility here is that impact now becomes a moderator on value. i.e. the impact of an attack determines how much of the asset's value is exposed.

The way things are now modeled, interfaces represent the value of a system. For the most part, all a system's interfaces should have the same value, because as we often see, even minor interfaces that expose limited functionality can often be abused for a full compromise. However, the actual attack (called threats in the tool) determined how much of that value is exposed. For example, a worst-case XSS is (depending on the system of course) probably going to expose less of the system's value than a malicious sysadmin publicly pwning it (once again, dependent on the system and controls in place).

Unfortunately, there's still no provable way to perform threat modeling, but we feel we can go quite far in providing a quick and useful way of enumerating and prioritising attacks (and hence defenses) across complex system.

In a future blog post, I hope to cover some of the really cool scenario planning the tool can let you do, and the pretty graphs it gave us an excuse to justify budgets with.

[ Credit to the Online LaTeX Equation Editor for the formulas, although if you'd like to copy paste the formula described above into the tool, here's an ascii version:

( ( ( lik * ( ( ( (6 - usr) + (6 - loc) ) / 2 ) * 0.2 ) ) + ( int * ( imp * 0.2 ) ) ) * 2.5 )

]