Wikto
Web Server Assessment Tool
Document Information
|
PUBLIC |
|
|
Completed On: |
2004-10-20 |
|
Document Number: |
Wikto_2004-10 |
|
Issue: |
1.1 |
|
Last Modified: |
2004-10-20 |
|
Author: |
Roelof Temmingh |
Revision History
|
Document Version |
Description |
Date |
Author |
|
1.0 |
First Draft |
2004-08-10 |
Roelof Temmingh |
|
1.1 |
Public release |
2004-10-20 |
Roelof Temmingh |
|
|
|
|
|
|
|
|
|
|
SensePost Contact Details
|
Physical Address
|
Postal Address |
Contact Number |
Fax Number |
||
|
Ground Floor Nieuw Muckleneuk 0181 |
P.O Centurion 0046 |
+27 12 460 0880 |
+27 12 460 0885 |
||
|
Contact E-Mail
Addresses |
|||||
|
General: |
www.sensepost.com |
||||
|
Training: |
|||||
|
Reseach: |
|||||
|
HackRack: |
|||||
Table of Contents
IMPORTANT NOTES – please read this:.............................................................................. 4
1 Features/bug fixed added in Version 1.51:............................................................. 4
1.1 Google Hack section............................................................................................ 4
2 Introduction............................................................................................................ 5
3 Uses........................................................................................................................ 6
4 AI/Fuzzy logic....................................................................................................... 10
5 Googler................................................................................................................ 12
5.1 Setup................................................................................................................ 12
5.2 Starting a run..................................................................................................... 13
6 Back-End miner.................................................................................................... 15
6.1 Adding/Editing
suspect directories....................................................................... 15
6.2 Starting a run..................................................................................................... 15
6.3 During mining..................................................................................................... 16
6.4 Exporting results................................................................................................ 17
7 Wikto CGI checker................................................................................................ 18
7.1 Setup................................................................................................................ 18
7.2 During the scan.................................................................................................. 20
7.3 Exporting results................................................................................................ 23
8 System wide settings............................................................................................ 23
IMPORTANT NOTES – please read this:
1. Wikto uses the .NET framework – it will not work if the .NET framework is not loaded. The framework can be obtained by doing a Google search for:
“.NET framework” download
2. Wikto is provided without any warrantee etc.
3. Feedback/suggestion can be addressed to research@sensepost.com.
4. The newest version of Wikto is always available at http://www.sensepost.com/research/wikto
5. Wikto is released with full source. If you want to contribute to Wikto please contact us. All help is appreciated.
1
Features/bug fixed added in Version 1.51:
- Wikto understands the Google
Hack DB, and can run all tests automatically
- Manual testing for Google
Hacks enabled
- Sockets in CLOSE_WAIT is
cleaned up using forced Garbage collection
- Proxy support for Wikto
section works
- BackEnd Miner shows minimum
and maximum AI values
1.1
Google Hack section
The Google Hack section is pretty straight forward. Johnny Long told me
he will make the GHDB (Google hack database) publicly available for download.
With this in mind Wikto now supports the DB, and can do all tests
automatically. The setup is rather straight forward:
Simply enter the target site (or domain, or even TLD), load the XML database, and click on “Start GH”. Make sure you have a valid GoogleAPI key set in the SystemConfig section. Results appear in the bottom window, descriptions of the issue in the middle window. If the mouse enters the top window (shaded in the screenshot), the descriptions will update as they are tested. You may also click on the results – if it’s a Google query then its description will appear in the middle window – else it will show the URL (so you can copy and paste it – we’ll get automatic IE surfing in the next release…promise).
You can also do a manual test – if you need to check something quickly. Wikto does not add the “site:” functionality to manual queries – so its just a good as a normal Google search. Manual testing can be done at any stage – even when a scan is running (results will be appended to the results screen).
It’s not yet clear how the Google Hack Database (GHDB) will be distributed – but you should be able to find a recent one quite easily. Automatic updates for the DB will be provided in upcoming releases.
The rest works pretty
much as you would expect it to work.
2
Introduction
In the last couple of years the testing of web servers has been largely “standardized”. It has come to the point where testers are reluctant to spend too much time on web servers and rather spend time on web applications. This is largely due to the fact that there have been little or no advances in web server testing tools, while new web application testing tools are appearing like mushrooms after a rainy night. This trend is totally understandable – these days very few web servers are exploitable using network application level vulnerabilities. System administrators tend to patch servers, and secure back-end interfaces. Why then spend more time on building yet another web server testing tool?
1. Testing web servers properly has been limited to people who are comfortable in the Unix environment – tools like Nikto and Nessus (the defaco open source tools) are native to the Unix environment. Porting these tools to the Windows world is totally possible (Newt is appearing on the schene at the time of writing this, and using Active State Perl you could port Nikto to Windows), but not wide spread.
2. Very few tools solve the issue of false positives. While assessment tools have been standing still the last couple of years, web server administrators are keeping with the times. Today there are many servers that do not return proper error messages (404s, 200s). Testing these servers with traditional testing tools is difficult and in some cases outright impossible. Later in this document we will see how we address this problem.
3. In order to really understand where developers make mistakes you need to be able to think like a developer. SensePost recently sent 3 engineers on C# training in order to learn how things really sit together in .NET environment. This tool came out as a by-product of this training.
4. The idea is that this tool is the definitive web server assessment
tool – this tool should be all you ever need to perform assessment on web
servers (important note – tools could NEVER replace the eye of an experienced
security analyst. Also – this tool is not applicable for web application level
testing – nor does it try to play in that space at all).
3
Uses
The tool that we build is called Wikto. It is apparent that the tool provides the same functionality as the Nikto tool. But it goes a little further. There are 3 main sections of the tool. These are:
· Back-End miner
· Nikto-like functionality
· Googler
This document will discuss these 3 sections and
their interactions in detail. But let’s first look at some screenshots:
The Back-End Miner:
Wikto CGI checker:
Googler tool
4
AI/Fuzzy logic
Wikto
has a small checkbox in the Wikto and Back-End section named “Use AI”. This checkbox
dramatically change the way that Wikto works. If this checkbox is unchecked,
Wikto will use standard error codes to determine if a file or directory is
present. In the Back-End section of the tool these error codes can be manually
set. In the Nikto-like section codes are determined by what ever is contained
in the Nikto scan database (usually 200).
The
problem with inspecting error codes is that many sites do not return clean
error codes – these are sites that will (for instance), redirect to a friendly
error page. The “error” codes returned for files that do not exist therefore
are 200 – making the scanner believe that the file exists. Nikto typically
responds like this:
…
+
Over 30 "OK" messages, this may be a by-product of the
+ server answering all requests with a
"200 OK" message. You should
+ manually verify your results.
…
Nessus
performs a little bit better dealing with “friendly 404s”, but not a lot.
When
checking the “use AI” checkbox, Wikto ignores error codes altogether and
activates its fuzzy logic / AI modules (call it what ever you want – bottom
line is that it doesn’t look at error codes anymoreJ). Wikto creates a signature based on three elements:
·
File extension (e.g. asp, pl, cgi etc.)
·
Location (e.g. /scripts, /admin)
·
Method (e.g. POST, GET, SEARCH)
Before
doing any check (be that for a file in the Nikto part, or for a directory in
the Back-End miner), it requests a non-existent file with the same extension,
method and location. The tool then builds a fingerprint of the request. Next it
requests the real file/directory and compares the fingerprint. The function
that compares the signatures/fingerprints returns a value of how close the two
prints match. The higher the value, the closer they match – the lower the
value, the more they differ. Theoretically, an exact match will result in a
value of 1 while a complete mismatch results in 0, but in real life values
higher than 1 have been seen…
The
analyst (the user of the tool), can decide on which value to trigger a hit. In
the Nikto-like section of the tool the user can do this at any time/in real
time – Wikto saves the result of every request. This means the analyst can run
the tool and after it has completed modify the trigger levels to see which
requests fall within the limit.
When the “Update” button is clicked after modifying the value, Wikto will display the new results. The “Show all” button simply sets the value to a large number (1000), and updates the display. When the “Reset” button is clicked the previous value is restored, and the display is updated. These buttons can be clicked at any time.
The Back-End tool (currently) does not keep a history of results, but the trigger level can be set in real time.
Both the Back-End tool and the Nikto-like tool’s fingerprint DB can be cleared at any time. When engaging another target, these databases should be cleared.
Keep in mind that the AI/Fuzzy logic only looks at the similarity in responses. This could sometimes lead to unexpected results. When requesting “/global.asa” on an IIS server, ACLs on the server denies the request. When requesting “/nothere.asa” however the ACL does not kick in and a normal 404 error is returned. For the AI component these are two different responses, and the file is thus marked as present. It is important to understand how the AI/Fuzzy logic works in order to interpret results.
5
Googler
The Googler tool’s primary function is to obtain additional directories for use in the Nikto-type tool and the Back-End miner tool. As an interesting by product it sometimes list files found on web servers that should be there.
5.1 Setup
The first step to use the Googler tool is to get
a Google API key. This is a painless process – having a Google API key is a
Good Thing ™, and it’s free! Simply visit http://api.google.com
and fill in the form.
Once you have a valid Google key enter it in the Google Key field found in the “SystemConfig” tab:
Note – the key that’s hard coded in the application DOES NOT WORK! YOU NEED YOUR OWN KEY!
As you can easily exhaust your Google key’s daily limit you might want to increase or decrease the number of pages to download. By default, Google provides results in batches of 10, with a maximum of 100 pages (or 1000 results). The default should be a good start.
The Googler tool’s logic works like this:
1. Perform a search for “<Google keyword> filetype: <filetype> site: <Site/Domain>”
2. Find directories in results.
3. Add new directories to “Mined directories”
These steps are performed for every file type. By modifying the “Google keyword”, “filetype” and “site/domain” very interesting results can be found. The Googler tool automatically updates the “Google keyword” section as the “site/domain” input field is modified, but this can be changed before the search is performed.
Finding interesting information by changing these values is left as an exercise to the reader…
5.2 Starting a run
When everything has been set up (again – the defaults should be a good start), the analyst can click on “Start Googling”. Full URLs will be returned in the main windows. Currently clicking on these will have no result.
Directories (and its parents) are immediately populated in the “mined directories” section. This is a normal text box, and can be edited. The small “cl” button clears the list. Note that subsequent runs will append to the list.
These directories can be imported to either the Wikto CGIDIRs list, or to the Back-End Miner’s suspect list. See later sections for more information on this.
6 Back-End miner
The Back-End miner section is used to find interesting files and directories on a web server.
6.1 Adding/Editing suspect directories
The suspect directories, file types and file names can be set manually by simply loading the files into the tool.
By default, these files, directories and file types are appended to the list. The lists can be cleared of all entries by clicking on the small “Cl” button at each list:
If the Googler tool has been run, the resultant directories for the search can be imported into the list by simply clicking on the “Import from Googler” button.
Again, directories are appended to the list.
6.2 Starting a run
To start a run, enter the web site’s DNS name into the IP/DNS name field. If the server is running on a non-standard port, the port can only specified:
The tool automatically updates the HTTP header field to accommodate virtual hosts. The header can be further modified to include Cookies, different Agent types etc. The HTTP header is located in the “SystemConfig” tab:
At this stage the analyst has to decide if she wants to use AI or not (see section on AI/fuzzy logic for more information). If AI is not used the trigger error codes for both directories and files can be set – the defaults should provide a good first iteration:
Next the analyst needs to decide to use GETs or HEADs. Normally HEADs are faster, but some servers could complain about this method and spew unexpected result codes. By default the HEAD method is used. To switch this to the GET method, check the GET/HEAD checkbox:
If using AI/fuzzy logic, and this is the second time the tool is used it could be that the analyst wants to clear the fingerprint database. This can be done by simply clicking the “Clear fingerprint DB” button.
At this stage all is set to start the scan, and the analyst can click on the “Start Mining button”:
Directories found are returned in the “Directories (results)” text box. Note that this is a plain text box and can be edited if needed. Files found are returned in the “Files (results)” box. By default the root “/” is populated in the directory results.
If the “preserve results” box is checked, the tool will not remove directories and files found. This is useful when the directories are known; the analyst can populate the resultant directories and click on “Skip to files” – see the next section for more details on this functionality.
6.3 During mining
The logic of the Back-End mining tool is as follows:
1. Check all suspect directories to see which exists.
2. Recursively checks within the directories found for the suspect directories (e.g. if it finds /admin it will check if /admin/cgi, /admin/scripts etc. exists).
3. If no more directories are found check each directory for each filename, and file type.
It is clear that this process can take quite a while. Imagine 12 directories, 10 file types and 100 filenames – this results in 12 x 10 x 100 requests, or 12000 requests. For the impatient, the Wikto tool provides a way to manually skip parts of the checks.
If the “Skip directory” button is clicked while the tool is checking directories it will skip the current directory and start with the next directory in the list. If the tool is busy with file checking it will skip the currently directory and move to the next. If the “Skip to files” button is clicked, the tool immediately skips all directory checks and proceeds directly to checking files.
In combination these two buttons allow the analyst to do an in-depth test, or just a quick glance over.
If the “Stop mining” button is clicked the tool will stop after completing the current request.
The directory progress bar is zero-ed for each run – this means it will be reset for each directory. The file progress bar attempts to give some indication of how far the file scan has progressed, but has been found to be slightly inaccurate...
6.4 Exporting results
Results can be exported by clicking on the “Export Results” button.
Results are saved in CSV format and include the files and directories found.
7 Wikto CGI checker
The Wikto tool tries to provide the same functionality as Nikto.
7.1 Setup
Wikto uses the same scan database as Nikto version 1.X. This database can be obtained from www.cirt.net (under the Nikto Web Scanner -> Plugins & DB section):
At this stage Wikto does not have an automatic database update function (but watch this space). The scan database can be simply downloaded (scan_database.db) locally.
Once the database is downloaded, it can loaded in the Wikto tool, by simply clicking on the “Load DB” button:
Once loaded, the all the checks appear in a list box:
The list box shows two elements per check (tab separated) – the trigger (be that a specific string or error code) and the request. Clicking on any of the test provides more information about the selected check:
NOTE: Wikto tries to identify and ignore XSS test! This is not a bug but by design.
After the database is loaded, the target can be configured. Again, if the server runs on a non-standard port, it can be configured here:
Like with the Back-End miner, the HTTP header is automatically updated, and can be edited manually. Note that if the Wikto tool and the Back-End Miner share the HTTP header field. If these two tools are in use at the same time the target should also be the same.
Nikto makes use of a variable called @CGIDIRS (as well as @ADMIN and a few others). Some tests will look for a certain script or CGI within a @CGIDIRS – this basically means that the tool should look through all possible CGI directories for the specific script. The Wikto tool populates this variable in three ways:
1. Using user input (e.g. manual entering of CGI directory names)
2. Importing from the Googler tool
3. Importing from the Back-End miner tool.
While some of the directories found in 2. and 3. are clearly not traditional CGI directories (e.g. they are not set to executable), you might find interesting results if you leave them there..
To import CGI directories from the other tools, simply click on the relevant import button:
Next, the analyst needs to decide if AI/Fuzzy logic should be used. See the section on AI/Fuzzy logic for information on this functionality.
Wikto can be configured to optimize scans – this basically means that tests for CGIs located in directories that are within the Back-End miner tool suspect list (the list of directories we test for), but that was not found are ignored. The logic here is that if the directory was tested and not found, a CGI in the directory will also not be found.
7.2 During the scan
If the mouse pointer is inside the boundaries of the scan database window (highlighted in blue), Wikto will show the current HTTP request and reply, and highlight the current test in the listbox:
If the mouse is outside of the boundary, the best way to know if the scan is still running is to look at the progress bar, or the single request progress bar (the tiny 3 bar progress in the right hand bottom corner).
Results are displayed in the right hand top window. The following elements are listed – Weight, trigger and request. Again, clicking on an item in the result window will display details in the bottom (“Description”) window.
The results window is populated with the following logic:
· If using AI/Fuzzy logic (see section on this for more details):
o If the fingerprint comparison is lower than the value specified on the UI, and the check uses error codes as trigger.
o If the check does not use error codes (but strings), and the reply contains the trigger string a value of 0.01 is given as fingerprint comparison.
· If AI/Fuzzy Logic is switched off:
o If the check uses error codes and the trigger error code is returned a value of 0.01 is retruned
o If the check uses strings and the trigger string is returned a value is 0.01 is returned
While this could seem confusing at the time of reading it is best explained when actually using the tool.
Keep in mind that the fingerprint comparison is stored with every result – this means that the analyst can change the value during the scan, or even when the scan is done.
7.3 Exporting results
Results can be exported at any time. All results in the results window will be exported. Keep in mind that the data in this window is dependant on AI/Fuzzy logic level (if used at all).