Google Hacking for Penetration Testers

If you think, “Oh, that Google search stuff isn’t very useful in a real-world
penetration test... that’s just playing around,” then you have no idea what you
are talking about.Whenever we conduct a detailed penetration test, we try to
schedule at least one or two days for a very thorough investigation to get a feel
for our target before firing a single packet from a scanner. If we can get even
more time from the client, we perform a much deeper investigation, starting
with a thorough interrogation of our favorite recon tool, Google.

Intro

Google’s Web Search Page
The interface is known for its clean lines, pleasingly uncluttered feel. Although the interface might seem relatively featureless at first glance, we will see that many different search functions can be performed right from this first page. The links above the search field (Web, Images, Groups, and so on) open the other search areas .The basic search functionality of each section is the same. Each search area of the Google Web interface has different capabilities and accepts different search operators, as we will see later. For example, the inauthor operator was designed to be used in the groups search area.

• Web, Images, Groups, Directory; News; Froogle; and more >> tabs: if you are a first time Google user, understand that these tabs are not always a replacement for the Submit Search button.
• I'm feeling lucky button: Instead of presenting a list of search results, this button will forward you to the highest-ranked page for the entered search term. Often this page is the most relevant page for the entered search term.
• Advanced Search: Some advanced features are not listed on this page.
• Preferences: select several options (which are stored in cookies on your machine for later retrieval). Available options include language selection, parental filters, number of results per page, and window options.
• Language tools: This link allows you to set many different language options and translate text to and from various languages.

Google Web Results Page
The top part of the search result page mimics the main Web search page. Notice the Images, Groups, News, and Froogle links at the top of the page. By clicking these links, you automatically resubmit your search as an Image, Group, News, or Froogle search, without having to retype your query.

The results line shows which results are displayed (i.e. 1–10), the approximate total number of matches (i.e. 634,000), the search query itself (including links to dictionary lookups of individual words), and the amount of time the query took to execute.The speed of the query is often overlooked, but it is quite impressive. Even large queries resulting in millions of hits are returned within a fraction of a second!

For each entry on the results page, Google lists

• the name of the site,
  
• a summary of the site (usually the first few lines of content),
  
• the URL of the page that matched,
  
• the size and date the page was last crawled,
  
• a cached link that shows the page as it appeared when Google last crawled it, and
  
• a link to pages with similarcontent.
  
• If the result page is written in a language other than your native language and Google supports the translation from that language into yours (set in he preferences screen), a link titled Translate this page will appear, allowing you to read an approximation of that page in your own language.
  

Translation Proxies It’s possible to use Google as a transparent proxy server via the translation service. When you click a Translate this page link, you are taken to a translated copy of that page hosted on Google’s servers. This serves as a sort of proxy server, fetching the page on your behalf. If the page you want to view requires no translation, you can still use the translation service as a proxy server by modifying the hl variable in the URL to match the native language of the page. Bear in mind that images are not proxied in this manner.

Google Groups
Due to the surge in popularity of Web-based discussion forums, blogs, mailing lists, and instant-messaging technologies, USENET newsgroups, the oldest of public discussion forums, have become an overlooked form of online public discussion. Thousands of users still post to USENET on a daily basis. A thorough discussion about what USENET encompasses can be found here. DejaNews (deja.com) was once considered the authoritative collection point for all past and present newsgroup messages until Google acquired deja.com in February 2001 (see here). This acquisition gave users the ability to search the entire archive of USENET messages posted since 1995 via the simple, straightforward Google search interface. Google refers to USENET groups as Google Groups. Today, Internet users around the globe turn to Google Groups for general discussion and problem solving. It is very common for IT practitioners to turn to Google’s Groups section for answers to all sorts of technology-related issues. The old USENET community still thrives and flourishes behind the sleek interface of the Google Groups search engine.

The Google Groups search can be accessed by clicking the Groups tab of the main Google Web page or by surfing to groups.google.com. The search interface looks a bit different from other Google search pages, yet the search capabilities operate in much the same way. The major difference between the Web search page and the Groups search page lies in the newsgroup browsing links.

• Advanced Groups Search: Not all advanced features are listed on this page. We will look at these advanced options later.
• Groups Help: This link takes you to the Google Groups FAQ page.
• alt., biz., comp., etc. links: These links reflect the topical hierarchy of USENET itself. By clicking on the links, you can browse through Google groups to read messages in a ‘threaded’ format.
  

Google Image Search
The Google Image search feature allows you to search (at the time of this writing) over 1 bilion graphic files that match your search criteria. Google will attempt to locate your search terms

• in the image filename,
  
• in the image caption,
  
• in the text surrounding the image, and
  
• in other undisclosed locations,
  

to return a “de-duplicated” list of images that match your search criteria. The
Google Image search operates identically to the Web search, with the exception
of a few of the advanced search terms, which we will discuss later. The search results page is also slightly different. The Show: line is unique to image results. This line allows you to select images of various sizes to show in the results. The default is to display images of all sizes. Each matching image is shown in a thumbnail view with the original resolution and size followed by the URL of the image.


Google Preferences
You can access the Preferences page by clicking the Preferences link from any
Google search page or by browsing to www.google.com/preferences.These options
primarily pertain to language and locality settings.

• The Interface Language option describes the language that Google will use when printing tips and informational messages. In addition, this setting controls the language of text printed on Google’s navigation items, such as buttons and links. Google assumes that the language you select here is your native language and will “speak” to you in this language whenever possible. Setting this option is not the same as using the translation features of Google (discussed in the following section). Web pages written in French will still appear in French, regardless of what you select here. To get an idea of how Google’s Web pages would be altered by a change in the interface language, take a look to see when Google’s main page rendered in “hacker speak” (Even though the main Google Web page is now rendered in “hacker speak”, Google is still searching for Web pages written in any language. If you are interested in locating Web pages that are written in a particular language, modify the Search Language setting on the Google preferences page. By default, Google will always try to locate Web pages written in any language). In addition to changing this setting on the preferences screen, you can access all the language-specific Google interfaces directly from the Language Tools screen at www.google.com/language_tools
• SafeSearch Filtering blocks explicit sexual content from appearing in Web searches. Although this is a welcome option for day-to-day Web searching, this
  option should be disabled when you’re performing searches as part of a vulnerability assessment. If sexually explicit content exists on a Web site whose primary content is not sexual in nature, the existence of this material may be of interest to the site owner.
  
• The Number of Results setting describes how many results are displayed on each search result page. This option is highly subjective, based on your tastes and Internet connection speed. However, you may quickly discover that the default setting of 10 hits per page is simply not enough. If you’re on a relatively fast connection, you should consider setting this to 100, the maximum number of results per page.
  
• When checked, the Results Window setting opens search results in a new browser window. This setting is subjective based on your personal tastes. Checking or unchecking this option should have no ill effects unless your browser (or other software) detects the new window as a pop-up advertisement and blocks it. If you notice that your Google results pages are not displaying after you click the Search button, you might want to uncheck this setting in your Google preferences.
  
  Proxy Server Language Hijinks
  Proxy servers can be used to help hide your location and identity while
  you’re surfing the Web. Depending on the geographical location of a
  proxy server, the language settings of the main Google page may change
  to match the language of the country where the proxy server is located.
  If your language settings change inexplicably, be sure to check your proxy
  server settings. It’s easy to lose track of when you are running under a
  proxy and when you’re not. As we will see later, language settings can be
  reverted directly via the URL
  
  

Language Tools
The Language Tools screen, accessed from the main Google page, offers several different utilities for locating and translating Web pages written in different languages. The first portion of the Language Tools screen allows you to perform a quick search for documents written in other languages as well as documents located in other countries.