Passive (OSINT)
Introduction
Open Source Intelligence (OSINT) could also be defined as a set of techniques and tools for gathering public information about a company, physical person, etc., analysing the collected data and correlating it, turning them into intelligence.
In the "Information Gathering" phase of a pentesting OSINT tools can help the pentester in different aspects:
Technical information: Discover services, hosts, domains, subdomains, git repositories, credentials, social networks, operating system versions, network diagrams, etc.
Human resources: Gather information about a specific person on social networks, government records, telephone numbers, leaked credentials, metadata...
As a goal, the before mentioned information can lead to different attacks:
Social engineering
Passwords Brute force Attacks
Target infiltration
Accounts take over
Identity theft
The OSINT Process
This sub-phase consists of concatenated processes that allow the attacker to obtain the required intelligence to perform the pentesting. The procedures are the following.
Requirements: What information do we need from the client as a starting point?
Sources of information: What sources of information can provide us with trusty information?
Harvesting: Retrieve data from the identified sources.
Data processing: Add format and process the obtained data, obtaining meaningful information.
Data analysis: Join data from multiple sources, producing intelligence.
Reporting: Create the final report.
Sources of information and tools
This section presents some tools and sources of information the pentester has to consider during the OSINT phase. However, an attacker does not have to rely only on these tools and must be aware of new tools that allow him to obtain further information and confirm the collected data.
OSINT Framework
OSINT Framework is an interactive web page that contains a vast set of links to free and open-source tools for different purposes such as Domains, emails addresses, usernames, telephone numbers, public records, social network profiles...
Note: Take into account that some sites will require registration or paying money to obtain extra data.
As an alternative to the OSINT framework, there are several webs containing sets of references to different OSINT tools such as Malfrat's OSINT Map, osint.link, osinttechniques.com and OSINT_Collection.
Search Engines
Search engines are an excellent option to conduct passive reconnaissance because they are populated with a ton of information that has been previously indexed.
Google Dorks
Google dorks is a technique that uses the google advanced search options for gathering efficient and precise data on any topic from any website on the internet.
The Google search engine interprets search operators and commands for retrieving or accessing sensitive information that was knowingly or unknowingly released on the Internet like credentials, configuration files, documents... However, Google Search results might vary depending on the location or device used, so use it to your advantage depending on your target.
Because Google Dorking can be pretty challenging, there are already tools that help you create Google queries for collecting specific data.
Dorksearch
DorkSearch is a tool that gives you a list of prebuilt templates for Google Dorks for different use cases. Furthermore, you can create your queries from the builder menu.
Google Hacking Database
Google Hacking Database (GHDB) is an open-source project which stores and updates a set of know dorks.
Each dork is categorised in one of these categories:
Footholds: Searches that can provide a foothold on a server.
Files containing usernames
Sensitive directories: Searches for directories with sensitive information.
Web server detection: searches for web servers with a particular technology
Vulnerable files
Vulnerable servers
Error messages: Searches for error messages of a specific type.
Files containing juicy info: Searches for files containing important information
Files containing passwords: Searches for files with passwords
Sensitive online shopping info: Searches for sensitive information from online shopping sites
Network or vulnerability data: Searches for specific network information or vulnerabilities.
Pages containing login portals: Searches for pages containing particular login portals
Various online devices: Searches for specific online devices
GitHub dorks
GitHub dorks are pretty similar to google dorks. This can be very handy when searching for sensitive files, API keys, passwords, hidden URLs, employees, etc.
To start with GitHub dorks, you can start from the GitHub Documentation: Search on GitHub or posts like GitHub Recon, Developers are unknowingly posting their credentials online and GitHub for Bug Bounty Hunters. Here are some valuable examples, but if you want more, visit these links: GitHub-dorks and keywords.
Furthermore, the success of GitHub dorks has driven to the creation of third-party tools like Gitrob and the result of GitHub Code Search.
Shodan
Shodan is a search engine for directly accessible devices connected to the Internet, discovering devices like cameras, traffic lights, power plants.
Furthermore, it counts with several filters obtaining devices with default credentials, vulnerable services, screenshots, cloud providers, location... However, the filters "tag" and "vuln" are not accessible with the free plan requiring an academic membership or a small business plan.
Thanks to shodan, the pentester can create shodan queries looking for physical assets that are part of the target company for a red teaming attack and vulnerable services exposed on the Internet that will allow them to get a foothold on the company's IT infrastructure.
Archives.org
When conducting OSINT research in these fast-paced digital times, analysts often need access to historical versions of websites or content that no longer exists. This is where The Wayback Machine comes into play.
For instance, if you are looking to see historical versions of a website due to the site being deleted or replaced with new content, the Wayback Machine can help. You may need to verify that a target previously worked at a company, but the site's current state does not have the targetâs information there. Furthermore, sometimes a target may intentionally hide information from their present website; looking at older dates of the site may reveal new information. Sometimes you can gather relevant data like names, phone numbers, email addresses, and even metadata from older website versions.
Methods
Quick Search Methods: The quickest method to see all the files archived on a particular site is by accessing the following URL.
Advanced Search Method: By directly visiting the archive advanced search page, the attacker can perform more targeted searches and sometimes find the email address associated with a user who uploaded a file. However, this requires you to register on the platform.
Change: The changes method identifies and displays changes in the content of archived URLs. However, please consider that it is still in beta nowadays, so it might not render both websites correctly.
Save pages: The Wayback machine accepts requests for a web page to be archived. However, this method is only available for web pages that allow crawlers.
Email Harvesting
Harvesting email addresses gives an attacker more information to conduct social engineering and password brute-forcing attacks.
The Harvester
TheHarverster is a command-line tool that is already installed in Kali Linux. The primary purpose of the harvester is pervasively gathering e-mail accounts, subdomain names, virtual hosts, open ports/ banners, and employee names from different public sources using various search engines, which has increased in variety for the past years. In recent versions, the authors added active techniques like the capability of doing DNS brute force, reverse IP resolution and Top-Level Domain.
We can obtain many results with just a simple command, thanks to its simplicity. The results can be complementary with other tools like maltego.
Maltego
Maltego is one of the most powerful open source intelligence tools on the market; It is characterised by its intuitive handling and its representation of information based on graphs that connect information for investigative tasks.
Maltego is used to map the relationships between pieces of information named Entities, resulting from running transformations.
Entities are bits of information that we have obtained from a data source (a physical location, a website, a company name, an email address, a personâs name and a telephone number).
Transformations are small pieces of code that fetch related information for a given input and format the results to be returned as Entities to Maltego.
However, be careful when running transformations because they can escalate too quickly, providing you with over information and turning into a gigantic graph populated with useless information.
Finally, to use maltego, you need to create an account, then you have to choose between different plans: Maltego Community Edition (CE); Case File; Classic; XL and One, being free the two formers.
HaveIBeenPwned
Once obtained information about a user like an email or phone, the pentester should check on sites like haveibeenpwned.com whether there are compromised accounts that have suffered a data breach, meaning the email and password for that siteâs account has been exposed to cybercriminals.
A pentester could obtain these credentials for different means, checking if employees share passwords across multiple accounts in the company's environment.
DeepSearch
DeepSearch is a deep website, a successor from pwndb2, where a pentester could obtain the credentials leaked in data breaches for free.
Another solution for obtaining leaked passwords could be dehashed.com, but it is not free.
Recon-Ng
Recon-NG is a CLI framework installed on Kali Linux that conducts open-source web-based reconnaissance. It can be used for obtaining SQL injections, IPs, IP lookup, port scanning, sub-domain information, etc.
Obtaining its maximum potential can be achieved using API keys that some modules require. However, they might be subscription-based.
For setting the API keys, you need to execute the following commands.
You will also find that no modules are installed by default, but they are easily installed with these commands.
Gives a list of all modules in the marketplace:
Install an individual module:
Install all modules in a category:
Remove a module:
Note: Some of the modules will require Python dependencies to be installed outside of Recon-NG. Modules with external dependencies will have an asterisk in the D column of Marketplace results, and those requiring an API key will have an asterisk in the K column. In both cases, Recon-NG will warn you about missing dependencies and API keys after installation.
Then, to run any module, you need to follow these steps:
If you want to take a deeper look at what a module does, you can use the marketplace info module followed by the module name or path.
Once you have decided which module you want to use, proceed with loading it using:
Find any module prerequisites.
Set the options.
Execute the module.
DNS or subdomain enumeration
An unsecured subdomain can lead to severe risks, so the pentester needs to check them. Here you have some useful tools for obtaining subdomains that do not require direct access to the client's infrastructure.
Other sources & tools
Gather organisation's email: ReconMail
Instagram: Osintgram
Obtain phone operator: NumeraciÃŗn & Operadores
Image Reverse: TinEye
GEO-SSID: Widdle
Watch videos frame by frame: WatchFrameByFrame
Last updated