# The Pentesting Guide ## Introduction On this website, you will find a guide and methodology that you can follow to perform a penetration test. ## What is a penetration test? A penetration test is an authorised simulated cyberattack against your computer system to evaluate its security at a given time, identifying and exploiting any vulnerability on your systems that attackers could exploit to demonstrate the business impacts of weaknesses in your systems. ## What is not a penetration test? A penetration test is not: * **A vulnerability assessment**: The process of identifying, quantifying, and prioritising (or ranking) the vulnerabilities in a system. * **Red Teaming**: The process of demonstrating how an organisation would face a real attack against its networks, applications, physical security controls and employees. * **Bug bounty program**: Independent hackers are paid per vulnerability and bugs found. Bounty programs usually continue for the product’s lifetime finding new vulnerabilities as the product changes. ## How to proceed? As you can see in the image below, this is not only a summary of what you are going to find on this site but also the steps you should follow in order to perform a penetration test. ![Pentesting phases](/files/uN3WfgCpKQ838PJrVWBa) ## Guide Licence This guide is under Attribution-NonCommercial 4.0 International (CC BY-NC 4.0), you are free to: * **Share**: copy and redistribute the material in any medium or format. * **Adapt**: remix, transform, and build upon the material Under the following terms: * **Attribution**: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. * **NonCommercial**: You may not use the material for commercial purposes ![Attribution-NonCommercial (CC BY-NC 4.0)](/files/UhqD8SzS5LlRxEbdz6ed) # 0 - Pre-Engagement ## Introduction This section contains several topics that a pentester must know before any penetration test. ## The Scope Defining the scope is the most crucial piece of a penetration test because it will prevent you from penalties, trials and unsatisfied customers. The scope must define explicitly what is going to be tested: a set of IPs, domains, subdomains, services, access points (BSSID & ESSID) and devices (any asset of the company). Furthermore, a well-defined scope will save you time and money for the customer. For instance, machines inside the same network might differ in services, requiring more or less time to evaluate the whole device. Moreover, you have to consider that an IP could contain several services like web services, and each web service might contain several domains with subdomains. Hence, it would be malpractice to define the scope IP based only. Dealing with Third Parties Companies outsource services like hosting a web application to third parties. Hence, you need to know if the third party allows testing the customer services hosted on their system. Then, you should ask the company to contact the third parties to inform them about the execution of an imminent pentesting. Failing to obtain the proper permissions brings the possibility of breaking the law, resulting in a complaint by third parties for attacking their systems. Finally, consider that the law might differ depending on which country the outsourced services are hosted. For instance, it might be the case that port scanning might be forbidden in that foreign country. ## Attack strategy Depending on the available information provided by the customer and the starting point the customer wants to set, there are different approaches for pentesting. ### Categories Depending on the amount of knowledge known by the attacker, three types of tests can tackle. #### White Box Attackers know detailed information about the targeted system, including source code, configurations and system documentation. It allows attackers to find as many weaknesses as possible in the shortest period of time. Unlike internal testing, white box testing does not include any account credentials. #### Gray Box Attackers have some information, but not enough, so time and resources will be invested in finding new assets, identifying vulnerabilities and threats based on the amount of information the customer provided to them. #### Black Box The analyst engages the target without prior knowledge of its defences, assets or channels. The target is not notified in advance of the audit scope, channels tested, or test vectors. This type of audit assesses the analyst's capabilities and the target's readiness against unknown attacks. ### Starting point Depending on the starting point of the attack, the following tests can be considered. #### External pentesting An attack against an organisation's external servers or devices, such as their website and network servers. The objective is to determine whether and how far an attacker can penetrate the company's infrastructure remotely. #### Internal pentesting An authorised user with standard access rights performs attacks, allowing you to determine what damage can be caused by an employee who has personal accounts concerning administrative accounts. ## Services Sometimes the customer will not be interested in performing a whole pentesting due to the lack of interest or money. However, you could offer a small set of sub-services. Here you have some examples. ### Public Footprinting Gather public data from the organisation located on the Internet looking for endpoints, vectors of attack and valuable information that can be helpful in the future like IP addresses, DNS information, operating systems, emails, phone numbers, employee information... ### Vulnerability Assessment Analyse the company's internal and external services looking for security weaknesses. ### Web Vulnerability Assessment Analyse web applications, APIs, or mobile web applications determined in the audit scope to find vulnerabilities like XSS, XXE, SQLi, Template injection... ### Wireless Network Assessment Identify Wi-Fi vulnerabilities on the company's access points and connected devices like AP/Client misconfigurations, MAC spoofing, rogue access points and WPS Attacks. Then, identify the connected devices for a later traffic analysis (looking for credentials) or Man-In-The-Middle attacks. ### Pentesting The vulnerabilities found during the vulnerability assessment are exploited to compromise servers, endpoints, web applications, network devices... Furthermore, the test continues with subsequent vulnerability scanning and exploitations of other internal resources in order to determine the impact of the vulnerabilities and the risk to which the business was exposed. ### Social Engineering Focuses on company employees, performing attacks like phishing, USB drops, and employee impersonation to gain access to the company's infrastructure or retrieve sensitive information. ### Red Teaming A large-scale security assessment across the entire organization, being the scope everysingle asset of the business. This service involves every previous service and can include software analysis and physical security to obtain access and escalate privilege into the company's infrastructure. ### Denial Of Service Assess server, network or application performance against a Denial Of Service attack by overloading the system. Rules of engagement While the scope defines what will be tested, the rules of engagement define when, where and how that testing will occur. ## Time Estimation A penetration test can not last indefinitely, resulting in a significant increase in the invoice and a waste of time for the auditor. For example, suppose you base your penetration test on finding an exploitable vulnerability for a certain amount of money. In that case, it may occur that you never discover a vulnerability in the system because it is fully patched, so you would be losing money. Hence, it would help to estimate the time based on your experience. However, if you are new to this profession, a good approach would be to set ample padding that will be tuned in future tests based on your previous results. Furthermore, being more skilled will reduce your analysis time, getting the same results in less time. ## Locations There would be cases where the penetration tester will have to operate in the company locations, such as a set of buildings in one or several cities. Hence, the costs of travel, hotels, and diets should be stipulated. ## Changes during the pentesting The auditor should notify the client that any changes made to the scope's environment during the period of testing, may affect the results of the assessment. So, it is important to postpone changes until the end of the evaluation. Finally, If objectives were changed during the course of the testing then all changes must be listed in the executive summary section of the report. Additionally, the letter of the amendment should be included in the appendix of the report and linked from the former section. ## Pentest schedule It might be the case that some customers require that all the testing or some parts are done outside business hours. This may mean working at night or at weekends, so the time of day requirements should be well established with the customer before testing begins. ## Attack strategy Depending on how the engagement is performed and the amount of information shared with the testing team, there might be the case that some services are not necessary. For instance, public footprinting might not be essential if you perform an internal vulnerability assessment. Thus, the client should be noticed. ## Client contact details Despite taking all the necessary precautions when testing, testing can sometimes go wrong for various reasons, such as exploiting a vulnerability, turning into a denial of service, or blocking accounts due to brute force. Therefore, it is crucial to have the correct contact information so that the pentester can reach the staff in charge if a down service needs to be up and running or an account needs to be unblocked, as soon as possible, preventing the company from incurring financial losses due to these issues. ## Sensitive data handling During test preparation and execution, the testing team will be provided with and may also find sensitive information about the company, the system, and/or its users. Consequently, sensitive data handling needs special attention. Here you have some security measures you should carry out: * Full disk encryption. * Sanitise your test machine between tests. * Check the various data protection laws that apply to each client. ## Secure communications During the engagement, you will deal with or find sensitive information that might be worth communicating to the client as vulnerabilities and malware found, the location of clear-text credentials or the final report; thus, must be encrypted. Before the engagement, you need to agree on which measures of secure communications are to be established. Good options would be: * **PGP Keys**: Using [Kleopatra](https://www.openpgp.org/software/kleopatra/) for encrypting files or emails. * Telephone * Face to Face meetings * Password-protected compressed files, using a robust pre-shared key or OTP before the engagement. ## Identifying the pentester Suppose you audit a company's external services or even internal services from external sources. In that case, an advertisement should be made containing the IP you are going to use to perform the attacks in order to distinguish you from an actual attacker. One way to do it is by hiring a static IP from your ISP, informing your client all the attacks will come from that IP. Another option would be to use your client's VPN, so every attack comes from their public IP so that they can analyse the traffic in case of an incident. ## Questionnaire One of the first steps in communication with the customer is sending questionnaires. Thanks to them, we can know the scope of the test and the estimated time required. In addition, the questions help the auditor understand what, how, when, where, and why the customer wants to evaluate their systems. A first approach would be to send a general questionnaire to obtain information about the company's motivation, contact information, location, IT infrastructure, security measures and hiring services. This file contains an example of a questionnaire you can use to copy the structure. {% file src="/files/4Y1wgejEgx4VYZdgtLk7" %} # 1 - Information Gathering ## Introduction This phase consists of gathering as much information as possible to gain more understanding of the target. Furthermore, it is the most overlooked and most important because in this phase, you will find and craft the attack vectors for the exploitation and post-exploitation phases. The vectors of attack could be: * Use of compromised or weak credentials * Phishing attacks. * Employee impersonation * The exploitation of service misconfiguration * Sniffing unencrypted or weakly encrypted communications * The exploitation of N-Day vulnerabilities ## Information Gathering Methods The methods used to collect information can be classified in two different ways. * **Passive**: Gather information without establishing a direct connection between the pentester and the client. It involves analysing public information on the Internet (blogs, news, social networks, etc.), public DNS queries, OSINT tools, etc. * **Active**: Gather information establishing contact between the pentester and the client. It involves HUMINT, port enumeration, private DNS queries, vulnerability scanning, credentials brute force, directory listing, etc. ### OSINT Open Source Intelligence (OSINT) extracts high-value intelligence by harvesting open-source records to create a comprehensive profile of specific targets. Open-source information can be defined as that information that is publicly available material that anyone can lawfully obtain by request, purchase or observation—as examples of public sources can be newspapers, journals, radio, television and the Internet, being the latter the most used nowadays. ### HUMINT Human Intelligence (HUMINT) is intelligence gathered from any human resource. Unlike OSINT, which employs passive information gathering methods, HUMINT requires direct contact with people; that is why it needs more qualified agents capable of influencing, persuading and/or manipulating their victims for obtaining semi-public or private information—using techniques like person impersonation, social engineering, eavesdropping, trashing, etc. Furthermore, since the obtained information is not public, it can not be contrasted; hence, the attacker must consider the sources' reliability. HUMINT techniques are used during a pentesting for testing employees' adherence to the security policies and practices defined by the company; and for the attacker to get access to sensitive information or facilities infrastructure. Finally, making a small comparison between OSINT and HUMINT, we know that the first one is more efficient, economical and challenging to detect. ## Types of information The information obtained can be categorised into different principal topics. ### Physical This information involves the client's different buildings (Headquarters, branch offices, franchises...) and the physical security of each office (camera placements, fences, sensors, guards, control access...). This information is helpful in physical pentesting or red teaming. ### Logical The logical category consists of getting information about relationships between business clients and partners, their main competitors, their offered products, marketing activities, company significant dates, job openings, court records, charity affiliations... Useful for HUMINT attacks. ### Organisational Breakdown Structure Obtain information about the organisation's structure and the relationships between ranks, jobs and employees. This information will help you to impersonate employees for spear phishing attacks. ### Employees Starting from the "Organisational Breakdown Structure", we can obtain specific data from each company employee like email, social networks (metadata), court records, tastes and hobbies, professional licenses or registries... The obtained information can be used during a pentesting or red teaming service: * The company will check each individual's trustworthiness. * The employee will know how much data they expose on the Internet. * The pentester will be able to craft spear-phishing attacks. ### Digital The information categorised as digital consists of the company's published documents with their metadata, company's email addresses and social networks, public IPs, domains and subdomains, opened ports, software used, external and internal infrastructure, public repositories and services, vulnerabilities, defence technologies... Thanks to the digital information, the pentester will find attack vectors later used in the pentesting. ## Finding vulnerabilities methodology This section shows the required steps you should follow in order to find vulnerabilities for each service on a target computer. 1. Identify the software running. * Obtain the banner. * Use Google Images to detect the software based on the service/web page logo. 2. Identify the software version. 3. Look for vulnerabilities in that version. * Search for vulnerabilities on **Metasploit** * Search for vulnerabilities on **Searchsploit** * Search for vulnerabilities on **Google** * Search for Proofs of Concept on **GitHub/GitLab**. 4. Look for software misconfiguration. 5. Try default credentials or use brute-forcing (Exploitation phase) * If some credentials are found, try them everywhere You can also use automatic tools to help you identify vulnerabilities in each service. But, you will have to check that the results do not contain false positives. Here there are some automatic tools for general services: * [Nessus](https://www.tenable.com/products/nessus/nessus-essentials) * [OpenVas](https://www.openvas.org) * [Nmap](https://nmap.org/nsedoc/categories/vuln.html) * [Acunetix](https://www.acunetix.com) * [Nuclei](https://nuclei.projectdiscovery.io) # Passive (OSINT) ## Introduction Open Source Intelligence (OSINT) could also be defined as a set of techniques and tools for gathering public information about a company, physical person, etc., analysing the collected data and correlating it, turning them into intelligence. In the "Information Gathering" phase of a pentesting OSINT tools can help the pentester in different aspects: * **Technical information**: Discover services, hosts, domains, subdomains, git repositories, credentials, social networks, operating system versions, network diagrams, etc. * **Human resources**: Gather information about a specific person on social networks, government records, telephone numbers, leaked credentials, metadata... As a goal, the before mentioned information can lead to different attacks: * Social engineering * Passwords Brute force Attacks * Target infiltration * Accounts take over * Identity theft ## The OSINT Process This sub-phase consists of concatenated processes that allow the attacker to obtain the required intelligence to perform the pentesting. The procedures are the following. 1. **Requirements**: What information do we need from the client as a starting point? 2. **Sources of information**: What sources of information can provide us with trusty information? 3. **Harvesting**: Retrieve data from the identified sources. 4. **Data processing**: Add format and process the obtained data, obtaining meaningful information. 5. **Data analysis**: Join data from multiple sources, producing intelligence. 6. **Reporting**: Create the final report. ## Sources of information and tools This section presents some tools and sources of information the pentester has to consider during the OSINT phase. However, an attacker does not have to rely only on these tools and must be aware of new tools that allow him to obtain further information and confirm the collected data. ### OSINT Framework [OSINT Framework](https://osintframework.com) is an interactive web page that contains a vast set of links to free and open-source tools for different purposes such as Domains, emails addresses, usernames, telephone numbers, public records, social network profiles... ![OSINT Framework](/files/7rvbq6nBUivO8j955ZLS) **Note**: Take into account that some sites will require registration or paying money to obtain extra data. As an alternative to the OSINT framework, there are several webs containing sets of references to different OSINT tools such as [Malfrat's OSINT Map](https://map.malfrats.industries), [osint.link](https://osint.link), [osinttechniques.com](https://www.osinttechniques.com) and [OSINT\_Collection](https://github.com/Ph055a/OSINT_Collection). ### Search Engines Search engines are an excellent option to conduct passive reconnaissance because they are populated with a ton of information that has been previously indexed. ### Google Dorks Google dorks is a technique that uses the google advanced search options for gathering efficient and precise data on any topic from any website on the internet. The Google search engine interprets search operators and commands for retrieving or accessing sensitive information that was knowingly or unknowingly released on the Internet like credentials, configuration files, documents... However, Google Search results might vary depending on the location or device used, so use it to your advantage depending on your target. Because Google Dorking can be pretty challenging, there are already tools that help you create Google queries for collecting specific data. ### Dorksearch [DorkSearch](https://dorksearch.com) is a tool that gives you a list of prebuilt templates for Google Dorks for different use cases. Furthermore, you can create your queries from the builder menu. ![](/files/JIotDHUdDkAMgU0lppzY) ### Google Hacking Database [Google Hacking Database (GHDB)](https://www.exploit-db.com/google-hacking-database) is an open-source project which stores and updates a set of know dorks. ![Google Hacking Database](/files/9EDi3tkdaswv2xarHqOp) Each dork is categorised in one of these categories: * **Footholds**: Searches that can provide a foothold on a server. * **Files containing usernames** * **Sensitive directories**: Searches for directories with sensitive information. * **Web server detection**: searches for web servers with a particular technology * **Vulnerable files** * **Vulnerable servers** * **Error messages**: Searches for error messages of a specific type. * **Files containing juicy info**: Searches for files containing important information * **Files containing passwords**: Searches for files with passwords * **Sensitive online shopping info**: Searches for sensitive information from online shopping sites * **Network or vulnerability data**: Searches for specific network information or vulnerabilities. * **Pages containing login portals**: Searches for pages containing particular login portals * **Various online devices**: Searches for specific online devices ### GitHub dorks GitHub dorks are pretty similar to google dorks. This can be very handy when searching for sensitive files, API keys, passwords, hidden URLs, employees, etc. To start with GitHub dorks, you can start from the GitHub Documentation: [Search on GitHub](https://docs.github.com/en/search-github) or posts like [GitHub Recon](https://web.archive.org/web/20220120021227/https://shahjerry33.medium.com/github-recon-its-really-deep-6553d6dfbb1f), [Developers are unknowingly posting their credentials online](https://medium.com/hackernoon/developers-are-unknowingly-posting-their-credentials-online-caa7626a6f84) and [GitHub for Bug Bounty Hunters](https://gist.github.com/EdOverflow/922549f610b258f459b219a32f92d10b). Here are some valuable examples, but if you want more, visit these links: [GitHub-dorks](https://github.com/techgaun/github-dorks/blob/master/github-dorks.txt) and [keywords](https://github.com/random-robbie/keywords/blob/master/keywords.txt). ```bash filename:.npmrc _auth filename:.dockercfg auth extension:pem private extension:ppk private filename:id_rsa or filename:id_dsa extension:sql mysql dump extension:sql mysql dump password filename:credentials aws_access_key_id filename:.s3cfg filename:wp-config.php filename:.htpasswd filename:.env DB_USERNAME NOT homestead filename:.env MAIL_HOST=smtp.gmail.com filename:.git-credentials language:python username language:php username language:sql username language:html password language:perl password language:shell username language:java api ``` Furthermore, the success of GitHub dorks has driven to the creation of third-party tools like [Gitrob](https://github.com/michenriksen/gitrob/) and the result of [GitHub Code Search](https://cs.github.com/about). ### Shodan Shodan is a search engine for directly accessible devices connected to the Internet, discovering devices like cameras, traffic lights, power plants. ![Shodan](/files/xOu9B3mvM2CQnuV5436t) Furthermore, it counts with several [filters](https://www.shodan.io/search/filters) obtaining devices with default credentials, vulnerable services, screenshots, cloud providers, location... However, the filters "tag" and "vuln" are not accessible with the free plan requiring an academic membership or a small business plan. Thanks to shodan, the pentester can create [shodan queries](https://www.shodan.io/search/examples) looking for physical assets that are part of the target company for a red teaming attack and vulnerable services exposed on the Internet that will allow them to get a foothold on the company's IT infrastructure. ### Archives.org When conducting OSINT research in these fast-paced digital times, analysts often need access to historical versions of websites or content that no longer exists. This is where The Wayback Machine comes into play. ![Archive.org](/files/jot0DavNB0Z1DGUumTu0) For instance, if you are looking to see historical versions of a website due to the site being deleted or replaced with new content, the Wayback Machine can help. You may need to verify that a target previously worked at a company, but the site's current state does not have the target’s information there. Furthermore, sometimes a target may intentionally hide information from their present website; looking at older dates of the site may reveal new information. Sometimes you can gather relevant data like names, phone numbers, email addresses, and even metadata from older website versions. **Methods** * **Quick Search Methods:** The quickest method to see all the files archived on a particular site is by accessing the following URL. ```bash https://web.archive.org/*/www.example.com/* ``` * **Advanced Search Method**: By directly visiting the archive [advanced search page](https://archive.org/advancedsearch.php), the attacker can perform more targeted searches and sometimes find the email address associated with a user who uploaded a file. However, this requires you to register on the platform. * **Change**: The [changes](https://web.archive.org/web/changes/) method identifies and displays changes in the content of archived URLs. However, please consider that it is still in beta nowadays, so it might not render both websites correctly. * **Save pages**: The Wayback machine accepts requests for a web page to be [archived](https://web.archive.org/save). However, this method is only available for web pages that allow crawlers. ## Email Harvesting Harvesting email addresses gives an attacker more information to conduct social engineering and password brute-forcing attacks. ### The Harvester [TheHarverster](https://github.com/laramies/theHarvester) is a command-line tool that is already installed in Kali Linux. The primary purpose of the harvester is pervasively gathering e-mail accounts, subdomain names, virtual hosts, open ports/ banners, and employee names from different public sources using various search engines, which has increased in variety for the past years. In recent versions, the authors added active techniques like the capability of doing DNS brute force, reverse IP resolution and Top-Level Domain. We can obtain many results with just a simple command, thanks to its simplicity. The results can be complementary with other tools like maltego. ```bash theHarvester -d -b [-l ] [-f ] ``` ### Maltego Maltego is one of the most powerful open source intelligence tools on the market; It is characterised by its intuitive handling and its representation of information based on graphs that connect information for investigative tasks. ![](/files/ot73pNz1wCYm5NTjzii5) Maltego is used to map the relationships between pieces of information named Entities, resulting from running transformations. * **Entities** are bits of information that we have obtained from a data source (a physical location, a website, a company name, an email address, a person’s name and a telephone number). * **Transformations** are small pieces of code that fetch related information for a given input and format the results to be returned as Entities to Maltego. However, be careful when running transformations because they can escalate too quickly, providing you with over information and turning into a gigantic graph populated with useless information. Finally, to use maltego, you need to create an account, then you have to choose between different [plans](https://www.maltego.com/pricing-plans/): Maltego Community Edition (CE); Case File; Classic; XL and One, being free the two formers. ### HaveIBeenPwned Once obtained information about a user like an email or phone, the pentester should check on sites like [haveibeenpwned.com](https://haveibeenpwned.com) whether there are compromised accounts that have suffered a data breach, meaning the email and password for that site’s account has been exposed to cybercriminals. ![HaveIBeenPwned](/files/yAkrIqaxkqavEbMhQpbG) A pentester could obtain these credentials for different means, checking if employees share passwords across multiple accounts in the company's environment. ### DeepSearch [DeepSearch](http://xjypo5vzgmo7jca6b322dnqbsdnp3amd24ybx26x5nxbusccjkm4pwid.onion/deepsearch) is a deep website, a successor from [pwndb2](https://pwndb2am4tzkvold.onion), where a pentester could obtain the credentials leaked in data breaches for free. ![DeepSearch](/files/2yk2Ins1yNGArqb7ZHfi) Another solution for obtaining leaked passwords could be [dehashed.com](https://www.dehashed.com/pricing), but it is not free. ### Recon-Ng [Recon-NG](https://github.com/lanmaster53/recon-ng) is a CLI framework installed on Kali Linux that conducts open-source web-based reconnaissance. It can be used for obtaining SQL injections, IPs, IP lookup, port scanning, sub-domain information, etc. Obtaining its maximum potential can be achieved using API keys that some modules require. However, they might be subscription-based. For setting the API keys, you need to execute the following commands. ```bash keys list keys add ``` You will also find that no modules are installed by default, but they are easily installed with these commands. Gives a list of all modules in the marketplace: ```bash marketplace search [] ``` Install an individual module: ```bash marketplace install ``` Install all modules in a category: ```bash marketplace install ``` Remove a module: ```bash marketplace remove recon/ports-hosts/ssl_scan ``` **Note**: Some of the modules will require Python dependencies to be installed outside of Recon-NG. Modules with external dependencies will have an asterisk in the D column of Marketplace results, and those requiring an API key will have an asterisk in the K column. In both cases, Recon-NG will warn you about missing dependencies and API keys after installation. Then, to run any module, you need to follow these steps: 1. If you want to take a deeper look at what a module does, you can use the marketplace info module followed by the module name or path. ```bash marketplace info ``` 1. Once you have decided which module you want to use, proceed with loading it using: ```bash modules load ``` 1. Find any module prerequisites. ```bash options list ``` 1. Set the options. ```bash options set