# External recon

> [`recon-ng`](https://github.com/lanmaster53/recon-ng) is an open source intelligence (OSINT) framework designed to provide "a powerful environment to conduct open source \[reconnaissance] quickly and thoroughly". `recon-ng` provides a layer of abstraction for numerous `APIs` that can be leveraged for external information gathering (domains and subdomains enumeration, reverse `DNS` lookups, searches in services and vulnerabilities datasets, etc.).\
> \
> `recon-ng` is built around modules, that will be referenced in the adequate sections of the present note.<br>

```bash
# Sets the USER-AGENT sent by recon-ng (by default "Recon-ng/vX") for a more OPSEC friendly approach.

options set USER-AGENT "<USER_AGENT_STRING>"

--------------------------------------------------------------------------------

# Workspaces: projects that will hold the related domains, hosts, ports, etc..
# Each workspace will be stored as a SQLite database on the filesystem.

# Creates a "workspace".
workspaces create <WORKSPACE_NAME>

# List the existing workspaces.
workspaces list

# Switches to the specified workspace.
workspaces load <WORKSPACE_NAME>

# Removes the specified workspace.
workspaces remove <WORKSPACE_NAME>

--------------------------------------------------------------------------------

# API keys operations.
# Each module lists the API key(s) it requires.

# Adds the specified key.
# Key names examples: bing_api, github_api, google_api, ipinfodb_api, shodan_api, spyse_api virustotal_api, whoxy_api, etc.
keys add <KEY_NAME> <KEY_VALUE>

# Removes the specified key.
keys remove <KEY_NAME>

# List the configured keys.
keys list

--------------------------------------------------------------------------------

# Database operations (adding / removing targets, listing current results, etc.).
# Supported tables (as of recon-ng v5.1.2): companies, contacts, credentials domains, hosts, leaks, locations, netblocks, ports, profiles, pushpins, repositories, and vulnerabilities.
# "companies" table: name of the companies to target (for Whois / ASN research for instance).
# "domains" table: domains to be targeted.
# "hosts" table: hosts enumerated (hostname and IP address information notably).
# "ports" table: ports enumerated, including information on the host, the protocol / service, etc.

# Displays the schema of the current's workspace database.
db schema

# Lists the values stored in the specified table (including their rowid, needed for various operations).
show <domains | hosts | ports | TABLE_NAME>
db query SELECT rowid, * FROM <domains | hosts | ports | TABLE_NAME>;

# Adds the specified entry in the given table.
db insert companies <COMPANY_NAME>~ ~
db insert domains <DOMAIN>~
db insert netblocks <CIDR>~
[...]

# Removes the specified domain from the "domains" table.
db delete domains <ROWID>

# Removes all entries from the specified table.
db query DELETE FROM <TABLE>;

--------------------------------------------------------------------------------

# The modules of the recon-ng framework are not provisioned / installed by default but are made available from the "Recon-ng Marketplace" (https://github.com/lanmaster53/recon-ng-marketplace).

# Lists all the modules available.
marketplace search

# Searches among the available modules for the specified keyword(s).
marketplace search <KEYWORD(S)>

# Retrieves information about all or the specified module(s) (description, last update date, required API keys and dependencies, etc.).
marketplace info <all | MODULE_PATH>

# Install all or the specified module.
marketplace install <all | MODULE_PATH>

--------------------------------------------------------------------------------

# The modules will usually require a <SOURCE> input.
# By default the source will be all the data from a recon-ng's table, but can be specified to be a single element, a file, or an SQL query to extract specific data.

# Lists the modules currently installed.
modules search

# Searches among the installed modules for the specified keyword(s).
modules search <KEYWORD(S)>

# Loads the specified module.
modules load <MODULE_PATH>

# Displays the help of the current module.
[recon-ng][<WORKSPACE>][<MODULE>] > info

# If required, set the <SOURCE> input for the current module.
[recon-ng][<WORKSPACE>][<MODULE>] > options set SOURCE <SINGLE_ELEMENT | FILE | SQL_QUERY>

# Execute the current module.
[recon-ng][<WORKSPACE>][<MODULE>] > run

--------------------------------------------------------------------------------

# Specific modules are designed for the importing / exporting of results.
# Input modules: import/csv_file, import/list, import/masscan, and import/nmap.
# Notable exporting modules: reporting/csv, reporting/list, reporting/json, reporting/html, and reporting/xlsx.

# Import the data in the file in the specified table's column.
# For example, to import subdomains: <TABLE> = hosts & <COLUMN> = host.
modules load import/list
[recon-ng][<WORKSPACE>][list] > options set FILENAME <FILENAME>
[recon-ng][<WORKSPACE>][list] > options set TABLE <TABLE>
[recon-ng][<WORKSPACE>][list] > options set COLUMN <COLUMN>
[recon-ng][<WORKSPACE>][list] > run
```

### Domain enumeration

**\[Passive] Initial domain enumeration**

Manual searches using search engines, such as `Google` or `Bing`, should first be conducted to identify a list of domains linked to the targeted entity.

The goal of the research is to:

* Gather an initial list of the domain directly linked to the targeted entity.
* Identify the possible subsidiaries of the entity and gather information about their associated domains.

Once the main domain names are identified:

* Queries to `Whois` records can be done to retrieve information about the `registrant` (registered holder of the domain) or the `registrar` (accredited organization that registers a domain on behalf of the `registrant`) of the domains.

  The `whois` Linux utility can be used to retrieve the `Whois` record of a specified domain:

  ```bash
  whois <DOMAIN>

  # Bash script to retrieve the Registrant of each domains (one by line) in the file given as input.

  #!/bin/bash
  while IFS= read -r domain; do
      registrant=$(whois $domain | grep "Registrant Organization" | cut -d ":" -f 2 | awk '{$1=$1};1')
      echo "\"$domain\",\"$registrant\""
  done < "$1"
  ```

  Additionally, the following online services can be used to make `Whois` queries:

  | Service                      | URL                              | Description                                                                                                                                                                                                                                                                                                                              |
  | ---------------------------- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | `who.is`                     | <https://who.is/>                | Free service through a web interface.                                                                                                                                                                                                                                                                                                    |
  | `Whoxy` Whois Lookup         | <https://www.whoxy.com/>         | <p>Free <code>Whois</code> queries through the web interface and paid <code>API</code> (with a low pricing: 1,000 Domain <code>Whois</code> <code>API</code> queries for 2$).<br><br><code>recon-ng</code>'s module (requires a <code>whoxy\_api</code> <code>API</code> key):<br><code>recon/domains-companies/whoxy\_whois</code>.</p> |
  | `DomainTools`'s Whois lookup | <https://whois.domaintools.com/> | Free service through a web interface.                                                                                                                                                                                                                                                                                                    |
* The `IP` address(es) associated with the domains can be enumerated through `DNS` resolutions (using utilities such as `dig`, `host`, or `nslookup`) or using proprietary databases. Note that more than one `IP` can be associated with a domain name (through `DNS` `A` or `AAAA` records) and result may vary depending on client `GeoIP` data, etc.

  ```bash
  host <DOMAIN>

  # Bash script to resolve one IP of each domains (one by line) in the file given as input.

  #!/bin/bash
  while IFS= read -r domain; do
      IP=$(dig +short $domain | sed ':a;N;$!ba;s/\n/ /g')
      echo "$domain,$IP"
  done < "$1"
  ```

  The `recon-ng`'s `recon/hosts-hosts/resolve` module can be used to resolve domain names (for the `hosts` table by default) using the nameserver configured with-in the framework (`Google`'s `DNS` 8.8.8.8 by default).

  Refer to the `[L7] DNS - Methodology` note for more information on `DNS` resolution if needed.

**\[Passive] Reverse Whois search**

Reverse research in `Whois` records consist of researching information related to the `registrant` (name or email address for example) in proprietary `Whois` records dataset. Such research can be used to enumerate the others domains possibly registered by the same `registrant`.

A number of online services can be used to make reverse `Whois` queries.

| Service                              | URL                                                | Description                                                                                                                                                                                                                                                                                                                                                                   |
| ------------------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ViewDNS.info`                       | <https://viewdns.info/reversewhois>                | <p>Free web interface, allows search based on company / person names or email addresses.<br><br><code>recon-ng</code>'s module:<br><code>recon/companies-domains/viewdns\_reverse\_whois</code>.<br><br>If <code>403 Forbidden</code> errors are returned by the <code>API</code>, the <code>recon-ng</code>'s <code>User-Agent</code> should be set to a legitimate one.</p> |
| `reversewhois.io`                    | <https://www.reversewhois.io>                      | Free web interface, allows search based on names or email addresses.                                                                                                                                                                                                                                                                                                          |
| `drs.WhoisXMLAPI.com`                | <https://drs.whoisxmlapi.com/reverse-whois-search> | Paid service, with limited `API` credit upon signup.                                                                                                                                                                                                                                                                                                                          |
| `DomainTools`'s reverse Whois lookup | <https://reversewhois.domaintools.com/>            | Paid service, with the most comprehensive results. The number of results returned by a query can be consulted freely but the retrieval of the domain names is subject to charge.                                                                                                                                                                                              |

**\[Passive] Reverse DNS lookup and IP sharing**

Reverse `DNS` lookup consist of retrieving, in proprietary datasets, the domain(s) (or subdomain(s)) associated to a given `IP` address. It can be used to identify the domains or subdomains sharing a common `IP` address (and possibly owned by the targeted entity).

A number of online services can be used to make reverse `DNS` lookup to identify `IP` sharing.

| Service                                     | URL                                                                                                                                           | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `api.hackertarget.com`                      | <https://api.hackertarget.com/reverseiplookup/?q=>                                                                                            | <p>Website and <code>API</code>, limited to 20 queries / day in the free tier (Python script provided below).<br><br><code>recon-ng</code>'s module:<br><code>recon/hosts-hosts/hackertarget\_reverse</code>.</p>                                                                                                                                                                                                                                                                                                                           |
| `host.io`                                   | <https://host.io/ip/>                                                                                                                         | <p>Free queries through the web interface with limited result.<br><br>Premium API, with a free plan: 1000 requests / month with max 5 results per page (i.e 500 results would require 100 requests).</p>                                                                                                                                                                                                                                                                                                                                    |
| `ThreatCrowd`                               | <https://www.threatcrowd.org/ip.php?ip=>                                                                                                      | <p>Free queries through a web interface and <code>API</code>.<br><br>The web interface displays results in the form of a graph, including additional results such as reverse <code>DNS</code> or <code>Whois</code> lookups.</p>                                                                                                                                                                                                                                                                                                            |
| `WhoisXMLAPI`'s Reverse `IP` / `DNS` lookup | <https://reverse-ip.whoisxmlapi.com/lookup>                                                                                                   | Website and `API`, with 100 free `API` calls upon signup.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `Bing` search engine                        | <p>Search queries using:<br><br><code>ip:\<IP></code><br><code>ip:\<IP> -site:\<EXCLUDE\_DOMAIN1> -site:\<EXCLUDE\_DOMAIN2> \[...]</code></p> | <p><strong>Web browsing</strong><br><br>Free and unlimited but requires manual and time-consuming browsing. Search requests are additionally limited to 1000 characters.<br><br>The following bash one-liner can be used to generate a exclude <code>-site:</code> string from a file contaning domain names:<br><br><code>while IFS= read -r line; do echo -n "-site:$line "; done < "$1"</code><br><br><strong>Bing API</strong><br><br>1,000 requests free per month, with a Microsoft Azure account (requires credit card details).</p> |

The following Python script leverage the `api.hackertarget.com` reverse `IP` lookup `API` to retrieve the `DNS` records associated with each `IP` in the specified `IP` ranges:

```
import ipaddress
import requests
import sys

API_URL = 'https://api.hackertarget.com/reverseiplookup/'

if (len(sys.argv) < 2):
  print("Usage: reverse_dns.py <IP_FILE>")
  exit(1)

IP_file = open(sys.argv[1], 'r')
IP_ranges = IP_file.read().splitlines()

for IP_range in IP_ranges:
  for IP in ipaddress.IPv4Network(IP_range):
  	r = requests.get(url = API_URL, params = {'q':IP})
  	if 'No DNS A records found' in r.text:
  	  continue
  	for record in r.text.splitlines():
  	  print(f'{IP},{record}')
```

**\[Passive] Leveraging Autonomous System Number**

An `Autonomous System Number (ASN)` is a unique number assigned to an `Autonomous System (AS)` by the `Internet Assigned Numbers Authority (IANA)`. An `AS` consists of blocks of `IP` addresses which have a distinctly defined policy for accessing external networks and are administered by a single organization (which may not be the targeted entity but an operator having the entity as a client).

*ASNRECON*

[`ASNRECON`](https://github.com/orlyjamie/asnrecon) is a Python script that:

* retrieve the `ASN` of a given domain,
* lookup the `IP` addresses / ranges part of the `ASN` (in the dataset downloaded from the [`pyasn`](https://github.com/hadiasghari/pyasn) project),
* and finally attempt to access, for each enumerated `IP`, an `HTTPS` service (on port `TCP` 443) to extract the `subject` defined in the `SSL` / `TLS` certificate.

Note that result may not usable if the `ASN` is not managed by the targeted entity.

```bash
# As asnrecon.py requires Python2 and a number of dependencies, it is recommended to use the Docker file provided in the repository to  build a Docker container.
docker build -f Dockerfile -t asnrecon .

# Either chose 1. to scan by domain name or 2. to only conduct the SSL / TLS subject name extraction on a given IP range.
docker run -it asnrecon
```

*ASN Lookup*

The [`asn`](https://github.com/nitefood/asn) Bash script can be used, among other features, to lookup `ASN` by `organization name` in order to retrieve the `IP` ranges linked to an entity.

Reverse `DNS` lookups, `SSL` / `TLS` certificates grabbing, etc. can then be performed on the enumerated `IPs` to retrieve additional domain names of the entity.

```
asn -o "<ENTITY_NAME>"
```

### Subdomains enumeration

**\[Passive] Search engines and passive DNS proprietary dataset**

Search engines, such as `Google`, `Bing`, etc., and proprietary databases (with historical data) operated by services such as `DNSdumpster`, `VirusTotal`, `DomainTools`, can be used to retrieve subdomains associated with a domain name.

| Service                                                                                        | URL / query                                                                                                                                     | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                      |               |
| ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | ------------- |
| `Rapid7`'s `Project Sonar`                                                                     | <https://opendata.rapid7.com/sonar.fdns\\_v2/>                                                                                                  | <p>massive offline dataset (hundreds of GB uncompressed) of domains and subdomains.<br><br>Gathered using various sources and using direct <code>DNS</code> resolutions queries (<code>ANY</code>, <code>A</code>, <code>AAAA</code>, <code>CNAME</code>, and <code>TXT</code> record lookups).<br><br>Offline query example (without extraction to disk to save disk space but which require new extraction for each search):<br><code>pigz -dc \<SONAR\_GZ>                                                                                                           | grep -F '\<DOMAIN>"' | jq</code></p> |
| <p><code>DNSdumpster</code> (web GUI)<br>/<br><code>HackerTarget</code> (<code>API</code>)</p> | <p><https://dnsdumpster.com/><br><br><https://hackertarget.com/find-dns-host-records/></p>                                                      | <p>Passive <code>DNS</code> service (with historical data) accessible using a web interface or an <code>API</code> through, respectively, <code>DNSdumpster</code> and <code>HackerTarget</code>.<br><br><code>recon-ng</code>'s module (for <code>HackerTarget</code>):<br><code>recon/domains-hosts/hackertarget</code>.</p>                                                                                                                                                                                                                                          |                      |               |
| `VirusTotal`                                                                                   | <https://www.virustotal.com/gui/home/search>                                                                                                    | <p>Initiated as an online scan engine relying on multiple anti-virus products, <code>VirusTotal</code> has a subdomains dataset (a priori based on historical lookups of files or URL submitted for scanning by users).<br><br>The free <code>API</code> is limited to 500 requests per day and a rate of 4 requests per minute.<br><br><code>recon-ng</code>'s module (requires a <code>virustotal\_api</code> <code>API</code> key):<br><code>recon/hosts-hosts/virustotal</code>.</p>                                                                                |                      |               |
| `SPYSE`                                                                                        | <https://spyse.com/tools/subdomain-finder>                                                                                                      | <p>Unlimited free search through the web interface with however heavy restrictions: up to 20 results and unexportable results.<br><br><code>API</code> available in the free tier, limited to 100 requests each month (with a maximum of 100 results per request).<br><br><code>recon-ng</code>'s module (requires a <code>spyse\_api</code> <code>API</code> key):<br><code>recon/domains-hosts/spyse\_subdomains</code>.</p>                                                                                                                                          |                      |               |
| `ThreatCrowd`                                                                                  | <https://www.threatcrowd.org/domain.php?domain=>                                                                                                | <p>Free queries through a web interface and <code>API</code>.<br><br>The web interface displays results in the form of a graph, including additional results such as reverse <code>DNS</code> or <code>Whois</code> lookups.<br><br><code>recon-ng</code>'s module:<br><code>recon/domains-hosts/threatcrowd</code>.</p>                                                                                                                                                                                                                                                |                      |               |
| `Google` search engine                                                                         | <p>Google dorks:<br><br><code>site</code><br><code>site:\<DOMAIN> -site:\<EXCLUDED\_SUBDOMAIN1> -site:\<EXCLUDED\_SUBDOMAIN2> \[...]</code></p> | <p>Free and unlimited (with eventual <code>reCAPTCHA</code> protection) but requires manual and time-consuming browsing. Search requests are additionally limited to 32 words.<br><br><code>recon-ng</code>'s module (which may fail du to CAPTCHA):<br><code>recon/domains-hosts/google\_site\_web</code>.</p>                                                                                                                                                                                                                                                         |                      |               |
| `Bing` search engine                                                                           | <p>Bing dorks:<br><br><code>site</code><br><code>site:\<DOMAIN> -site:\<EXCLUDED\_SUBDOMAIN1> -site:\<EXCLUDED\_SUBDOMAIN2> \[...]</code></p>   | <p>Similar to <code>Google</code> dorks.<br><code>API</code> available, with 1,000 requests free per month for authenticated users (Microsoft Azure account, which requires credit card details).<br><br><code>recon-ng</code>'s module that scrape <code>Bing</code> search results (no <code>API</code> key requied):<br><code>recon/domains-hosts/bing\_domain\_web</code>.<br><br><code>recon-ng</code>'s module for the <code>Bing API</code> (requires a <code>bing\_api</code> <code>API</code> key):<br><code>recon/domains-hosts/bing\_domain\_api</code>.</p> |                      |               |

**\[Passive] SSL / TLS certificates passive search**

<https://crt.sh/> <https://transparencyreport.google.com/https/certificates>

`recon-ng`'s module `recon/domains-hosts/certificate_transparency`.

**\[Active] SSL / TLS certificates grabbing**

```
nmap -v -sT -T 2 -Pn -p 443 -sV -sC -oA <OUTPUT_FILES> [-iL <INPUT_FILE> | <HOST | IP | CIDR | IP_RANGE>
```

**\[Active] Forward / reverse DNS brute force**

Forward `DNS` lookup brute force consist of attempting to guess valid subdomains through `DNS` resolution. In addition, if the `DNS` `PTR records`, used for mail services, are configured for the domain, reverse lookup brute force may possible. Both actions require direct interactions with the `DNS` nameservers of the targeted entity.

Refer to the `[L7] DNS - Methodology` note for techniques and tooling to conduct forward / reverse `DNS` brute forcing.

**\[Passive / Active] Automated enumeration tools**

Multiple tools can be used to automate the process of passively enumerating subdomains using public resources. While the tools introduced below generally produce fairly similar results, slight differences may arise and executing each tool can ensure a more comprehensive enumeration.

| Tool                                                      | Description                                                                                                                                                                                                                                                                                                                                                                                                 |
| --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`Amass`](https://github.com/OWASP/Amass)                 | Go project retrieving data from [numerous sources](https://github.com/OWASP/Amass/blob/master/README.md).                                                                                                                                                                                                                                                                                                   |
| [`assetfinder`](https://github.com/tomnomnom/assetfinder) | Go project retrieving data from `crt.sh`, `Certspotter`, `HackerTarget`, `ThreatCrowd`, the Wayback Machine, and `dns.bufferover.run`, as well as `Facebook`, `VirusTotal`, and `SPYSE` if `API` keys are provided.                                                                                                                                                                                         |
| [`Sublist3r`](https://github.com/aboul3la/Sublist3r)      | <p>Python script retrieving data from:<br>- Search engines (<code>Google</code>, <code>Bing</code>, <code>Yahoo</code>, <code>Baidu</code>, <code>Ask</code>, and <code>Netcraft</code>).<br>- Proprietary <code>DNS</code> datasets (<code>DNSdumpster</code>, <code>VirusTotal</code>, and <code>ThreatCrowd</code>)<br>- <code>SSL</code> / <code>TLS</code> certificates using <code>crt.sh</code>.</p> |

*Amass*

```
# Execution of Amass on a single domain.
amass enum -passive -d <DOMAIN>

# Uses of interlace to multi-thread the execution of Amass on multiples domain.
echo 'amass enum -passive -d _target_ > _output_/_cleantarget_-amass.txt' > Amass_cmd_file.txt
interlace -tL <INPUT_DOMAIN_FILE> -o <OUTPUT_FOLDER> -cL Amass_cmd_file.txt
```

*assetfinder*

```
# Execution of assetfinder on a single domain.
assetfinder <DOMAIN>

# Uses of interlace to multi-thread the execution of assetfinder on multiples domain.
echo 'assetfinder _target_ > _output_/_cleantarget_-assetfinder.txt' > assetfinder_cmd_file.txt
interlace -tL <INPUT_DOMAIN_FILE> -o <OUTPUT_FOLDER> -cL assetfinder_cmd_file.txt
```

*Sublist3r*

```
# Execution of Sublist3r on a single domain.
python sublist3r.py -d <DOMAIN>

# Uses of interlace to multi-thread the execution of Sublist3r on multiples domain.
echo 'python3 sublist3r.py -n -d _target_ | grep _target_ | grep -v "Enumerating subdomains now for" > _output_/_cleantarget_-sublist3r.txt' > Sublist3r_cmd_file.txt
interlace -tL <INPUT_DOMAIN_FILE> -o <OUTPUT_FOLDER> -cL Sublist3r_cmd_file.txt
```

### IPs and services exposure

**Shodan**

TODO

### Code repository enumeration and research

TODO

### Employees contacts gathering

TODO

### Leaked credentials

**\[Passive] Dehashed**

[`dehashed.com`](https://dehashed.com/) is a website that indexes multiple billions of credentials leaked in various data breaches and datasets. It allows searches to be conducted by domain name, username, emails, name, password, etc.

While leaked email entries can be consulted freely, `dehashed.com` requires a paid subscription to view the password or hash associated with an entry. Additional `API` credits are also required to programmatically retrieve credentials. As of the end of 2021, the prices are:

* 5.49$ for a one week access, 15.49$ monthly or 179.99$ annually.
* 2.5$ for 100 API requests (with a maximum of 10 000 results by request).

The `API` request can be used to retrieve the entries with an email containing the specified domain:

```bash
# Example of a valid <EMAIL_DOMAIN>: test.com.
# Searches can also be conducted using the username, password, name, ip_address keywords.

curl -o <OUTPUT_JSON_FILE> 'https://api.dehashed.com/search?query=email:<EMAIL_DOMAIN>&size=10000' \
-u <EMAIL>:<API_KEY>  \
-H 'Accept: application/json'
```

The following Python snippet convert the `JSON` file produced by the `API` request above to a `CSV` file:

```python
import json
import csv

json_file = open(r'<JSON_INPUT_FILE>', encoding="utf8")
csv_file = open(r'<CSV_OUTPUT_FILE>', 'w', encoding="utf8", newline='')

dehashed = json.load(json_file)

csv_writer = csv.DictWriter(csv_file, quoting=csv.QUOTE_ALL, fieldnames = ["email", "password", "hashed_password", "username", "name", "database_name", "ip_address", "address", "phone", "vin", "id"])
csv_writer.writeheader()

for entry in dehashed['entries']:
    csv_writer.writerow(entry)

json_file.close()
csv_file.close()
```

### Username enumeration and password bruteforce / spraying

**Externally facing Exchange server**

If an Internet facing `Outlook Web Access (OWA)` / `Exchange Web Services` / `Exchange Active Sync (EAS)` portal is identified, it may be possible to validate usernames (in a time-based attack) or to bruteforce credentials.

* `OWA` is a browser client web application designed for users to access their mailboxes. Allows to both validate usernames and conduct bruteforcing attacks.
* `EWS` is a non-user facing service allowing applications to communicate with the `Exchange` server. Can be leveraged for (faster) bruteforcing attacks.
* `EAS` is a proprietary protocol designed for the synchronization of email, contacts, calendar (etc.) from Exchange servers. Allows to both validate usernames and conduct bruteforcing attacks. The portal is usually exposed at `https://<EXCHANGE_SERVER>/Microsoft-Server-ActiveSync/`.

Note that the usernames validation technique against `OWA` / `EAS` rely on a time difference between response time for authentication attempts with a valid or invalid username. An unsuccessful authentication will thus be triggered for any valid account found. As the accounts are (most likely) subject to the Active Directory domain password policy, precautions should be taken to avoid locking out the accounts by limiting the enumeration / password guessing attempts.

The PowerShell [`MailSniper`](https://github.com/dafthack/MailSniper) toolkit and the [`SprayingToolkit`](https://github.com/byt3bl33d3r/SprayingToolkit) Python script can be used to conduct password spraying attacks against `OWA` / `EWS` portals.

```bash
# Determines a the domain name associated with the Exchange instance.
Invoke-DomainHarvestOWA -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP>

# Attempts to valid usernames through a time-based attack.
Invoke-UsernameHarvestOWA -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -Domain <DOMAIN_NAME> -UserList <USER_FILE> -OutFile <OUTPUT_FILE>
Invoke-UsernameHarvestEAS -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -Domain <DOMAIN_NAME> -UserList <USER_FILE> -OutFile <OUTPUT_FILE>

# According to tests performed by the MailSniper author, bruteforcing through EWS is significantly faster than bruteforcing through OWA / EAS.
Invoke-PasswordSprayOWA -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -UserList <USER_FILE> -Password <PASSWORD>
Invoke-PasswordSprayEAS -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -UserList <USER_FILE> -Password <PASSWORD>
Invoke-PasswordSprayEWS [-ExchangeVersion <Exchange2013_SP1 | EXCHANGE_VERSION>] -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -UserList <USER_FILE> -Password <PASSWORD>
```

If valid credentials are found, `MailSniper`'s `MailSniper` cmdlet can be used to harvest email addresses. The cmdlet will first attempt to connect to an `OWA` portal to use the `FindPeople` method (available starting from `Exchange2013`). If this attempt does not succeed, the cmdlet will attempt to retrieve the `Global Address List` over `EWS`.

```bash
Get-GlobalAddressList -ExchHostname <EXCHANGE_HOSTNAME | EXCHANGE_IP> -UserName <DOMAIN>\<USERNAME> -Password <PASSWORD> -OutFile <OUTPUF_FILE>
```

***

### References

<https://github.com/appsecco/the-art-of-subdomain-enumeration>

<https://www.whatismyip.com/asn/>

<https://www.twelve21.io/getting-started-with-recon-ng/>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://notes.qazeer.io/general/external_recon.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.