Often when performing the pentests, the client asks me to scan the external network (I hope that your customers are well aware of the risks, and besides scanning the external network they will give you access to the internal network, but this topic is a separate article).In the case of small organizations, I mostly use nmap at all stages of the scan.When it comes to large networks, try first to divide the scan procedure into separate stages:

1. General Network Scan: Find IPv4 addresses that have work services (search for “live” hosts).
2. Port scanning: search for open TCP and UDP ports on target systems.
4. Version detection: Determine the version of services and protocols that use open TCP and UDP ports.

If the range of external IP addresses is about 10 thousand units or less,nmap copes well with its work in the case of all the above needs.However, in large companies that own hundreds of thousands of IP addresses, the task of determining “live” hosts within a reasonable time (for example, within a few hours) becomes more complicated.By default, nmap sends several test requests.In case of failure, the host is marked as “dead”, and subsequent requests are not sent.We can not do a general network scan using the -Pn option, but then nmap will start checking all the specified ports for each IPv4 address.Since there are no running services on most external IPv4 addresses, a large network check can take weeks, months or even years.Thus, our task is to find an efficient way of searching for IP-addresses with working services.Next, the found list will already be checked in detail for specific ports and protocol versions.

Nmap has difficulties with a large range of addresses, because this utility operates on a synchronous scanner principle, monitors connection requests and waits for a response.If there is no response to the TCP connection request (SYN), the timeout expires, and nmap assigns the filtered status to the service (under the filter).Nmap runs several test requests in parallel, but services with filtered status (and inactive IP addresses) greatly slow the overall process.

In addition to utilities using synchronization, one of which is nmap, there are several asynchronous scanners that do not track connections:scanrand,ZMapand my favorite masscan.
I prefer to use masscan for several reasons.The first, and most important, reason is that the masscan syntax is very similar to nmap.Secondly, masscan is one of the fastest even among asynchronous scanners.With proper network interfaces and drivers, the efficiency of this scanner is limited by the width of your channel.When using two 10 Gigabit Ethernet adapters manufactured by Intel, you can scan the entire range of IPv4 addresses in six minutes, when 10 million packets are transmitted every second.  

First, consider the basic syntax of masscan when applied to scanning TCP ports over a wide range of networks (for example, 16 million IPv4 addresses used by Apple).

$ sudo masscan 17.0.0.0/8 -p0-1023

Scanning speed

By default, masscan will send 100 packets per second.In each packet, 18 bytes are allocated for the Ethernet header, 20 bytes for the TCP header, and more than 20 bytes for the IPv4 header.As a result, 5800 bytes (or about 46 kilobytes) per second are sent.Since the masscan, when scanning ports and hosts, distributes resources evenly, respectively, the band will also be distributed evenly.On a wide band, an unintentional DDOS attack may occur in the case of a small network scan, but problems should not arise at 1-10 megabits per second (or 20 thousand packets per second, -rate 20,000).On virtual machines, the speed can easily reach up to 200,000 packets per second (-rate 200,000), which is equivalent to 93 megabits per second of outgoing traffic.However, the need to use such high speeds should be agreed with the client.

Network Scanning

How can we determine that there are working TCP services on a specific IPv4 address?The easiest way is to scan 65536 ports (from 0 to 65535).However, in large networks this method will take too much time, even at high speeds.I usually choose 100 or 1000 most popular ports by nmap version.If the IP address responds to any SYN packet (in response, either RST is sent, which indicates a closed port / connection, or SYN-ACK, which indicates an open port / connection), we store this IP address in a separate list for subsequent analysis using nmap or, for example, the Nessus vulnerability scanner.
Use a small trick to get a list of the most popular ports.We will scan our own system and display the list of ports in XML format.The XML format displays the parameters used for scanning, and, more importantly, lists the ports in a human-readable form.I chose the first 100 ports, but you can easily change this value (for example, output the first 10 or 1000 ports).

$ nmap localhost –top-ports 100 -oX – |grep services
<scaninfo type = “connect” protocol = “tcp” numservices = “100” services = “7,9,13,21-23,25-26,37,53,79-81,88,106,110-111,113,119,135,139,143-144,179,199,389,427,443- 445.465.513-515.543-544.548.554.587.631.646.873,990.993,995,1025-1029,1110,1433,1720,1723,1755,1900,2000-2001,2049,2121,2717,3000,3128,3306,3389,3986,4899,5000,5009,5051, 5060.5101.5190.5357.5432.5631.5666.5800.5900.6000-6001.6646.7070.8000.8008-8009.8080-8081.8443.8888.9100,9999-10000.32768.49152- 49157 “/>

Next, we use the resulting list as values of the -p parameter when scanning the target range.As an example, we also use the Apple network.The speed of 100,000 packets per second is equivalent to 32 megabits per second.

$ sudo masscan 17.0.0.0/8 -oG apple-masscan.gnmap -p 7,9,13,21-23,25-26,37,53,79-81,88,106,110-111,113,119,135,139,143-144,179,199,389,427,443-445,465,513-515,543-544,548,554,587,631,646,873,990,993,995 , 1025-1029,1110,1433,1720,1723,1755,1900,2000-2001,2049,2121,2717,3000,3128,3306,3389,3986,4899,5000,5009,5051,5060,5101,5190 , 5357.5432.5631.5666.5800.5900.6000-6001.6646.7070.8000.8008-8009.8080-8081.8443.8888.9100,9999-10000.32768.49152-49157 –rate 100,000

Note that masscan supports the same -oG filename.gnmap option as nmap.Next, we will process the resulting list (in greppable format) for analyzing hosts with open ports.Scanning 16 million addresses for 100 ports will take about 5 hours.In my opinion, this time is quite reasonable.Consider the first few lines of the resulting file:

# Masscan 1.0.3 scan initiated Thu Jul 20 22:24:40 2017
# Ports scanned: TCP (1; 7-7,) UDP (0;) SCTP (0;) PROTOCOLS (0;)
Host: 17.179.241.56 ( ) Ports: 443 / open / tcp ////
Host: 17.253.84.72 () Ports: 179 / open / tcp ////
Host: 17.188.161.148 () Ports: 8081 / open / tcp ////
Host: 17.188.161.212 () Ports: 8081 / open / tcp ////

Since we only need IP addresses, we will use the egrep utility to find strings starting with “Host:” and then process the data with the cut utility to extract the second field.We will also sort the list using the sort utility and remove duplicates with uniq.

$ egrep ‘^ Host:’ apple-masscan.gnmap |cut -d “” -f2 |sort |uniq> apple-alive

Thus, our list has become significantly smaller, and further we can apply nmap for more detailed analysis:

# nmap -PN -n -A -iL apple-alive -oA apple-nmap-advanced-scan

Now, based on the file generated by the masscan, the nmap scanner will be able to do its job much faster.

Write in the comments, whether you helped my method in the work.In general, I like nmap, but sometimes it is better to perform more intensive tasks with more specialized utilities. 

(Cyberops, Cyberops Infosec, VAPT, Cyber Security, Ethical Hacking, Secured Application Development)