Understanding Network Logs In Cybersecurity

understanding network logs in cybersecurity

The tcpdump command-line tool is a very popular packet analyzer. It can display packet captures in real-time or write packet captures to a file. It captures detailed packet protocol and content data. Wireshark is a GUI built on tcpdump functionality.

The structure of tcpdump captures varies depending on the protocol captured and the fields requested.

25.3.2

NetFlow

NetFlow is a protocol that was developed by Cisco as a tool for network troubleshooting and session-based accounting. NetFlow efficiently provides an important set of services for IP applications, including network traffic accounting, usage-based network billing, network planning, security, Denial-of-Service monitoring capabilities, and network monitoring. NetFlow provides valuable information about network users and applications, peak usage times, and traffic routing.

 

NetFlow does not do a full packet capture or capture the actual content in the packet. NetFlow records information about the packet flow including metadata. Cisco developed NetFlow and than allowing it to be used as a basis for an IETF standard called IPFIX. IPFIX is based on Cisco NetFlow Version 9.

 

NetFlow information can be viewed with tools such as the nfdump. Similar to tcpdump, nfdump provides a command-line utility for viewing NetFlow data from the nfcapd capture daemon, or collector. Tools exist that add GUI functionality to viewing flows. The figure shows a screen from the open-source FlowViewer tool.

FlowViewer NetFlow Session Data Dashboard

The figure show graphs on the left and right and tables in the middle. The tables in the middle are labeled left top, right top, left middle top, etc. and can be used to select the graphs shown. Buttons up top are flow viewer, flow tracker, and flow grapher.

Traditionally, an IP Flow is based on a set of 5 to 7 IP packet attributes flowing in a single direction. A flow consists of all packets transmitted until the TCP conversation terminates. IP Packet attributes used by NetFlow are:

  • IP source address
  • IP destination address
  • Source port
  • Destination port
  • Layer 3 protocol type
  • Class of Service
  • Router or switch interface

All packets with the same source/destination IP address, source/destination ports, protocol interface and class of service are grouped into a flow, and than packets and bytes are tallied. This methodology of fingerprinting or determining a flow is scalable because a large amount of network information is condensed into a database of NetFlow information called the NetFlow cache.

All NetFlow flow records will contain the first five items in the list above, and flow start and end timestamps. The additional information that may appear is highly variable and can be configured on the NetFlow Exporter device. Exporters are devices that can be configured to create flow records and transmit those flow records for storage on a NetFlow collector device. An example of a basic NetFlow flow record, in two different formats, is shown in the figure.

Simple NetFlow v5 Records

Date     flow start      Duration  Proto Src IP Addr:Port     Dst IP Addr:Port  Flags Tos Packets Bytes Flows2017-08-30 00:09:12.596  00.010    TCP   10.1.1.2:80      -> 13.1.1.2:8974     .AP.SF  0   62      3512   1
Traffic Contribution: 8% (3/37)Flow information:IPV4 SOURCE ADDRESS:10.1.1.2IPV4 DESTINATION ADDRESS:13.1.1.2INTERFACE INPUT:Se0/0/1TRNS SOURCE PORT:8974TRNS DESTINATION PORT:80IP TOS:0x00IP PROTOCOL:6FLOW SAMPLER ID:0FLOW DIRECTION:Inputipv4 source mask:/0ipv4 destination mask:/8counter bytes:205ipv4 next hop address:13.1.1.2tcp flags:0x1binterface output:Fa0/0counter packets:5timestamp first:00:09:12.596timestamp last:00:09:12.606ip source as:0ip destination as:0

A large number of attributes for a flow are available. The IANA registry of IPFIX entities lists several hundred, with the first 128 being the most common.

Although NetFlow was not initially conceived as tool for network security monitoring, it is seen as a useful tool in the analysis of network security incidents. It can be used to construct a timeline of compromise, understand individual host behavior, or to track the movement of an attacker or exploit from host to host within a network. The Cisco/Lancope Stealthwatch technology enhances the use of NetFlow data for NSM.

25.3.3

Application Visibility and Control

The Cisco Application Visibility and Control (AVC) system, which is shown in the figure, combines multiple technologies to recognize, analyze, and control over 1000 applications. These include voice and video, email, file sharing, gaming, peer-to-peer (P2P), and cloud-based applications. AVC uses Cisco next-generation network-based application recognition version 2 (NBAR2), also known as Next-Generation NBAR, to discover and classify the applications in use on the network. The NBAR2 application recognition engine supports over 1000 network applications.

To truly understand the importance of this technology, consider the figure. Identification of network applications by port provides very little granularity and visibility into user behavior. However, application visibility through the identification of application signatures identifies what users are doing, whether it be teleconferencing or downloading movies to there phones.

The figure has 4 columns. The leftmost column has a router with a magnifying glass on top of it: Application recognition identify applications using L3 to L7 data 1000+ applications: cloud services, Cisco Web ex, You tube, Skype, P 2 P. In italics is N bar 2. The next column is a graphic of charts, metrics collection, collect metrics for export to management tool: bandwidth usage, response time, latency, packet loss, jitter, P 2 P, and the following in italics: net flow 9, flexible net flow, and i p fix. The third column has a router with the words management and reporting tools: management and reporting provision the network, collect data, and report on applications performance: report generation and policy management, and in italics Cisco Prime and other 3rd party software. The last column has a router with a red light beside it, high: v o i p, medium browsing, low streaming, and blocked p 2 p. Words under are control control application use to maximize network performance: application prioritization and a

Cisco Application Visibility and Control

A management and reporting system, such as Cisco Prime, analyzes and presents the application analysis data into dashboard reports for use by network monitoring personnel. Application usage can also be controlled through the quality of service classification and policies based on the AVC information.

The figure shows on the left port monitoring with applications down the side of unknown, h t t p, h t t p s, i c a, s i p, d n s, c i f s, h s r p, i c m p, l d a p, m s n p, and s a p. Horizontal bars go out from each app with the longest bar up top beside unknown and the next largest bar beside h t t p and these have a dotted box around them. In the application monitoring section, apps are listed on the left with a horizontal bar beside each one. The longest horizontal bar is with the first app listed with each horizontal bar that follows being smaller in size. Apples: BitTorrent, net flix, share point, gtalk v o i p, google docs, r t p, citrix, s s l, s i p, skype, web ex meeting, h t t p s, flash video, d n s, and face book.

Port Monitoring vs. Application Monitoring

Content Filter Logs

Devices that provide content filtering, such as the Cisco Email Security Appliance (ESA) and the Cisco Web Security Appliance (WSA), provide a wide range of functionalities for security monitoring. Logging is available for many of these functionalities.

The ESA, for example, has more than 30 logs that can be used to monitor most aspects of email delivery, system functioning, antivirus, antispam operations, and blacklist and whitelist decisions. Most of the logs are stored in text files and can be collected on syslog servers, or can be pushed to FTP or SCP servers. In addition, alerts regarding the functioning of the appliance itself and it’s subsystems can be monitored by email to administrators who are responsible for monitoring and operating the device.

WSA devices offer a similar depth of functioning. WSA effectively acts as a web proxy, meaning that it logs all inbound and outbound transaction information for HTTP traffic. These logs can be quite detailed and are customizable. They can be configured in a W3C compatibility format. The WSA can be configured to submit the logs to a server in various ways, including syslog, FTP, and SCP.

Other logs that are available to the WSA include ACL decision logs, malware scan logs, and web reputation filtering logs.

The figure illustrates the “drill-down” dashboards available from Cisco content filtering devices. By clicking components of the Overview reports, more relevant details are displayed. Target searches provide the most focused information.

The figure on the left shows windows that have charts with vertical bars, charts with horizontal bars, and charts with icons and data. In the middle are the detailed reports with two charts up top with horizontal bars shown followed by a table at the bottom with rows and columns. On the right is the targeted search with blank textboxes available.

25.3.5

Logging from Cisco Devices

Cisco security devices can be configured to submit events and alerts to security management platforms using SNMP or syslog. The figure illustrates a syslog message generated by a Cisco ASA device and a syslog message generated by a Cisco IOS device.

The figure shows Cisco a s a device. The line starts with an asterisk and has the words n t p status pointing down to the letter m of Mar. The timestamp is Mar19 11:22:07.289 EDT: % than a s a which is the Cisco facility – 3 which is the severity – 201008 which is the message i d followed by the message text: disallowing new connections. The figure also shows Cisco I O S device and a line that starts with *Sep 16 08:50:47.359 EDT: % and S Y S for the cisco facility – 5 for the severity – CONFIG_I for the mnemonic followed by Configured from console by con0.

Cisco Syslog Message Formats

Note that there are two meanings used for the term facility in Cisco Syslog messages. The first is the standard set of Facility values that were established by the Syslog standards. These values are used in the PRI message part of the Syslog packet to calculate the message priority. Cisco uses some of the values between 15 and 23 to identify Cisco log Facilities, depending on the platform. For example, Cisco ASA devices use Syslog Facility 20 by default, which corresponds to local4. The other Facility value is assigned by Cisco and occurs in the MSG part of the Syslog message.

 

Cisco devices may use slightly different Syslog message formats and may use mnemonics instead of message IDs, as shown in the figure. A dictionary of Cisco ASA Syslog messages is available on the Cisco website.

Proxy Logs

Proxy servers, such as those used for web and DNS requests, contain valuable logs that are a primary source of data for network security monitoring.

Proxy servers are devices that act as intermediaries for network clients. For example, an enterprise may configure a web proxy to handle web requests on the behalf of clients. Instead of requests for web resources being sent directly to the server from the client, the request is sent to a proxy server first. The proxy server requests the resources and returns them to the client. The proxy server generates logs of all requests and responses. These logs can than be analyzed to determine which hosts are making the requests, whether the destinations are safe or potentially malicious, and to also gain insights into the kind of resources that have been downloaded.

Web proxies provide data that helps determine whether responses from the web were generated in response to legitimate requests or have been manipulated to appear to be responses but are in fact exploits. It is also possible to use web proxies to inspect outgoing traffic as means of data loss prevention (DLP). DLP involves scanning outgoing traffic to detect whether the data that is leaving the web contains sensitive, confidential, or secret information. Examples of popular web proxies are Squid, CCProxy, Apache Traffic Server, and WinGate.

An example of a Squid web proxy log in the Squid-native forma appears below. Explanations of the field values appear in the table below the log entry.

DNS Proxy Log Example

1265939281.764     19478 172.16.167.228 TCP_MISS/200 864 GEThttp://www.example.com//images/home.png - NONE/- image/png
Proxy Log Value Explanation
1265939281.764 Time  in Unix epoch timestamp format with milliseconds
19478 Duration – the elapsed time for the request and response from Squid
172.16.167.228 Client IP address
TCP_MISS/200 Result – Squid result codes and HTTP status code separated by a slash
864 Size – the bytes of data delivered
GET Request – HTTP request made by the client
http://www.example.com//images/home.png URI/URL – address of the resource that was requested
Client identity  RFC 1413 value for the client that made the request. Not used by default.
NONE/- Peering code/Peer host – neighbor cache server consulted
image/png Type – MIME content type from the Content-Type value in the HTTP response header

Note: Open web proxies, which are proxies that are available to any internet user, can be used to obfuscate threat actor IP addresses. Open proxy addresses may be used in blacklisting internet traffic.

Cisco Umbrella

Cisco Umbrella, formerly OpenDNS, offers a hosted DNS service that extends the capability of DNS to include security enhancements. Rather than organizations hosting and maintaining blacklisting, phishing protection, and other DNS-related security, Cisco Umbrella provides these protections in it’s own DNS service. Cisco Umbrella is able to apply many more resources to managing DNS than most organizations can afford. Cisco Umbrella functions in part as a DNS super proxy in this regard. The Cisco Umbrella suite of security products apply real-time threat intelligence to managing DNS access and the security of DNS records. DNS access logs are available from Cisco Umbrella for the subscribed enterprise. Instead of using local or ISP DNS servers, an organization can choose to subscribe to Cisco Umbrella for DNS and other security services. An example of a DNS proxy log appears below. The table explains the meaning of the fields in the log entry.

DNS Proxy Log Example

"2015-01-16 17:48:41","ActiveDirectoryUserName",
"ActiveDirectoryUserName,ADSite,Network",
"10.10.1.100","24.123.132.133","Allowed","1 (A)",
"NOERROR","domain-visited.com.",
"Chat,Photo Sharing,Social Networking,Allow List"
Field Example Explanation
Timestamp 2015-01-16 17:48:41 This is when this request was made in UTC. This is different than the Umbrella dashboard, which converts the time to your specified time zone.
Policy Identity ActiveDirectoryUserName The first identity that matched the request.
Identities ActiveDirectoryUserName,ADSite,Network All identities associated with this request.
Internal Ip 10.10.1.100 The internal IP address that made the request.
External Ip 24.123.132.133 The external IP address that made the request.
Action Allowed Whether the request was allowed or blocked.
QueryType 1 (A) The type of DNS request that was made.
ResponseCode NOERROR The DNS return code for this request.
Domain domain-visited.com. This is the domain that was requested.
Categories Chat, Photo Sharing, Social Networking The security or content categories that the destination matches.

Next-Generation Firewalls

Next-Generation or NextGen Firewall devices extend network security beyond IP addresses and Layer 4 port numbers to the application layer and beyond. NexGen Firewalls are advanced devices that provided much more functionality than previous generations of network security devices. One of those functionalities is reporting dashboards with interactive features that allow quick point-and-click reports on very specific information without the need for SIEM or other event correlators.

Cisco’s line of NextGen Firewall devices (NGFW) use Firepower Services to consolidate multiple security layers into a single platform. This helps to contain costs and simplify management. Firepower services include application visibility and control, Firepower Next-Generation IPS (NGIPS), reputation and category-based URL filtering, and Advanced Malware Protection (AMP). Firepower devices allow monitoring network security through a web-enabled GUI called Event Viewer.

Common NGFW events include:

  • Connection Event – Connection logs contain data about sessions that are detected directly by the NGIPS. Connection events include basic connection properties such as timestamps, source and destination IP addresses, and metadata about why the connection was logged, such as which access control rule logged the event.
  • Intrusion Event – The system examines the packets that traverse the network for malicious activity that could affect the availability, integrity, and confidentiality of a host and it’s data. When the system identifies a possible intrusion, it generates an intrusion event, which is a record of the date, time, type of exploit, and contextual information about the source of the attack and it’s target.
  • Host or Endpoint Event – When a host appears on the network it can be detected by the system and details of the device hardware, IP addressing, and the last known presence on the network can be logged.
  • Network Discovery Event – Network discovery events represent changes that have been detected in the monitored network. These changes are logged in response to network discovery policies that specify the kinds of data to be collected, the network segments to be monitored, and the hardware interfaces of the device that should be used for event collection.
  • Netflow Event -Network discovery can use a number of mechanisms, one of which is to use exported NetFlow flow records to generate new events for hosts and servers.

The figure shows 7 sections with icons within each section: intrusion prevention (subscription), application visibility and control, firepower analytics and automation, advanced malware protection and sandboxing (subscription), build in network profiling, u r l filtering (subscription) and identify policy control and v p n.

Action Point

I know you might agree with some of the points that I have raised in this article. You might not agree with some of the issues raised. Let me know your views about the topic discussed. We will appreciate it if you can drop your comment. Thanks in anticipation.

Download Our App.

                    

CEHNigeria On Google Playstore

Download Our Blog App On Google Playstore.

 

                   

GET SEOPOZ. OUTSMART YOUR BLOG COMPETITORS

Have a deeper understanding of Google Search Console. Joint SEOPOZ for free.

Joint Our Whatsapp Group Here

                    

Joint Our Whatsapp Group

Follow Us On Twitter and I will Follow Back

                    

Follow Us On Twitter

Kindly follow me on Twitter and I promise I will follow back. Aside you will get updated when we post new articles.

About Adeniyi Salau 791 Articles
I am an IT enthusiast and a man of many parts. I am a Certified Digital Marketer, Project Manager and a Real Estate Consultant. I love writing because that's what keeps me going. I am running this blog to share what I know with others. I am also a Superlife Stem Cell Distributor. Our Stem Cell Products can cure many ailments.

Be the first to comment

Leave a Reply

Your email address will not be published.


*


CommentLuv badge