over 7 years ago

Men & Mice - Challenge

Project Proposal: Creating value from DNSTAP data

What network blindspots can be uncovered by analyzing events and data trends buried in DNSTAP in a data lake.

Introduction

The Men & Mice Suite is an overlay solution that is used to help people manage their DNS, DHCP and IPAM infrastructure. In a large environment, the suite can create large amounts of valuable data. This data can be for example:

Logs from Central or DNS / DHCP servers
Lease history from DHCP servers
DNSTAP feed from DNS servers or Men & Mice DDI Appliances

What is of special interest is the DNSTAP feed. DNSTAP is a method to capture and log DNS traffic and DNS related information from a DNS server in an efficient way. It is most interesting for those that need to know what is happening on the network. DNS is a very important protocol in networking and DNSTAP offers a way to extract DNS information without a performance impact on the DNS server. The amount of data gathered with DNSTAP can be huge in an environment where the DNS server is under a heavy load. The image below portrays a scenario where clients send requests to DNS servers, which in turn reply to those queries accordingly.

Project Description

Participants will be provided with access to an Azure Data lake containing DNSTAP data collected from Men & Mice Applicances. The participants can get free Azure student accounts with credit to use in the Azure Machine Learning studio. The goal of the project is to process and analyze the DNSTAP data provided and identify interesting events and data trends. A few topics/ideas of trends to try to identify will be listed below but participants are also encouraged to try to come up with their own ideas and to explore the data as much as possible.

Interesting events/trends to identify

Identifying security risks
Periods where traffic is high where additional performance might be needed
Periods where traffic is consistently low that could be used for maintainance

Additional information

Example DNSTAP messages

Client Query:

type: MESSAGE identity: "chase" version: "unbound 1.4.21+dnstap1" message: type: CLIENT_QUERY query_time: !!timestamp 2014-02-12 02:08:07.858612 socket_family: INET socket_protocol: UDP query_address: 127.0.0.1 query_port: 34591 query_message: | ;; opcode: QUERY, status: NOERROR, id: 472 ;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION: ;dnstap.info. IN A

;; ADDITIONAL SECTION:

;; OPT PSEUDOSECTION: ; EDNS: version 0; flags: ; udp: 4096

Resolver query and response to client

type: MESSAGE identity: "chase" version: "unbound 1.4.21+dnstap1" message:

type: RESOLVER_QUERY query_time: !!timestamp 2014-02-12 02:08:07.858604 socket_family: INET socket_protocol: UDP response_address: 192.5.5.241 response_port: 53 query_zone: "." query_message: |

;; opcode: QUERY, status: NOERROR, id: 35998 ;; flags: cd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION: ;. IN NS

;; ADDITIONAL SECTION:

;; OPT PSEUDOSECTION: ; EDNS: version 0; flags: do; udp: 4096 --- type: MESSAGE identity: "chase" version: "unbound 1.4.21+dnstap1" message:

type: RESOLVER_RESPONSE query_time: !!timestamp 2014-02-12 02:08:07.858604 response_time: !!timestamp 2014-02-12 02:08:07.872825 socket_family: INET socket_protocol: UDP response_address: 192.5.5.241 response_port: 53 query_zone: "." response_message: |

;; opcode: QUERY, status: NOERROR, id: 35998 ;; flags: qr aa cd; QUERY: 1, ANSWER: 14, AUTHORITY: 0, ADDITIONAL: 23

;; QUESTION SECTION: ;. IN NS

;; ANSWER SECTION: . 518400 IN NS j.root-servers.net.

. 518400 IN NS h.root-servers.net. . 518400 IN NS l.root-servers.net. . 518400 IN NS b.root-servers.net. . 518400 IN NS g.root-servers.net. . 518400 IN NS c.root-servers.net. . 518400 IN NS f.root-servers.net. . 518400 IN NS i.root-servers.net. . 518400 IN NS e.root-servers.net. . 518400 IN NS k.root-servers.net. . 518400 IN NS d.root-servers.net. . 518400 IN NS m.root-servers.net. . 518400 IN NS a.root-servers.net. . 518400 IN RRSIG NS 8 0 518400 20140218000000 20140210230000 33655 . SBq90M9BywFdFjI6ymZ4PTqB3Bbp6xVFhSapQgwTe1QQ0fg+tP93/eXhzaQxSv/nCBRaLLN7NsPThb sgBAykVs7FWYB7DwCWo9qdbIFPzcQsyRVhkns+Qr17QdfinUERLXhxmFhbj58jiVOUjsXsTVL/WxoV 5rDXdlNN7XtSda8=

;; ADDITIONAL SECTION: a.root-servers.net. 3600000 IN A 198.41.0.4 b.root-servers.net. 3600000 IN A 192.228.79.201 c.root-servers.net. 3600000 IN A 192.33.4.12 d.root-servers.net. 3600000 IN A 199.7.91.13 e.root-servers.net. 3600000 IN A 192.203.230.10 f.root-servers.net. 3600000 IN A 192.5.5.241 g.root-servers.net. 3600000 IN A 192.112.36.4 h.root-servers.net. 3600000 IN A 128.63.2.53 i.root-servers.net. 3600000 IN A 192.36.148.17 j.root-servers.net. 3600000 IN A 192.58.128.30 k.root-servers.net. 3600000 IN A 193.0.14.129 l.root-servers.net. 3600000 IN A 199.7.83.42 m.root-servers.net. 3600000 IN A 202.12.27.33 a.root-servers.net. 3600000 IN AAAA 2001:503:ba3e::2:30 d.root-servers.net. 3600000 IN AAAA 2001:500:2d::d f.root-servers.net. 3600000 IN AAAA 2001:500:2f::f h.root-servers.net. 3600000 IN AAAA 2001:500:1::803f:235 i.root-servers.net. 3600000 IN AAAA 2001:7fe::53 j.root-servers.net. 3600000 IN AAAA 2001:503:c27::2:30 k.root-servers.net. 3600000 IN AAAA 2001:7fd::1 l.root-servers.net. 3600000 IN AAAA 2001:500:3::42 m.root-servers.net. 3600000 IN AAAA 2001:dc3::35

;; OPT PSEUDOSECTION: ; EDNS: version 0; flags: do; udp: 4096
-- Paula Gould, Head of Brand & Comms, Men & Mice paula@menandmice.com +354 8950058

Questions?

If you have any questions about the hackathon, please post on the discussion forum.