قالب وردپرس درنا توس
Home / Tips and Tricks / How to capture Wi-Fi data with Jupyter Notebook «Null Byte :: WonderHowTo

How to capture Wi-Fi data with Jupyter Notebook «Null Byte :: WonderHowTo



Wireshark is platform independent and can capture huge amounts of data. Understanding that data is a completely different task. That is where Jupyter Notebook comes in. It can help analyze Wi-Fi packages and determine which networks a particular phone is previously connected to, so that we gain insight into the identity of the owner.

In general, data can be confusing, especially when there is much of it, which is both a blessing and a curse. It makes it more likely to include important patterns, but also to obscure them earlier. Searching for meaningful patterns in raw data can be like finding a needle in a haystack, but free tools for big data analysis, such as Jupyter Notebook, make it easier.

Wireshark for Wi-Fi data

Wireshark is an incredible tool for collecting Wi-Fi data, and it can quickly fill a screen with information. That data can tell you a lot, depending on what you are looking for, but it often says too much to be able to easily recognize patterns. Wireshark comes with built-in ways to analyze data, but sharing the results can be difficult and the tools built into Wireshark may not be able to tell you what you need.

The type of data you can get from a Wireshark capture includes the MAC addresses of each device that transmits or receives in the neighborhood, information about which clients are currently connected to which networks, and even information about nearby networks with which clients are connected in the past.

All this can be very valuable for a hacker interested in following the location or learning information about a topic. By finding Wi-Fi networks to which a device has previously been connected, an attacker can also create fake versions of trusted networks with which the victim's device connects automatically.

Jupyter Notebook for Data Analysis

Once we have collected information in Wireshark, we can export it as a CSV file and import it into Jupyter Notebook. The advantage of this is that we can quickly browse the data generated by Wireshark using the Python Panda & # 39; s library. That library can work with CSV files as data frames, which can easily plot graphs and data graphs to display relationships.

One of the great things about working with Jupyter Notebook is how easy it is to search through information for insights. A significant difference between data and intelligence is the processing of the data into insights, which we can do with data in Jupyter.

In our example, we use Jupyter Notebook to investigate a Wireshark recording on a public train. In the recording we include the devices of passengers responding to the Arduino " Beacon Spammer" sketch, which makes 40 fake Wi-Fi APs with the names of popular open networks. Any device that has joined a hotspot with the same name as one of the fake devices tries to connect. We can record these responses in Jupyter Notebook and display them in a graph to see which fake networks most phones respond to and to which fake network names each phone has been connected in the past.

What do you need

together you need a computer with Wireshark. It helps to have a card that can be put into monitor mode but you can also use data from the recording on GitHub to follow if you cannot generate it yourself. You can download the sample CSV dataset by cloning the repo, going to the directory and mentioning its contents.

  ~ # git clone https://github.com/skickar/Research.git

Cloning in & # 39; research & # 39; ...
remote: list objects: 21, done.
remote: objects count: 100% (21/21), ready.
external: compress objects: 100% (21/21), ready.
external: total 107 (delta 9), reused 0 (delta 0), reused pack 86
Receiving objects: 100% (107/107), 1.18 MiB | 4.91 MiB / s, ready.
Delta & # 39; s solution: 100% (47/47), ready.

~ # cd Research
~ / Research # ls

command.json jsonpayload.json sash.json
dataCleaner.py MetroWilshireVermonttoHollywoodHighland.csv SocalDecloaker.ino
fades.py NullByteSpecial.ipynb twitter0.json
FindWiFiDistance README.md twitter1.json
json.json RedLineResearch.ipynb 

You also need Python3 to run Jupyter, so make sure your system has it before you continue.

To retry the experiment and capture your own data, you must use the Beacon Spammer sketch on an ESP8266-based device set to stay on the channel of your choice. You must add common network names that do not require a password in your region, up to 1,000. When you flash the sketch and connect the ESP8266, you should see the fake networks appear.

Then you only have to record in Wireshark on the channel you specified. Record all traffic while the Beacon Spammer is running and look for devices that are trying to connect. The resulting data should work well for us to analyze.

Step 1: Export a Wireshark Capture as a CSV file

First we take a PCAP file in Wireshark and change it to a CSV file for use in Jupyter notebook. To do this, open your recording in Wireshark and add all required data from packages to the columns by right-clicking on the field you want to add and select "Apply as column"

Then add display filters that you want to remove to remove unwanted packages. Here I say that I only want packages with a destination address that matches the partial MAC address "6c: 3f: 23". This recording filter also uses the [0:3] to indicate that I want to look from the beginning to the third octet of the MAC address.

Once the recording filter is activated, only packages sent to the Beacon Spammer are seen. Every time we start the Beacon Spammer, it chooses a MAC address and then changes the last half of the MAC for each fake network that is created. That makes it easy to select all packages sent to our fake networks by only telling Wireshark that packages are sent to addresses with the same partial MAC address.

It is finally time to export our selected packages. Under "File", click "Export package distributions" and select "As CSV" to export in the correct format.

Here you can name the file and specify whether you want the displayed packages to be exported or all packages in the recording.

Once that is done, a CSV file with the data must be stored on your computer, ready for import into Jupyter Notebook.

Step 2: Install Jupyter & Open a New Notebook

Jupyter is a data analysis program written in Python, making it easy to install on any computer on which Python3 is installed. You can do this by running the following command in a new terminal window:

  ~ # pip3 install jupyter

Required already: jupyter in /usr/local/lib/python3.7/dist-packages (1.0.0)
Required though: notebook in /usr/local/lib/python3.7/dist-packages (from jupyter) (6.0.2)
Required though: jupyter console in /usr/local/lib/python3.7/dist-packages (from jupyter) (6.0.0)
Required though: nbconvert in /usr/local/lib/python3.7/dist-packages (from jupyter) (5.6.1)
Required though: ipywidgets in /usr/local/lib/python3.7/dist-packages (from jupyter) (7.5.1)
Required though: ipykernel in /usr/local/lib/python3.7/dist-packages (from jupyter) (5.1.3)
Required though: qtconsole in /usr/local/lib/python3.7/dist-packages (from jupyter) (4.6.0)
Required though: jupyter client> = 5.3.4 in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (5.3.4)
Required though: Send2Trash in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (1.5.0)
Required though: prometheus client in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (0.7.1)
Required though: pyzmq> = 17 in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (18.1.1)
Required though: terminado> = 0.8.1 in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (0.8.3)
Required though: jupyter-core> = 4.6.0 in /usr/local/lib/python3.7/dist-packages (from notebook-> jupyter) (4.6.1)
Required al: nbformat in / usr / lib / python3 / dist-packages (from notebook-> jupyter) (4.4.0)
Required though: traitlets> = 4.2.1 in / usr / lib / python3 / dist-packages (from notebook-> jupyter) (4.3.2)
Required though: jinja2 in / usr / lib / python3 / dist-packages (from notebook-> jupyter) (2.10)
Required though: tornado> = 5.0 in / usr / lib / python3 / dist-packages (from notebook-> jupyter) (5.1.1)
Required though: ipython genutils in / usr / lib / python3 / dist-packages (from notebook-> jupyter) (0.2.0)
Required though: prompt-toolkit <2.1.0,> = 2.0.0 in /usr/local/lib/python3.7/dist-packages (from jupyter-console-> jupyter) (2.0.10)
Required though: pygments in / usr / lib / python3 / dist-packages (from jupyter-console-> jupyter) (2.3.1)
Required though: ipython in /usr/local/lib/python3.7/dist-packages (from jupyter-console-> jupyter) (7.10.2)
Required though: test path in /usr/local/lib/python3.7/dist-packages (from nbconvert-> jupyter) (0.4.4)
Required though: mistune <2,> = 0.8.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert-> jupyter) (0.8.4)
Required though: pandoc filters> = 1.4.1 in /usr/local/lib/python3.7/dist-packages (from nbconvert-> jupyter) (1.4.2)
Requirement already met: entry points> = 0.2.2 in / usr / lib / python3 / dist-packages (from nbconvert-> jupyter) (0.3)
Required though: defusedxml in /usr/local/lib/python3.7/dist-packages (from nbconvert-> jupyter) (0.6.0)
Required though: bleach in /usr/local/lib/python3.7/dist-packages (from nbconvert-> jupyter) (3.1.0)
Required though: widgetsnbextension ~ = 3.5.0 in /usr/local/lib/python3.7/dist-packages (from ipywidgets-> jupyter) (3.5.1)
Required though: python-dateutil> = 2.1 in / usr / lib / python3 / dist-packages (from jupyter client> = 5.3.4-> notebook-> jupyter) (2.7.3)
Requirement already met: ptyprocess; os_name! = "nt" in /usr/local/lib/python3.7/dist-packages (from terminado> = 0.8.1-> notebook-> jupyter) (0.6.0)
Required already: six> = 1.9.0 in / usr / lib / python3 / dist-packages (from prompt-toolkit <2.1.0,> = 2.0.0-> jupyter-console-> jupyter) (1.12.0)
Required though: toilet width in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit <2.1.0,> = 2.0.0-> jupyter-console-> jupyter) (0.1.7)
Required though: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython-> jupyter-console-> jupyter) (0.7.5)
Required though: backcall in /usr/local/lib/python3.7/dist-packages (from ipython-> jupyter-console-> jupyter) (0.1.0)
Required already: decorator in / usr / lib / python3 / dist-packages (from ipython-> jupyter-console-> jupyter) (4.3.0)
Requirement already met: pexpect; sys_platform! = "win32" in /usr/local/lib/python3.7/dist-packages (from ipython-> jupyter-console-> jupyter) (4.7.0)
Required though: set-up tools> = 18.5 in / usr / lib / python3 / dist-packages (from ipython-> jupyter-console-> jupyter) (40.8.0)
Required though: jedi> = 0.10 in /usr/local/lib/python3.7/dist-packages (from ipython-> jupyter-console-> jupyter) (0.15.2)
Requirement already met: web coding in / usr / lib / python3 / dist-packages (from bleach-> nbconvert-> jupyter) (0.5.1)
Required though: parso> = 0.5.2 in /usr/local/lib/python3.7/dist-packages (from jedi> = 0.10-> ipython-> jupyter-console-> jupyter) (0.5.2) [19659014] After Jupyter is installed, we can run it to create a web interface and open it automatically. To do this, type the following. If you are running as root, you will be notified that this is not recommended and you can choose to continue or switch accounts. 

  ~ # jupyter notebook

[I 00:26:51.488 NotebookApp] JupyterLab extension loaded from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/jupyterlab
[I 00:26:51.488 NotebookApp] JupyterLab application directory is /Library/Frameworks/Python.framework/Versions/3.6/share/jupyter/lab
[I 00:26:51.490 NotebookApp] Presenting laptops from local folder: / Users / skickar
[I 00:26:51.491 NotebookApp] The Jupyter Notebook runs on:
[I 00:26:51.491 NotebookApp] http: // localhost: 8888 /? Token = 4th1f5eba5b656d903d08298a831a11ba97a581e3e575cda
[I 00:26:51.491 NotebookApp] Use Control-C to stop this server and close all kernels (twice to skip the confirmation).
[C 00:26:51.496 NotebookApp]

To access the notebook, open this file in a browser:
file: ///Users/skickar/Library/Jupyter/runtime/nbserver-54785-open.html
Or copy and paste one of these URL & # 39; s:
http: // localhost: 8888 /? token = 4de1f5eba5b656d903d08298a831a11ba97a581e3e575cda 

A web browser must be opened with which you can select previous projects or open a new one.

Step 3: Importing data into Jupyter

Let's look at the layout: At the top we see options to save, insert an entry and execute commands, and we also see the "In" prompt, waiting on us to add some Python 3.

Now we have to import our Wireshark CSV file into a Pandas data frame. The Panda & # 39; s and Matplotlib libraries help us easily work with captured data and manipulate it.

First we will import Panda & # 39; s and give it the nickname pd t while we do the same with the Pyplot library of Matplotlib as plt . Now, when we refer to pd, Python 3 knows that we are talking about Panda & # 39; s and the same with plt and Pyplot.

  In [ ]: import panda & # 39; s as pd
from matplotlib import pyplot as plt 

Next we use the Panda & # 39; s read_csv to specify the location of the CSV file that we work with, the type of character that separates the data (a comma in CSV files or a tab in TSV files), the encoding type and the row in which the header with the names of the cells is located. All this will be put in a variable that I call wd or wireless data but you can call yours whatever you want.

If you use data such as a Wigle Wifi CSV on an Android phone, it does not use the standard row 0 for headers. You can change the value in this last argument to the row that contains the column labels to correctly label the data.

Finally, we will sample 10 random data points using the .sample (10) method to get the results.

  In [ ]: wd = pd.read_csv (& # 39; PATH / TO / FILE.csv & # 39 ;, delimiter = & # 39;, & # 39 ;, encoding = & # 39; latin-1 & # 39 ;, header = 0)
wd.sample (10) 

Now that we have imported our data, we can start working with it to see what we can learn from the patterns in it.

Step 4: Plotting basic data in a graph

Visualizing data is one of the biggest points to put it in Jupyter, so let's dive into it exploring relationships with graphs. Go down under the output to a new entry, and we start by cutting the data.

To do this, we must learn how to access the data in a data frame. Our complete data frame is our variable, wd which contains all the information that we have imported from our CSV file. To access part of the data, we use the column names we learned from sampling the data in the last command to access the data in those columns.

If we want to plot the values ​​in the Time column, I can access them using wd [Time].

Knowing that I can make a graph by first defining the size with the command rcParams and then plotting to engrave the elements I with the plt.plot (wd [A] wd [B]) command, where & # 39; o & # 39; is added to indicate that we want to use points instead of lines around our data. Finally, we can set the color with the last color variable.

  In [ ]: plt.rcParams ["figure.figsize"] = (20,10)
plt.plot (wd ['Time'] wd ['Transmitter address'] & # 39; o & # 39 ;, color = & # 39; DarkGreen & # 39;) 

The final step will be to put our X and Y axis in the label chart.

  In [ ]: plt.xlabel (& # 39; Time & # 39;)
plt.ylabel (& # 39; Real Clients Directed Packets Fake Networks & # 39;) 

If this is done, we can add plt.show () to plot the number.

  In [ ]: plt.show () 

Now do the same to plot the "Destination Address" column against time. The end result should look something like this.

  In [ ]: ## Plotting targeted packages (unmasked clients) that connect to fake networks over time
# Here we analyze in the first figure when a client connects to a fake network over time.
# In the second, we analyze which fake networks receive targeted packets from unmasked clients over time.
plt.rcParams ["figure.figsize"] = (20.10)
plt.plot (wd ['Time'] wd ['Transmitter address'] & # 39; o & # 39 ;, color = & # 39; DarkGreen & # 39;)
plt.title (& # 39; Customers who connect to fake networks over time & # 39;)
plt.xlabel (& # 39; Time & # 39;)
plt.ylabel (& # 39; Real customers send targeted packages to fake networks & # 39;)
plt.show ()
plt.plot (wd ['Time'] wd ['Destination address'] & # 39; o & # 39 ;, color = & # 39; DarkBlue & # 39;)
plt.title (& # 39; Fake Networks Broadcasted by the Beacon Spammer Received Directed Packets & # 39;)
plt.xlabel (& # 39; Time & # 39;)
plt.ylabel (& # 39; Beacon Spammer MAC from Fake Networks Receiving Hit & # 39;)
plt.show () 

Click "Execute" while you have selected this entry to view the result.

Now we can see when every real device was sending a packet and when every fake network received a packet. Quite cool, but let's look at fingerprint-engaging networks and clients.

Step 5: Plotting destination and sender against each other

Now we are going to plot the columns "Destination address" and "Sender address" against each graph. If you do this, you will see on which networks each device has responded and on which network most devices have responded.

To prevent things from getting too long in the graph's labels, we use the .str method to retrieve only the last five characters of the MAC address in the sender and destination fields . It must be unique enough for our data set. To access the last five, we can use an index of -5 and then an : to indicate that we want to go to the end of the string. If we wanted the opposite, we could specify : 5 to grab the first five. When we call wd [Destination address] .str-5: we enter the last five characters in the "Destination Address" columns.

After labeling the X and Y axes, the code should look like this:

  In [ ]: ## Disable which MAC addresses of clients respond to which fake MAC addresses of networks
Here we see a fingerprint for each client device on the left.
We can scan the row that a device is in to determine which unique fake networks it will respond to.
We can scan a column to see which fake networks cause most client devices to respond.
plt.rcParams ["figure.figsize"] = (25.10)
plt.plot (wd ['Destination address'] .str [-5:] wd ['Transmitter address'] .str [-5:] & # 39; o & # 39 ;,)
plt.title (& # 39; Clients Connecting to Fake Networks & # 39;)
plt.xlabel (& # 39; Beacon Spammer Fake Networks & # 39;)
plt.ylabel (& # 39; Real Clients Connecting & # 39;)
plt.show () 

Click on "Run" to view the results.

Wow! We can see that every phone seems to have a unique fingerprint. We can also see that one network in particular was much more popular than the other. That means that a hacker can use the network to connect many phones.

Step 6: Find the most popular fake network

To find the most popular fake network name in the neighborhood to which devices have responded, we use the same approach to matching the last five characters of the MAC address. In our dataset there are two types of packages, probe requests and authentication . Probe requests contain the SSID of the network, while authentication does not.

We have recorded many more authentications than probe requests, so to find the name of the network to which all these devices responded, we can search for probe requests with a destination MAC address that matches the last five characters of our popular network "70:11".

We do this with the method .str.contains () that we can use both for matching the string 70:11 and the string Probe frame . For example, the command wd [wd['Destination address'] .str.contains (& # 39; 70: 11 & # 39;) will check whether the Destination Address field contains the string that we are looking for in the MAC address. [19659002] Together we will use a variable called popular in which we add rows that both have a destination address that matches that of our mystery network and the word Probe indicating that it is a probe is request. That should make a list of only probe requests targeted at the relevant fake network in the popular variable.

  In [ ]: ## Find the most popular network:
popular = wd [wd['Destination address'] .str.contains (& # 39; 70: 11 & # 39;) & wd ['Type/Subtype'] .str.contains (& # 39; Probe & # 39;)] 

Once we have this list of probe requests, we can make a graph with the data in the "SSID" cell with the following command:

  In [ ]: popular ['SSID'] .value_counts (). plot (& # 39; barh & # 39;) 

There we go! Through our analysis we have established that the SSID "attwifi" ensures that the majority of devices in the neighborhood connects automatically to our fake network.

If we want to share these results with others, we can click on the export option to download our notebook in various formats, including PDF, markdown and Jupyter & # 39 ; s native .ipynb format, perfectly displayed on GitHub.

Jupyter Notebook makes it easy to analyze Wi-Fi

Jupyter Notebook hits the right place for analyzing Wi-Fi information, making simple datasets easy to manipulate with simple Python commands. With a little more effort it is possible to import entire PCAP files raw, but the use of capture filters and columns in Wireshark to export data in CSV format is much more beginner-friendly. I learned Jupyter Notebook one evening with just a little bit of Python experience, so I encourage anyone looking for patterns in CSV files to view this free and easy-to-use resource.

I hope you enjoyed this tutorial for Jupyter Notebook and Wireshark! If you have any questions about this tutorial, please leave a comment and feel free to reach me on Twitter @KodyKinzie .

Don't miss it: Program MicroPython NeoPixel Holiday Lights to Animate You Want

Begin your journey with White-Hat Hacker with Null Byte & # 39; s Beginner's Guide to Managing Linux eBook.

Buy now for $ 29.99>

Cover image and screenshots of Kody / Null Byte




Source link