thePacketGeek

Improved Retryable Logic with Macro Setup

2020-08-12T00:00:00+00:00

We've now built a simple retry!</code> macro</a> where the limited</em> logic is contained in the macro_rules. This article will focus on an improved Retryable</code> struct (with RetryStrategy</code>) that we'll then build a new macro to simplify instantation for.</p>

`Retryable & RetryStrategy</h2> Forgoing macros for a bit, let's setup some retry structs and implementations. First is a Retryable</code> struct to contain our function/closure to retry, and a RetryStrategy</code> with options for retrying (number of retries, delay, etc.):</p>`


retry! Macro with Repetitive Matching and Nesting
2020-07-19T00:00:00+00:00
We previously</a> covered a very basic declarative macro that wrapped some timing logic around a function to be run. This article expands on that with some additional matching techinques to build a retry!</code> macro to be used like:</p>
let res = retry!(|| { sometimes_fail(10) });
assert!(res.is_ok());

let res = retry!(sometimes_fail, 10; retries = 3);
assert!(res.is_ok());
</code></pre>


First Macro: timeit!
2020-07-03T00:00:00+00:00
This series serves as a practical (but not-exhaustive</em>) introduction to declarative macro_rules!</code>. I've put together some Rust macro examples</a> to show how macros can be helpful for improving ergonomics around repetitive or error-prone tasks. The examples cover some great scenarios for macros like:</p>

Print out the time a block of code takes to execute</li>
Adding retries around intermittently fallible code</li>
Making data structure initialization easier for common options</li>
</ul>


Hide Cargo.lock in git Diff
2020-06-28T00:00:00+00:00
Whenever you add or update dependencies in Cargo.toml</code>, the diff output in Cargo.lock</code> can be verbose and distracting when I'm looking at git diff</code> output:</p>
$ git diff --stat
 Cargo.lock | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Cargo.toml |  3 +++
 2 files changed, 57 insertions(+)
</code></pre>
$ git diff
diff --git a/Cargo.lock b/Cargo.lock
index c287579..a41401e 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -151,6 +151,7 @@ dependencies = [
  "jsonrpsee",
  "log",
  "net2",
+ "pcap-file",
  "prettytable-rs",
  "serde",
  "serde_json",
@@ -344,6 +345,16 @@ dependencies = [
  "memchr",
 ]

+[[package]]
+ ... pages of [[package]] updates
</code></pre>


Create a Custom Protocol
2020-06-27T00:00:00+00:00
In this series so far we've learned how to read & write bytes</a> with TcpStream</code> and then how to abstract over that with a LinesCodec</code></a> for sending and receiving String</code> messages. In this post we'll look into what it takes to build a custom protocol for message passing more than a single type of thing (like a String</code>).</p>
Defining our Message Structs</h2>
To give our client and server more options for communicating we'll create:</p>

A Request</code> message that allow the client to request either</em>:

Echo a string</li>
Jumble a string with a specified amount of jumbling entropy</li>
</ul>
</li>
A Response</code> message for the server to respond with the successfully echo/jumbled String</code></li>
</ul>


Building a LinesCodec
2020-06-27T00:00:00+00:00
In the previous post</a> we learned how to read & write bytes with TcpStream</code>, but the calling code had to be aware of that and do the serialization/deserialization.  In this demo, we'll continue using BufRead</code> and BufReader</code> to build:</p>

A LinesCodec</code> that abstracts away String</code> serialization/deserialization & TcpStream I/O</li>
A client that uses the LinesCodec</code> to send and print returned String</code>s</li>
A server that also uses the LinesCodec</code> and reverses String</code>s before echoing them back</li>
</ul>


Reading and Writing Data
2020-06-27T00:00:00+00:00
For the start of our journey with TcpStream</a> we'll take a look at what it takes to send and receive raw bytes.</p>
This demo is going to build up some concepts that result in a demo found here</a>. The working concept is that we want to have a client send some data (a String</code>) to a server that will echo the message back to the client. In further demos, we'll progress this concept to do slightly more interesting things with the data on the server before sending back.</p>


Scapy p.01
2019-05-11T00:00:00+00:00
What is Scapy?</h3>
No one can introduce Scapy better than the creator of the project himself:</p>

Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more. It can easily handle most classical tasks like scanning, tracerouting, probing, unit tests, attacks or network discovery</p>
It also performs very well at a lot of other specific tasks that most other tools can't handle, like sending invalid frames, injecting your own 802.11 frames, combining technics (VLAN hopping+ARP cache poisoning, VOIP decoding on WEP encrypted channel, ...), etc.</p>
</blockquote>

Phil @  secdev.org</a></li>
</ul>


Scapy p.02
2019-05-11T00:00:00+00:00
Installing Python</h4>
Scapy was originally written for Python 2, but since the 2.4 release (March 2018), you can now use Scapy with Python 3.4+! I will prefer Python 3 in examples but will also include notes about big differences between each python version and Scapy if they exist.</p>


Scapy p.03
2019-05-11T00:00:00+00:00
Running Scapy</h4>
Scapy can be run in two different modes, interactively from a terminal window and programmatically from a Python script. Let's start getting familiar with Scapy using the interactive mode.</p>


Scapy p.04
2019-05-11T00:00:00+00:00
Packets, Layers, and Fields. Oh My!</h4>
Scapy uses Python dictionaries as the data structure for packets. Each packet is a collection of nested dictionaries with each layer being a child dictionary of the previous layer, built from the lowest layer up. Visualizing the nested packet layers would look something like this:</p>
</p>


Scapy p.05
2019-05-11T00:00:00+00:00
With a good understanding of how to view our packets we can now move onto some packet generation. Let's talk a bit about sniffing first and how existing packets are our best tool for creating new ones.</p>


Scapy p.07
2019-05-11T00:00:00+00:00
Using Scapy in a Python Script</h4>
So far we've been working with Scapy in interactive mode. It's very powerful but there are times when it would be easier to work with a Python script instead. In order to use Scapy, we have to import the Scapy module like this:</p>
from scapy.all import *
</code></pre>
This will import all Scapy functions, but if you know that you will only need a few of the functions, you can import them individually as well like this:</p>
from scapy.all import sr1,IP,ICMP
</code></pre>


Scapy p.08
2019-05-11T00:00:00+00:00
We've doing a lot of packet sniffing, analysis, and even some basic packet crafting of our own. With the ICMP packets we created, we only set the destination we wanted to use and let Scapy take care of the rest.</p>


Scapy p.09
2019-05-11T00:00:00+00:00
We've been able to work with Ethernet, ARP, IP, ICMP, and TCP pretty easily so far thanks to Scapy's built in protocol support. Next on our list of protocols to work with are UDP and DNS.</p>
DNS Request and Response</h4>
Using the sr1()</code> function, we can craft a DNS request and capture the returned DNS response.


Scapy p.10
2019-05-11T00:00:00+00:00
We've seen a lot of cool applications for scapy in your network tools, but a good inspiration for new tools is to look at existing tools to figure out how they do their job. We will be emulating some nmap & Angry IP Scanner type features and creating the following tools:</p>

TCP Port Range Scanner</a></li>
ICMP Ping Sweep</a></li>
Combining the Two</a></li>
</ul>


Scapy p.11
2019-05-11T00:00:00+00:00
I hope you had as much fun as I did getting started with Scapy. These are all starter ideas, but we've barely uncovered the tip of the iceberg. I'll continue to write articles about cool Scapy tools I come up with but you should dig into the docs below and see what you find. If you have any questions or comments about this guide, feel free to contact me.</p>


Scapy Sniffing with Custom Actions
2019-05-11T00:00:00+00:00
Scapy has a sniff</code> function that is great for getting packets off the wire, but there's much more to show off how great this function really is! sniff</code> has  an argument prn</code> that allows you to pass a function that executes with each packet sniffed. The intended purpose of this function is to control how the packet prints out in the console allowing you to replace the default packet printing display with a format of your choice.</p>
The prn</code> argument is defined as:</p>

prn: function to apply to each packet. If something is returned, it is displayed. For instance you can use prn = lambda x: x.summary().</p>
</blockquote>


Scapy Sniffing with Custom Actions
2019-05-11T00:00:00+00:00
In the previous article I demonstrated how to add a custom function to change the formatting of the packet output in the console or do some sort of custom action with each packet. That example passed the function (a Callable) without any additional args. When the prn passed function is called (with each packet), it receives a single argument of the packet that was just sniffed.</p>
Using nested functions to harness the power of closure, you can bind any number of arguments to the function that is executed on each packet by Scapy. In order to bind additional arguments to the prn function, we have to use nested functions (similar to a decorator). Check out this example, created to upload the scapy packet info to an API via the Python Requests module:</p>


Processing and Storing BGP Messages in MongoDB
2015-06-27T00:00:00+00:00
Now that you know how to advertise prefixes to BGP peers with ExaBGP and are familiar with how to use this to influence traffic in your network, let's change gears and look at processing BGP messages between ExaBGP and its peers. Since ExaBGP uses JSON for message data, I figured it would be a good opportunity to use MongoDB</a> so the message information can easily be stored into a database for data collection and analysis.</p>
</p>


Control BGP Advertisements
2015-06-23T00:00:00+00:00
We know that ExaBGP can be used to inject BGP routes into an AS that you control. It's a very helpful feature that allows you to automate reachability and traffic flows within your network. But what if you want to you ExaBGP to influence what is advertised, or in this case, not advertised to external peers that are not willing to peer with your ExaBGP service?</p>
</p>


Advanced Route Announcement
2015-05-24T00:00:00+00:00
Now that you're familiar with the basics of setting up a peer with ExaBGP and advertising routes, let's look at expanding those concepts towards something that might actually be used in production.</p>
Peering with Multiple BGP Routers</h2>
</p>


Give ExaBGP an HTTP API
2015-05-22T00:00:00+00:00
We've covered how to setup ExaBGP and peer with a router, and then how to use python to add  and remove advertised routes in BGP either with static definitions or dynamically through health checking. There may be some of you out there with some sort of application that is already monitoring routes and you're trying to figure out how to connect it with ExaBGP for the actual interaction part? Well, what if we add an HTTP API to ExaBGP to give programmatic access to ExaBGP from some external utility? I'll go over two ways to do this using the Python built-in SimpleHTTPServer</code> or Flask</a>.</p>


Using service health checks to automate ExaBGP
2015-05-21T00:00:00+00:00
If you haven't read the intro post to this series, Getting Started with ExaBGP</a>, check that article out first as this post will be expanding on the previous example to show how to use python for more automated interaction with ExaBGP.</p>


ExaBGP and Python
2015-05-20T00:00:00+00:00
I'm really excited about these next few posts. I've been doing some research on BGP and automating routing decisions with python, which led to my discovery of ExaBGP</a>. ExaBGP is dubbed "The BGP swiss army knife", and I'm early in my experimentation with this tool, but it seems to be a very easy way to peer with your BGP routers and control the advertisement of networks.</p>


Writing Packets to Trace File with Scapy
2015-05-20T00:00:00+00:00
This is a follow-up post to accompany the previous importing packets from trace files with scapy</a> post. So you've sniffed or generated some packets with scapy and it's time to write them to file to analyze and double-check your work. Here's a simple example of how to save those packets.</p>
localhost:~ packetgeek$ scapy
>>> packets = sniff(count=10)
>>> packets
<Sniffed: TCP:0 UDP:3 ICMP:0 Other:7>
>>> wrpcap('sniffed.pcap', packets)
</code></pre>


Using the Packet Object
2014-11-12T00:00:00+00:00
So far in this series we've done a lot with capturing packets and working with the capture object, but finally we're going to get to the fun part and finally start playing with some PACKETS!!!!</p>
When we have captured packets in a capture object, they are stored as a list of packet objects.  These packet objects will have methods and attributes that give us access to the header and payload info of each packet.  As stated in a previous post we have control for how much info about the packets we store in each packet option through the only_summaries</code> argument in the LiveCapture and ReadCapture modules.</p>


Using the Capture Object
2014-11-11T00:00:00+00:00
Now that we know how to use the FileCapture and LiveCapture modules to capture some packets, let's see what options we have with the returned capture object (truncated list for brevity):</p>
>>> dir(cap)
Out[3]:
['apply_on_packets',
 'close',
 'current_packet',
 'display_filter',
 'encryption',
 'input_filename',
 'next',
 'next_packet']
</code></pre>


FileCapture and LiveCapture modules
2014-11-10T00:00:00+00:00
The two typical ways to start analyzing packets are via PyShark's FileCapture and LiveCapture modules. The first will import packets from a saved capture file, and the latter will sniff from a network interface on the local machine. Running these modules will return a capture object which I will cover in depth in the next post. For now, let's see what we can do with these two modules.</p>


Intro to PyShark
2014-11-08T00:00:00+00:00
I can hardly believe it took me this long to find PyShark, but I am very glad I did! PyShark is a wrapper for the Wireshark CLI interface, tshark, so all of the Wireshark decoders are available to PyShark! It is so amazing that I started a new project just so I could use this amazing new tool: Cloud-Pcap</a>.</p>
You can use PyShark to sniff from a interface or open a saved capture file, as the docs show on the overview page here</a>:</p>
import pyshark

# Open saved trace file 
cap = pyshark.FileCapture('/tmp/mycapture.cap')

# Sniff from interface
capture = pyshark.LiveCapture(interface='eth0')
capture.sniff(timeout=10)
<LiveCapture (5 packets)>
</code></pre>


Importing packets from trace files with Scapy
2014-09-25T00:00:00+00:00
Scapy is amazingly flexible when it comes to creating packets, but in some cases you may want to mangle or change packets that you've sniffed and saved in a trace file. Scapy currently supports .cap, .pcap, and .pcapng files.  Reading these files are possible through the rdpcap()</code> function:</p>
localhost:~ packetgeek$ scapy
>>> packets = rdpcap('IBGP_adjacency.cap')
>>> packets
<IBGP_adjacency.cap: TCP:17 UDP:0 ICMP:0 Other:0>
</code></pre>


Scapy p.06
2013-10-29T00:00:00+00:00
We've sniffed some packets, dig down into packet layers and fields, and even sent some packets. Great job! It's time to step up our game with Scapy and start really using some of the power Scapy contains. Please Note: this next example is for education and example only. Please be responsible on your network, especially at work!</p>

thePacketGeek

Improved Retryable Logic with Macro Setup

Retryable & RetryStrategy</h2> Forgoing macros for a bit, let's setup some retry structs and implementations. First is a Retryable</code> struct to contain our function/closure to retry, and a RetryStrategy</code> with options for retrying (number of retries, delay, etc.):</p>

retry! Macro with Repetitive Matching and Nesting

First Macro: timeit!

Hide Cargo.lock in git Diff

Create a Custom Protocol

Building a LinesCodec

Reading and Writing Data

Scapy p.01

What is Scapy?</h3> No one can introduce Scapy better than the creator of the project himself:</p>

Scapy p.02

Installing Python</h4> Scapy was originally written for Python 2, but since the 2.4 release (March 2018), you can now use Scapy with Python 3.4+! I will prefer Python 3 in examples but will also include notes about big differences between each python version and Scapy if they exist.</p>

Scapy p.03

Running Scapy</h4> Scapy can be run in two different modes, interactively from a terminal window and programmatically from a Python script. Let's start getting familiar with Scapy using the interactive mode.</p>

Scapy p.04

Scapy p.05

Scapy p.07

Scapy p.08

Scapy p.09

DNS Request and Response</h4> Using the sr1()</code> function, we can craft a DNS request and capture the returned DNS response.

Scapy p.10

Scapy p.11

Scapy Sniffing with Custom Actions

Scapy Sniffing with Custom Actions

Processing and Storing BGP Messages in MongoDB

Control BGP Advertisements

Advanced Route Announcement

Peering with Multiple BGP Routers</h2> </p>

Give ExaBGP an HTTP API

Using service health checks to automate ExaBGP

ExaBGP and Python

Writing Packets to Trace File with Scapy

Using the Packet Object

Using the Capture Object

FileCapture and LiveCapture modules

Intro to PyShark

Importing packets from trace files with Scapy

Scapy p.06

`Retryable & RetryStrategy</h2> Forgoing macros for a bit, let's setup some retry structs and implementations. First is a Retryable</code> struct to contain our function/closure to retry, and a RetryStrategy</code> with options for retrying (number of retries, delay, etc.):</p>`

Installing Python</h4>
Scapy was originally written for Python 2, but since the 2.4 release (March 2018), you can now use Scapy with Python 3.4+! I will prefer Python 3 in examples but will also include notes about big differences between each python version and Scapy if they exist.</p>

Running Scapy</h4>
Scapy can be run in two different modes, interactively from a terminal window and programmatically from a Python script. Let's start getting familiar with Scapy using the interactive mode.</p>

DNS Request and Response</h4>
Using the `sr1()</code> function, we can craft a DNS request and capture the returned DNS response.`

Peering with Multiple BGP Routers</h2>
</p>