Exporting raw packet data?

[Pete Fraser]

I'm new to Wireshark, so sorry if this is a dumb question.

I want to export packet data in raw format, so that I end up with a
binary file.

If the packets are TCP I can use Analyze->Follow TCP Stream then Save As Raw.
For any type of packet, I can select packet data in the bottom pane
and do File->Export->Selected Packet Bytes.

What I want to do, but can't work out how, is to export a lot of
packet data as a raw binary file.
I develop the appropriate filter so that only the packets of interest
are visible, then do File->Export->File..., select "All packets",
"Displayed", and "Packet Bytes" for the only Packet Format. I would
hope that I can then save as raw, but I only find ASCII, PS, XML,
etc. What am I doing wrong?

Thanks in advance.

[Guy Harris]

On Nov 13, 2006, at 5:52 PM, Pete Fraser wrote:

> I want to export packet data in raw format, so that I end up with a
> binary file.

"Raw" in what sense?

And what part of the packet data do you want to export?

And do you want to export from one packet, or multiple packets?

And, if it's multiple packets, to you just want to concatenate the
data, or do you want some sort of record format to keep the data from
different packets separated?

> What I want to do, but can't work out how, is to export a lot of
> packet data as a raw binary file.
> I develop the appropriate filter so that only the packets of interest
> are visible, then do File->Export->File..., select "All packets",
> "Displayed", and "Packet Bytes" for the only Packet Format. I would
> hope that I can then save as raw, but I only find ASCII, PS, XML,
> etc. What am I doing wrong?

What you're doing wrong is assuming that Wireshark has such a
capability.

In order to add such a capability, we first need to know what it would
do, hence the questions.

[Pete Fraser]

At 06:24 PM 11/13/2006, Guy Harris wrote:

>On Nov 13, 2006, at 5:52 PM, Pete Fraser wrote:
>
> > I want to export packet data in raw format, so that I end up with a
> > binary file.
>
>"Raw" in what sense?

In the sense that it's used in the Analyze->Follow TCP Stream dialogue.
That is, binary data; not an ASCII representation of HEX data.


>And what part of the packet data do you want to export?

The payload.
Again, the Analyze->Follow TCP Stream capability seems to do exactly
what I want (for TCP packets, but not UDP).
I can select a TCP packet from a webcam, do a raw save with
Analyze->Follow TCP Stream, and end up with a binary motion JPEG file
that many viewers will play (after I remove some ASCII header material).


>And do you want to export from one packet, or multiple packets?

Multiple packets. I think I can do it from one with
File->Export->Selected Packet Bytes....


>And, if it's multiple packets, to you just want to concatenate the
>data, or do you want some sort of record format to keep the data from
>different packets separated?

Concatenate.


> > What I want to do, but can't work out how, is to export a lot of
> > packet data as a raw binary file.
> > I develop the appropriate filter so that only the packets of interest
> > are visible, then do File->Export->File..., select "All packets",
> > "Displayed", and "Packet Bytes" for the only Packet Format. I would
> > hope that I can then save as raw, but I only find ASCII, PS, XML,
> > etc. What am I doing wrong?
>
>What you're doing wrong is assuming that Wireshark has such a
>capability.

Sorry. It had the capability for TCP packets, so I assumed the same for UDP.
I can write some code to take the text output from the
File->Export->File.. process, and convert it to binary, but I thought
that capability was probably in there alreadfy (it's such a great program).


>In order to add such a capability, we first need to know what it would
>do, hence the questions.


Thanks for considering it.

Pete

[Small, James]

Pete,

I didn't even realize you could do this until I read your question, but
here is one way (not sure if this is exactly what you want):
Open a capture
Narrow down the interesting packets
(For example, I do a lot of web traffic analysis so I might use a filter
such as http.content_length > 20000)
Now, let's say I see a Flash file, a GIF, or a JPEG that I want to save
- just the actual binary data, not the packet headers.
I would click on the interesting packet (assuming I have TCP and HTTP
reassembly enabled)
Next, in the packet details window (middle pane) I would click on the
relevant data portion. So for a JPEG image this would be the part that
reads JPEG File Interchange Format.
Finally, I would use the File->Export->Selected Packet Bytes menu item.
Then I would name the file and I personally change the save as type to
*.* so I can set the file extension (not completely sure this is
necessary but I do it out of habit).
Now, if I open up this file with a graphics viewing I will see that I
have a valid JPEG.

Pretty cool stuff.

You can also filter by TCP streams (but I believe you can't save as raw
from the TCP Streams page). Once you filter by TCP Stream, close the
Follow TCP Stream page. Now, you need to select the packet that has the
upper layer info you're interested in. There should only be one packet
like this. The rest of the packets will be flow start (SYN, SYN/ACK,
ACK), flow stop (FIN/ACK, ACK, FIN/ACK, ACK), and reassembled PDUs (TCP
Segment of a reassembled PDU), or maybe an occasional ReSeT. In my
case, I look for the one packet that says HTTP/1.1 200 OK (JPEG JFIF
image).

Hope that helps,
--Jim

-----Original Message-----
From: wireshark-users-***@wireshark.org
[mailto:wireshark-users-***@wireshark.org] On Behalf Of Pete Fraser
Sent: Monday, November 13, 2006 8:52 PM
To: wireshark-***@wireshark.org
Subject: [Wireshark-users] Exporting raw packet data?

I'm new to Wireshark, so sorry if this is a dumb question.

I want to export packet data in raw format, so that I end up with a
binary file.

If the packets are TCP I can use Analyze->Follow TCP Stream then Save As
Raw.
For any type of packet, I can select packet data in the bottom pane
and do File->Export->Selected Packet Bytes.

What I want to do, but can't work out how, is to export a lot of
packet data as a raw binary file.
I develop the appropriate filter so that only the packets of interest
are visible, then do File->Export->File..., select "All packets",
"Displayed", and "Packet Bytes" for the only Packet Format. I would
hope that I can then save as raw, but I only find ASCII, PS, XML,
etc. What am I doing wrong?

Thanks in advance.


_______________________________________________
Wireshark-users mailing list
Wireshark-***@wireshark.org
http://www.wireshark.org/mailman/listinfo/wireshark-users

[Pete Fraser]

At 07:05 PM 11/13/2006, Jim Small wrote:
>Pete,
>
>I didn't even realize you could do this until I read your question, but
>here is one way (not sure if this is exactly what you want):
>Open a capture
>Narrow down the interesting packets
>(For example, I do a lot of web traffic analysis so I might use a filter
>such as http.content_length > 20000)
>Now, let's say I see a Flash file, a GIF, or a JPEG that I want to save
>- just the actual binary data, not the packet headers.
>I would click on the interesting packet (assuming I have TCP and HTTP
>reassembly enabled)
>Next, in the packet details window (middle pane) I would click on the
>relevant data portion. So for a JPEG image this would be the part that
>reads JPEG File Interchange Format.
>Finally, I would use the File->Export->Selected Packet Bytes menu item.
>Then I would name the file and I personally change the save as type to
>*.* so I can set the file extension (not completely sure this is
>necessary but I do it out of habit).
>Now, if I open up this file with a graphics viewing I will see that I
>have a valid JPEG.
>
>Pretty cool stuff.

I think that would work for small amounts of data, but I'm dealing
with video streams over hundreds of packets.


>You can also filter by TCP streams (but I believe you can't save as raw
>from the TCP Streams page).

You can save as raw. It's great for video streams over TCP.
I was hoping for a similar capability for UDP streams, after I'd
applied a filter.



Thanks

Pete

[Small, James]

Replies in-line below...

> >I didn't even realize you could do this until I read your question,
but
> >here is one way (not sure if this is exactly what you want):
> >Open a capture
> >Narrow down the interesting packets
> >(For example, I do a lot of web traffic analysis so I might use a
filter
> >such as http.content_length > 20000)
> >Now, let's say I see a Flash file, a GIF, or a JPEG that I want to
save
> >- just the actual binary data, not the packet headers.
> >I would click on the interesting packet (assuming I have TCP and HTTP
> >reassembly enabled)
> >Next, in the packet details window (middle pane) I would click on the
> >relevant data portion. So for a JPEG image this would be the part
that
> >reads JPEG File Interchange Format.
> >Finally, I would use the File->Export->Selected Packet Bytes menu
item.
> >Then I would name the file and I personally change the save as type
to
> >*.* so I can set the file extension (not completely sure this is
> >necessary but I do it out of habit).
> >Now, if I open up this file with a graphics viewing I will see that I
> >have a valid JPEG.
> >
> >Pretty cool stuff.
>
> I think that would work for small amounts of data, but I'm dealing
> with video streams over hundreds of packets.
>

Out of curiosity, I just tried it on a 4.4MB video file and while a
little slow, it worked well. This is definitely a slick program!

>
> >You can also filter by TCP streams (but I believe you can't save as
raw
> >from the TCP Streams page).
>
> You can save as raw. It's great for video streams over TCP.
> I was hoping for a similar capability for UDP streams, after I'd
> applied a filter.
>

You're right of course - there is a save as raw option. I noticed
though that this option also saves the headers. Thus for a binary file
such as an image, you have to use a hex editor or binary editing program
so you don't corrupt the file when you remove the headers. The other
way it just saves the binary data so it's a small convenience that saves
you from removing the headers.

I agree that it would be nice to have something like this for UDP but
that means someone would have to write the dissector/re-assembler.
Probably not an easy task.

--Jim

[Stephen Fisher]

On Mon, Nov 13, 2006 at 11:03:19PM -0500, Small, James wrote:

> I agree that it would be nice to have something like this for UDP but
> that means someone would have to write the dissector/re-assembler.
> Probably not an easy task.

Feel free to add this to the wish list at
http://wiki.wireshark.org/WishList and perhaps someone will have a
chance to add the feature :)


Steve

[Hans Nilsson]

You could try saving it as a pcap-file and stripping out the headers. Or
exporting only the packet bytes as plain-text and using sed, awk or any
other tool to extract the right data.


On Mon, 13 Nov 2006 17:52:21 -0800, "Pete Fraser" <***@covad.net>
said:
> I'm new to Wireshark, so sorry if this is a dumb question.
>
> I want to export packet data in raw format, so that I end up with a
> binary file.
>
> If the packets are TCP I can use Analyze->Follow TCP Stream then Save As
> Raw.
> For any type of packet, I can select packet data in the bottom pane
> and do File->Export->Selected Packet Bytes.
>
> What I want to do, but can't work out how, is to export a lot of
> packet data as a raw binary file.
> I develop the appropriate filter so that only the packets of interest
> are visible, then do File->Export->File..., select "All packets",
> "Displayed", and "Packet Bytes" for the only Packet Format. I would
> hope that I can then save as raw, but I only find ASCII, PS, XML,
> etc. What am I doing wrong?
>
> Thanks in advance.
>
>
> _______________________________________________
> Wireshark-users mailing list
> Wireshark-***@wireshark.org
> http://www.wireshark.org/mailman/listinfo/wireshark-users
--
Hans Nilsson
***@ftml.net

--
http://www.fastmail.fm - Same, same, but different…

[Sake Blok]

On Mon, Nov 13, 2006 at 09:02:41PM -1100, Hans Nilsson wrote:
> You could try saving it as a pcap-file and stripping out the headers. Or
> exporting only the packet bytes as plain-text and using sed, awk or any
> other tool to extract the right data.

Based on your challenge, I wrote a little perl-script that I think
would do the trick.

The perl-script will take all udp-packets from a saved trace-file
and will extract the udp-payload to a file. If you use (wire|t)shark
to select only the UDP-stream that you want, I think it will produce
exactly what you are looking for :)

Cheers,


Sake

[Pete Fraser]

At 04:59 AM 11/14/2006, Sake Block wrote:



>Based on your challenge, I wrote a little perl-script that I think
>would do the trick.
>
>The perl-script will take all udp-packets from a saved trace-file
>and will extract the udp-payload to a file. If you use (wire|t)shark
>to select only the UDP-stream that you want, I think it will produce
>exactly what you are looking for :)


Thanks very much for doing this. Unfortunately I'm also new to perl.
I downloaded ActivePerl 819, and tried executing your script from a
command window, but it failed at line 9.
I then downloaded Net-PcapUtils-0.01, NetPacket-0.04 and
NetPcap-0.14, unpacked them, and placed the folders in
C:\Perl\site\lib, but that didn't work.
Could somebody point me to a turorial on how to install this stuff?


Pete

[Sake Blok]

On Tue, Nov 14, 2006 at 03:50:42PM -0800, Pete Fraser wrote:
> At 04:59 AM 11/14/2006, Sake Block wrote:

* Wonders how the C got in his last-name? ;) *


> >Based on your challenge, I wrote a little perl-script that I think
> >would do the trick.
>
> Thanks very much for doing this. Unfortunately I'm also new to perl.
> I downloaded ActivePerl 819, and tried executing your script from a
> command window, but it failed at line 9.
> I then downloaded Net-PcapUtils-0.01, NetPacket-0.04 and
> NetPcap-0.14, unpacked them, and placed the folders in
> C:\Perl\site\lib, but that didn't work.
> Could somebody point me to a turorial on how to install this stuff?

I used perl on a FreeBSD system. I'm not using ActivePerl on
windows. I'm afraid I can't help you out there.

Since I'm a cygwin-user, if someone knows how to get these
modules to work under cygwin, I'd too would be interested :)

Cheers,


Sake

[Nicoson Dave]

> Could somebody point me to a turorial on how to install this stuff?
> Pete

Try this:

ppm help quickstart

[Small, James]

Pretty cool Sake.

I don't have any UDP streams to coalesce at the moment, but just looking
at your perl script gave me some ideas.

Thanks,
--Jim

> -----Original Message-----
> From: wireshark-users-***@wireshark.org [mailto:wireshark-users-
> ***@wireshark.org] On Behalf Of Sake Blok
> Sent: Tuesday, November 14, 2006 7:59 AM
> To: Community support list for Wireshark
> Subject: Re: [Wireshark-users] Exporting raw packet data?
>
> On Mon, Nov 13, 2006 at 09:02:41PM -1100, Hans Nilsson wrote:
> > You could try saving it as a pcap-file and stripping out the
headers. Or
> > exporting only the packet bytes as plain-text and using sed, awk or
any
> > other tool to extract the right data.
>
> Based on your challenge, I wrote a little perl-script that I think
> would do the trick.
>
> The perl-script will take all udp-packets from a saved trace-file
> and will extract the udp-payload to a file. If you use (wire|t)shark
> to select only the UDP-stream that you want, I think it will produce
> exactly what you are looking for :)
>
> Cheers,
>
>
> Sake