Hello,
I am trying to map HTTP Requests to Responses from a pcap file. I use the following script =>
I'd like to put each request/response into a csv file with some attributes (Host, User-Agent, Status Code, Content-Length, etc...). Is it possible to do that directly with tshark ?
Use the -T fields option to selectively output fields. For example:
tshark -r "$1" -T fields -e http.server -e http.user_agent -e http.response.code -e http.content_length_header -e http.response_for.uri -Y http
Note: follow tcp-stream cannot be used if you want CSV format.
And in your example there is a space between 'ascii "$stream' that should be removed. See also tshark documentation.
I tried to use the -T fields (see example below) but I got a large number of empty lines : it seems that there is a line by stream (http or not http ...) =>
for stream in `tshark -r "$1" -2 -R "tcp and (http.request or http.response)" -T fields -e tcp.stream | sort -n | uniq`
do
tshark -q -r "$1" -z follow,http,ascii,"$stream" -T fields -e http.server -e http.user_agent >> results.txt
done
You omitted the display filter (-Y). So for every packet that does not contain a requested field and empty field is outputted, resulting in a lot of lines with only tabs.
Update : This script meets my need (the "json.txt" file contains only the pairs requests/responses in JSON format) =>
for stream in `tshark -r "$1" -2 -R "tcp and (http.request or http.response)" -T fields -e tcp.stream | sort -n | uniq`
do
tshark -q -r "$1" -z follow,http,ascii,"$stream" -Y "tcp.stream == "$stream" and (tcp and (http.request or http.response))" -T json -j "http" >> results.txt
done
sed "/"==================================================================="/,/"==================================================================="/d" results.txt > json.txt
Instead of using the sed command to remove unwanted text, it is better to instruct tshark not to output it in the first place by removing the -z option (and -q).
I don’t see the need for the for-loop. You can output all http requests and responses in one go.
Comments