THIS IS A TEST INSTANCE. Feel free to ask and answer questions, but take care to avoid triggering too many notifications.

Revision history  [back]

editcap file splitting issue when little data

I am experiencing an issue with editcap that I was hoping someone could help with.  I have a very large pcap file say 20Gb that I split up into 1 sec .pcapng chunks.  The data captured in the large file represents a large amount of traffic flowing through a connection.  The data rate is equal to the line speed of the given connection.   I have found an occasional problem that occurs (sometimes) when the traffic source stops communicating.  When the traffic source is not operating I get very little data.  I would have expected that if a given 1 sec chunk contained no data I would either get an empty .pcapng file for the given time period or I would get no .pcapng at all.  I could live with either of these x2 scenarios.  However what I actually get is something quite different.  I have copied the file listing below with the file name, file size in bytes and how many bytes are shown in the given .pcapng file when opened in Wireshark. Consider the lines below where I am missing a 20230216074903.pcapng but then get x2  20230216074905 files.  I should say that this occurs only when the traffic source has stopped communicating on the network ie during period with very few packets per second say 0 to 3 per second.

The editcap command used is of the following format - "editcap Largefilepath.pcapng Smallfilepath.pcapng -i 1" in the example output below.

editcap Largefilepath.pcapng Smallfilepath.pcapng -i 1

C:\testdata\small_00877_20230216074901.pcapng   2284    19 packets
C:\testdata\small_00878_20230216074902.pcapng   912 6 packets
C:\testdata\small_00879_20230216074904.pcapng   388 1 packet
C:\testdata\small_00880_20230216074905.pcapng   296 Empty
C:\testdata\small_00881_20230216074905.pcapng   744 2 packets
C:\testdata\small_00882_20230216074908.pcapng   296 Empty
C:\testdata\small_00883_20230216074908.pcapng   388 1 packet
C:\testdata\small_00884_20230216074910.pcapng   296 Empty
C:\testdata\small_00885_20230216074910.pcapng   388 1 packet
C:\testdata\small_00886_20230216074910.pcapng   652 1 packet
C:\testdata\small_00887_20230216074912.pcapng   388 1 packet
C:\testdata\small_00888_20230216074914.pcapng   296 Empty
C:\testdata\small_00889_20230216074914.pcapng   388 1 packet
C:\testdata\small_00890_20230216074915.pcapng   296 Empty
C:\testdata\small_00891_20230216074915.pcapng   744 1 packet
C:\testdata\small_00892_20230216074918.pcapng   296 Empty
C:\testdata\small_00893_20230216074918.pcapng   388 1 packet
C:\testdata\small_00894_20230216074920.pcapng   296 Empty
C:\testdata\small_00895_20230216074920.pcapng   388 1 packet
C:\testdata\small_00896_20230216074920.pcapng   652 1 packet
C:\testdata\small_00897_20230216074922.pcapng   388 1 packet
C:\testdata\small_00898_20230216074924.pcapng   296 Empty
C:\testdata\small_00899_20230216074924.pcapng   388 1 packet
C:\testdata\small_00900_20230216074925.pcapng   296 Empty
C:\testdata\small_00901_20230216074925.pcapng   744 2 packets
C:\testdata\small_00902_20230216074928.pcapng   296 Empty
C:\testdata\small_00903_20230216074928.pcapng   388 1 packet
C:\testdata\small_00904_20230216074930.pcapng   296 Empty
C:\testdata\small_00905_20230216074930.pcapng   388 1 packet
C:\testdata\small_00906_20230216074930.pcapng   652 1 packet

I tried repeating the above but instead of 1 sec chunks I tried 2 sec and 5 secs. When I do this I get the same sort of issues albeit not always at the same points in time.

Has anyone got any experience of this or has an idea of what could be happening?

Wireshark 3.6.3 (v3.6.3-0-g6d348e4611e2)

Copyright 1998-2022 Gerald Combs [email protected] and contributors. License GPLv2+: GNU GPL version 2 or later https://www.gnu.org/licenses/gpl-2.0.html This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) using Microsoft Visual Studio 2019 (VC++ 14.29, build 30139), with Qt 5.15.2, with libpcap, with GLib 2.66.4, with zlib 1.2.11, with Lua 5.2.4, with GnuTLS 3.6.3 and PKCS #11 support, with Gcrypt 1.8.3, with MIT Kerberos, with MaxMind DB resolver, with nghttp2 1.44.0, with brotli, with LZ4, with Zstandard, with Snappy, with libxml2 2.9.10, with libsmi 0.4.8, with QtMultimedia, with automatic updates using WinSparkle 0.5.7, with AirPcap, with SpeexDSP (using bundled resampler), with Minizip.

Running on 64-bit Windows 10 (21H2), build 19044, with Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (with SSE4.2), with 130823 MB of physical memory, with GLib 2.66.4, with Qt 5.15.2, with Npcap version 1.55, based on libpcap version 1.10.2-PRE-GIT, with c-ares 1.17.0, with GnuTLS 3.6.3, with Gcrypt 1.8.3, with nghttp2 1.44.0, with brotli 1.0.9, with LZ4 1.9.3, with Zstandard 1.4.0, without AirPcap, with LC_TYPE=English_United Kingdom.utf8, binary plugins supported (0 loaded).

editcap file splitting issue when little data

I am experiencing an issue with editcap that I was hoping someone could help with.  I have a very large pcap file say 20Gb that I split up into 1 sec .pcapng chunks.  The data captured in the large file represents a large amount of traffic flowing through a connection.  The data rate is equal to the line speed of the given connection.   I have found an occasional problem that occurs (sometimes) when the traffic source stops communicating.  When the traffic source is not operating I get very little data.  I would have expected that if a given 1 sec chunk contained no data I would either get an empty .pcapng file for the given time period or I would get no .pcapng at all.  I could live with either of these x2 scenarios.  However what I actually get is something quite different.  I have copied the file listing below with the file name, file size in bytes and how many bytes are shown in the given .pcapng file when opened in Wireshark. Consider the lines below where I am missing a 20230216074903.pcapng but then get x2  20230216074905 files.  I should say that this occurs only when the traffic source has stopped communicating on the network ie during period with very few packets per second say 0 to 3 per second.

The editcap command used is of the following format - "editcap Largefilepath.pcapng Smallfilepath.pcapng -i 1" in the example output below.

editcap Largefilepath.pcapng Smallfilepath.pcapng -i 1

C:\testdata\small_00877_20230216074901.pcapng 2284 19 packets C:\testdata\small_00878_20230216074902.pcapng 912 6 packets C:\testdata\small_00879_20230216074904.pcapng 388 1 packet C:\testdata\small_00880_20230216074905.pcapng 296 Empty C:\testdata\small_00881_20230216074905.pcapng 744 2 packets C:\testdata\small_00882_20230216074908.pcapng 296 Empty C:\testdata\small_00883_20230216074908.pcapng 388 1 packet C:\testdata\small_00884_20230216074910.pcapng 296 Empty C:\testdata\small_00885_20230216074910.pcapng 388 1 packet C:\testdata\small_00886_20230216074910.pcapng 652 1 packet C:\testdata\small_00887_20230216074912.pcapng 388 1 packet C:\testdata\small_00888_20230216074914.pcapng 296 Empty C:\testdata\small_00889_20230216074914.pcapng 388 1 packet C:\testdata\small_00890_20230216074915.pcapng 296 Empty C:\testdata\small_00891_20230216074915.pcapng 744 1 packet C:\testdata\small_00892_20230216074918.pcapng 296 Empty C:\testdata\small_00893_20230216074918.pcapng 388 1 packet C:\testdata\small_00894_20230216074920.pcapng 296 Empty C:\testdata\small_00895_20230216074920.pcapng 388 1 packet C:\testdata\small_00896_20230216074920.pcapng 652 1 packet C:\testdata\small_00897_20230216074922.pcapng 388 1 packet C:\testdata\small_00898_20230216074924.pcapng 296 Empty C:\testdata\small_00899_20230216074924.pcapng 388 1 packet C:\testdata\small_00900_20230216074925.pcapng 296 Empty C:\testdata\small_00901_20230216074925.pcapng 744 2 packets C:\testdata\small_00902_20230216074928.pcapng 296 Empty C:\testdata\small_00903_20230216074928.pcapng 388 1 packet C:\testdata\small_00904_20230216074930.pcapng 296 Empty C:\testdata\small_00905_20230216074930.pcapng 388 1 packet C:\testdata\small_00906_20230216074930.pcapng 652 1 packet

packet

I tried repeating the above but instead of 1 sec chunks I tried 2 sec and 5 secs. When I do this I get the same sort of issues albeit not always at the same points in time.

Has anyone got any experience of this or has an idea of what could be happening?

Wireshark 3.6.3 (v3.6.3-0-g6d348e4611e2)

Copyright 1998-2022 Gerald Combs [email protected] and contributors. License GPLv2+: GNU GPL version 2 or later https://www.gnu.org/licenses/gpl-2.0.html This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) using Microsoft Visual Studio 2019 (VC++ 14.29, build 30139), with Qt 5.15.2, with libpcap, with GLib 2.66.4, with zlib 1.2.11, with Lua 5.2.4, with GnuTLS 3.6.3 and PKCS #11 support, with Gcrypt 1.8.3, with MIT Kerberos, with MaxMind DB resolver, with nghttp2 1.44.0, with brotli, with LZ4, with Zstandard, with Snappy, with libxml2 2.9.10, with libsmi 0.4.8, with QtMultimedia, with automatic updates using WinSparkle 0.5.7, with AirPcap, with SpeexDSP (using bundled resampler), with Minizip.

Running on 64-bit Windows 10 (21H2), build 19044, with Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (with SSE4.2), with 130823 MB of physical memory, with GLib 2.66.4, with Qt 5.15.2, with Npcap version 1.55, based on libpcap version 1.10.2-PRE-GIT, with c-ares 1.17.0, with GnuTLS 3.6.3, with Gcrypt 1.8.3, with nghttp2 1.44.0, with brotli 1.0.9, with LZ4 1.9.3, with Zstandard 1.4.0, without AirPcap, with LC_TYPE=English_United Kingdom.utf8, binary plugins supported (0 loaded).