fixhdr breaking mseeds with variable record lengths
Data group has encountered at least two experiments where miniseed files with variable record length (variable byte size) are effectively corrupted by operating on their header values with fix header. The resulting files are unreadable by either PQL or python/obspy in some cases, and in cases where they can be read the trace data is destroyed/lost. Header information w/i the files does seem to be preserved, and even files which can't be opened by a waveform view can be parsed by msi/fixhdr etc (programs which just read headers).
Errors when trying to open affected files with a waveform viewer are of the following nature. This snippet is from attempting to load w/ pql, but obspy errors are similar in their content. Loading produces many more errors than this (one for each blockette I assume), but they are all of a similar nature:
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3504 samples of 3510 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=156519588, Xn=304
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3551 samples of 3553 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=278568843, Xn=104
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3589 samples of 3595 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=506010270, Xn=110
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3570 samples of 3578 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=365871865, Xn=202
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3559 samples of 3565 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=2133, Xn=73
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=2591, Xn=271
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=140140461, Xn=-204
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3563 samples of 3565 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=1419, Xn=118
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=435085330, Xn=215
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3576 samples of 3584 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=313036712, Xn=310
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3528 samples of 3532 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=296657097, Xn=27
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3552 samples of 3556 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=382250066, Xn=-173
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3495 samples of 3501 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=140140582, Xn=66
Error: msr_unpack_data(XX_ITBAD__CHZ_D): only decoded 3504 samples of 3505 expected
XX_ITBAD__CHZ_D: Warning: Data integrity check for Steim2 failed, Last sample=3003, Xn=293
The source files are all from recent deployments of pegasus dataloggers, we are unsure if this issue is platform ambiguous and would affect any mseed with variable record length, or whether there is anything specific about the pegasus generated files. However mseeds from the same station/deployment/logger which do not have variable record length do not seem to be affected by the issue, so we believe that the variable record lengths are likely relevant here.
It is unknown what version of fixhdr has been used in all cases, but 2023.3.0 causes this issue when tested locally. fixhdr does not produce any errors when operating on the files.
EPIC data group can provide example files.