Read two data points instead of one

Currently, we follow logpeek and qpeek's example and read only one data point from each data record/packet. In order to solve #154 (closed), we now read two data points.

Closes #154 (closed)

There are some issues that arise from this merge request:

• The data is not downsampled after being read anymore. This is in response to a problem with the downsampling algorithm. More details can be found in a discussion in this MR.
• A new way of calculating end time of a data record/packet causes #229 (closed). This will be fixed in another MR.

assigned to @kienle

requested review from @ldam

I run different data set to compare between this MR and master with wf and SOH

Centaur-DataTest key Edit:

	master	this MR
total read	206	275
total plot	0.0436	1.1288
total plot	0.0333	1.0848

Q330 5083

key 2468

	master	this MR
total read	273.66	370.14
total plot	0.216	0.0434
total plot	0.0543	0.0555

key ICEZ

	master	this MR
total plot	0.0555	1.1842
total plot	0.0559	1.2879

Q330 5281

	master	this MR
total read	183.07	189.32
total plot	0.0361	0.0315
total plot	0.0451	0.0446

For Q330 5281, plotting isn't correct for HH* channels:

In master:

This MR:

This is a problem with the downsampling algorithm. Currently, the downsampling threshold is 1 million points. The first case has fewer points than that per graph, so no downsampling happened. The second case, however, has double the points in each graph, which mean the data was downsampled. To fix this, the downsampling process was disabled.

added 1 commit

• da6aa1b8 - Stop downsampling data

Compare with previous version

requested review from @ldam

This is the comparison of the time of reading and plotting data between this MR and master.

Timing code

general_data.py/reftek.py

    def processing_data(self):
        import time
        start = time.time()
        if self.creator_thread.isInterruptionRequested():
            raise ThreadStopped()
        self.read_folders()
        step1 = time.time()
        read_folder = step1 - start
        self.selected_key = self.select_key()
        step2 = time.time()
        self.fill_empty_data()
        if self.creator_thread.isInterruptionRequested():
            raise ThreadStopped()
        self.finalize_data()
        end = time.time()
        finalize = end - step2
        print("read:", (read_folder + finalize))
        """
        rt130
        end = time.time()
        finalize = end - start
        print("read:", finalize)
        """

multi_threaded_plotting_widget.py

    def plot_channels(self, d_obj, key, start_tm, end_tm, time_ticks_total,
                      pref_order=[]):
        self.start = time.time()
        ...

        def done(self):
        ...
        self.end = time.time()
        print(f"plot {self.name}:", (self.end - self.start))

Centaur-DataTest key Edit:

	master	this MR
read	184	304
plot WF	7.37	18.63
plot SOH	8.81	23.71

Q330 5083

key 2468

	master	this MR
total read	272.36	275.22
plot SOH	3.14	3.39
plot WF	4.03	4.32

key ICEZ

	master	this MR
plot SOH	15.55	17.34
plot WF	17.30	20.13

Q330 5281

	master	this MR
total read	182.85	188.45
plot WF	11.18	15.01
plot SOH	11.89	15.50

RT130 2016174.9AC4

	master	this MR
total read	70.05	59.90
plot WF	4.60	6.58
plot SOH	6.27	8.24

RT130 9926

	master	this MR
total read	37.92	30.74
plot WF	3.69	4.39
plot SOH	5.06	5.85

6407.sdr

	master	this MR
total read	12.61	14.33
plot WF	11.69	11.12
plot SOH	12.81	12.37

Should this line be removed?

changed this line in version 3 of the diff

In the description, you should give the explanation why there are more gaps for Centaur in this MR.

RT130 9926: The first value of SOH/Data Def doesn't show up in this MR

mentioned in merge request !240 (merged)

I think the code is ok. You can resolve the last thread and fix failed unittests and this will be ready to be merged.

added 45 commits

• da6aa1b8...aeb30088 - 38 commits from branch master
• 25b83719 - Fix record end time calculation
• 3a6049a6 - Get two MSeed data points instead of one
• d986943d - Fix tests
• bc249981 - Get two data points from RT130 data
• e1bb5573 - Fix reading non-compressed MSeed data
• 0e2bc30c - Stop downsampling data
• eec52728 - Remove unneeded print

Compare with previous version

changed the description

added 2 commits

• d3a58e89 - Test unneeded tests
• 9a2d690f - flake8

Compare with previous version

added 1 commit

• 6d544389 - flake8

Compare with previous version

requested review from @ldam

Let's remove this feature for RT130 data for now. It only works on waveform data, and due to the high sample rate, the chance of #154 (closed) happening is pretty low.

added 2 commits

• f2ab6635 - Revert "Get two data points from RT130 data"
• f69552a1 - Import needed type

Compare with previous version

added 14 commits

• f69552a1...e30769c0 - 2 commits from branch master
• e30769c0...9af969c8 - 2 earlier commits
• d6dc27ea - Fix tests
• 08e0ccfa - Get two data points from RT130 data
• 082adad7 - Fix reading non-compressed MSeed data
• c3abaed4 - Stop downsampling data
• e7984f91 - Remove unneeded print
• 47860d39 - Test unneeded tests
• 18574a9a - flake8
• 7cc5941e - flake8
• 86a00b55 - Revert "Get two data points from RT130 data"
• e0a8144a - Import needed type

Compare with previous version

docstring please.