Concatenating segmented files in NPMK
Occasionally during recording sessions, users may encounter brief packet loss or they may pause and resume file recording themselves. When this happens, Central File Recording will write a new header to the file, causing the data to become segmented and will be formatted slightly differently from standard files when opened in MATLAB using Blackrock's MATLAB file loader.
In order to combine the different segments into one continuous segment you can then use the MATLAB commands nan() (which creates a NaN array of a specified length) and horzcat() (which horizontally concatenates the input matrices)
The NSx.Data field, instead of being a single n x m matrix (n being the number of channels and m being the number of samples), will be segmented into multiple cells, one for each break in the file:
In the MetaTags file, the NSx.MetaTags.TimeStamp field will be populated with multiple values - one for each cell in the file. Each value in the TimeStamp field corresponds with the timestamp of the first sample of each cell.
In the previous screenshot, we can see that the Timestamp of the second segment is 1023677 and the first value of DataPoints is 997080, meaning that the first segment has 997080 values in it. Because the Blackrock system is capable at recording files at different sampling rates, we need to ensure that the DataPoints value and Timestamp value are directly comparable before performing operations on them. In order to do so, we multiply the DataPoints value by a conversion factor of NS5.MetaTags.TimeRes / NS5.MetaTags.SamplingFreq (in this case 30000/30000 = 1). Since the timestamp of the first value of the second segment is 1023677, we now know that the file was paused for 1023677-997080-1 = 26596 samples, corresponding to the gap between the first segment and the second segment.
In order to find the gap between the second segment and third segment, we need to look at the second value of the DataPoints field and the third value of the Timestamp field. From the DataPoints field, we can see that there were 628466 samples in the second segment - to find the timestamp of the last sample of the second segment, we add this to the first value of the DataPoints field (997080) and the number of samples in the gap between the first and second segment (26596), for a total of 1652142. The first sample of the third segment has a timestamp of 1683229, so we subtract 1683229-1652142-1 = 31086, meaning there is a gap of 31086.
The following commands take the first three segments and the gap lengths we have found, convert them from timestamps to Datapoints by dividing them by the conversion factor and concatenate them into a single data matrix for convenience. The variable numChans will depend on the number of channels in NS5.Data
numChans = 16;
conversionFactor = NS5.MetaTags.TimeRes/NS5.MetaTags.SamplingFreq;
A = nan(numChans, 26596/conversionFactor) %creates a nan array of the same length as our first gap
B = nan(numChans, 31086/conversionFactor) %creates a nan array of the same length as our second gap
x = horzcat(NS5.Data{1}, A, NS5.Data{2}, B, NS5.Data{3})
Note: If you would prefer to instead split the segments into separate files instead of combining them into a single segment, this can be done with the NPMK function splitNSxPauses.
Documentation on the MATLAB functions utilized:
Blackrock's MATLAB loader can be found here:
For an introduction to our MATLAB loader, see the following video:
Related Articles
Loading Data into MClust
PS: for MClust 4.0+ please download and use NPMK 4.2.2 Please also find attached PDF version with screenshots -------------------------------------------------------------------------------------- 1, Download MClust (V3.5 or later) and the latest ...MATLAB Interfaces For Blackrock Microsystems Data Acquisition Systems
Introduction Blackrock has several functions for looking at and interacting with data collected through our neural signal processors. These functions are available for post-recording processing (Neural Processing MATLAB Kit - NPMK) or for interacting ...Loading Blackrock Event and Continuous Data Files into Plexon Offline Sorter (POS or OFS)
Installation Note: Version 4.0+ of offline sorter can natively load .NEV files, so the steps below do not need to be followed. Note: Plexon's offline sorter is not compatible with Blackrock's File Spec 3.0 (Available on Central version 7.5.0 and ...PTP Data Alignment
Note: If you are recording data with both the NSP and Gemini hubs, update to 7.6.1. Background Timestamping has fundamentally changed from operating on a nominal 30 kHz clock to Precision Time Protocol (PTP) on Gemini hardware. This is a departure ...'nozeropad' in openNSx()
Occasionally in Blackrock continuous files, the timestamp of the first datapoint written to the file will be non-zero. By default, when opening files with non-zero first timestamp using the openNSx() function from NPMK, the function will modify the ...