mdf3reader module documentation — mdfreader 2.8 documentation

# mdf3reader module documentation

Measured Data Format file reader module for version 3.x

## Platform and python version

With Unix and Windows for python 2.6+ and 3.2+

Author: | Aymeric Rateau

Created on Sun Oct 10 12:57:28 2010

## Dependencies

- Python >2.6, >3.2 <http://www.python.org>
- Numpy >1.6 <http://numpy.scipy.org>
- Sympy to convert channels with formula

## Attributes

PythonVersion : float
  Python version currently running, needed for compatibility of both python 2.6+ and 3.2+

## mdf3reader module

class `mdfreader.mdf3reader.` `DATA`(fid, pointer)

  Bases: `dict`

  DATA class is organizing record classes itself made of channel. This class inherits from dict. Keys are corresponding to channel group recordID. A DATA class corresponds to a data block, a dict of record classes (one per channel group). Each record class contains a list of channel class representing the structure of channel record.

  Attributes

  fid | (io.open) file identifier
  pointerToData | (int) position of Data block in mdf file
  BlockLength | (int) total size of data block

  Methods

  addRecord(record) | Adds a new record in DATA class dict
  read(channelSet) | Reads data block
  loadSorted(record, nameList=None) | Reads sorted data block from record definition
  loadUnSorted(nameList=None) | Reads unsorted data block, not yet implemented

  `addRecord`(record)

    Adds a new record in DATA class dict

    Parameters: | record class channel group definition listing record channel classes

  `loadSorted`(record, nameList=None)

    Reads sorted data block from record definition

    Parameters: | record class channel group definition listing record channel classes channelSet : set of str, optional list of channel names
    Returns: | numpy recarray of data

  `loadUnSorted`(nameList=None)

    Reads unsorted data block from record definition

    Parameters: | record class channel group definition listing record channel classes channelSet : set of str, optional list of channel names
    Returns: | numpy recarray of data

  `read`(channelSet, filename)

    Reads data block

    Parameters: | channelSet : set of str, optional list of channel names filename : str name of file

`mdfreader.mdf3reader.` `expConv`(data, conv)

  apply exponential conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `formulaConv`(data, conv)

  apply formula conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

  Notes

  Requires sympy module

`mdfreader.mdf3reader.` `linearConv`(data, conv)

  apply linear conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `logConv`(data, conv)

  apply logarithmic conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

class `mdfreader.mdf3reader.` `mdf3`(fileName=None, channelList=None, convertAfterRead=True, filterChannelNames=False, noDataLoading=False, compression=False, convertTables=False, metadata=2)

  Bases: `mdfreader.mdf.mdf_skeleton`

  mdf file version 3.0 to 3.3 class

  Attributes

  fileName | (str) file name
  MDFVersionNumber | (int) mdf file version number
  masterChannelList | (dict) Represents data structure: a key per master channel with corresponding value containing a list of channels One key or master channel represents then a data group having same sampling interval.
  multiProc | (bool) Flag to request channel conversion multi processed for performance improvement. One thread per data group.
  convertAfterRead | (bool) flag to convert raw data to physical just after read
  filterChannelNames | (bool) flag to filter long channel names from its module names separated by ‘.’
  file_metadata | (dict) file metadata with minimum keys: author, organisation, project, subject, comment, time, date

  Methods

  read3( fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True) | Reads mdf 3.x file data and stores it in dict
  _getChannelData3(channelName) | Returns channel numpy array
  _convertChannel3(channelName) | converts specific channel from raw to physical data according to CCBlock information
  _convertAllChannel3() | Converts all channels from raw data to converted data according to CCBlock information
  write3(fileName=None) | Writes simple mdf 3.3 file

  `read3`(fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True, filterChannelNames=False, compression=False, metadata=2)

    Reads mdf 3.x file data and stores it in dict

    Parameters: | fileName : str, optional file name info : mdfinfo3.info3 class info3 class containing all MDF Blocks multiProc : bool flag to activate multiprocessing of channel data conversion channelList : list of str, optional list of channel names to be read If you use channelList, reading might be much slower but it will save you memory. Can be used to read big files convertAfterRead : bool, optional flag to convert channel after read, True by default If you use convertAfterRead by setting it to false, all data from channels will be kept raw, no conversion applied. If many float are stored in file, you can gain from 3 to 4 times memory footprint To calculate value from channel, you can then use method .getChannelData() filterChannelNames : bool, optional flag to filter long channel names from its module names separated by ‘.’ compression : bool, optional flag to activate data compression with blosc metadata: int, optional, default = 2 Reading metadata has impact on performance, especially for mdf 4.x using xml. 2: minimal metadata reading (mostly channel blocks) 1: used for noDataLoading 0: all metadata reading

  `write3`(fileName=None)

    Writes simple mdf 3.3 file

    Parameters: | fileName : str, optional Name of file If file name is not input, written file name will be the one read with appended ‘_new’ string before extension

    Notes

    All channels will be converted to physical data, so size might be bigger than original file

`mdfreader.mdf3reader.` `polyConv`(data, conv)

  apply polynomial conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `rationalConv`(data, conv)

  apply rational conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

class `mdfreader.mdf3reader.` `record`(dataGroup, channelGroup)

  Bases: `list`

  record class lists Channel classes,
    it is representing a channel group

  Attributes

  CGrecordLength | (int) length of record from channel group block information in Byte
  recordLength | (int) length of record from channels information in Byte
  numberOfRecords | (int) number of records in data block
  recordID | (int) recordID corresponding to channel group
  recordIDnumber | (int) size of recordID
  dataGroup | (int:) data group number
  channelGroup | (int) channel group number
  numpyDataRecordFormat | (list) list of numpy (dtype) for each channel
  dataRecordName | (list) list of channel names used for recarray attribute definition
  master | (dict) define name and number of master channel
  recordToChannelMatching | (dict) helps to identify nested bits in byte
  channelNames | (set) channel names to be stored, useful for low memory consumption but slow
  hiddenBytes | (Bool, False by default) flag in case of non declared channels in record
  byte_aligned | (Bool, True by default) flag for byte aligned record

  Methods

  addChannel(info, channelNumber) |
  loadInfo(info) |
  readSortedRecord(fid, pointer, channelSet=None) |
  readRecordBuf(buf, channelSet=None) |
  readRecordBits(bita, channelSet=None) |

  `addChannel`(info, channelNumber)

    add a channel in class

    Parameters: | info : mdfinfo3.info3 class channelNumber : int channel number in mdfinfo3.info3 class

  `loadInfo`(info)

    gathers records related from info class

    Parameters: | info : mdfinfo3.info3 class

  `readRecordBits`(bita, channelSet=None)

    read stream of record bits by bits in case of not aligned or hidden bytes

    Parameters: | buf : stream stream of bytes read in file channelSet : Set of str, optional list of channel to read
    Returns: | rec : dict returns dictionary of channel with its corresponding values

  `readRecordBuf`(buf, channelSet=None)

    read stream of record bytes

    Parameters: | buf : stream stream of bytes read in file channelSet : Set of str, optional list of channel to read
    Returns: | rec : dict returns dictionary of channel with its corresponding values

  `readSortedRecord`(fid, pointer, channelSet=None)

    reads record, only one channel group per datagroup

    Parameters: | fid : float file identifier pointer position in file of data block beginning channelSet : Set of str, optional list of channel to read
    Returns: | rec : numpy recarray contains a matrix of raw data in a recarray (attributes corresponding to channel name)

    Notes

    If channelSet is None, read data using numpy.core.records.fromfile that is rather quick. However, in case of large file, you can use channelSet to load only interesting channels or only one channel on demand, but be aware it might be much slower.

`mdfreader.mdf3reader.` `tabConv`(data, conv)

  apply Tabular conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `tabInterpConv`(data, conv)

  apply Tabular interpolation conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `textRangeTableConv`(data, conv)

  apply text range table conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value

`mdfreader.mdf3reader.` `textTableConv`(data, conv)

  apply text table conversion to data

  Parameters: | data : numpy 1D array raw data to be converted to physical value conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
  Returns: | converted data to physical value