Navigation

mdf3reader module documentation

Measured Data Format file reader module for version 3.x

Platform and python version

With Unix and Windows for python 2.6+ and 3.2+

Author:Aymeric Rateau

Created on Sun Oct 10 12:57:28 2010

Dependencies

Attributes

PythonVersion : float
Python version currently running, needed for compatibility of both python 2.6+ and 3.2+

mdf3reader module

class mdf3reader.DATA(fid, pointer)

Bases: dict

DATA class is organizing record classes itself made of recordchannel. This class inherits from dict. Keys are corresponding to channel group recordID A DATA class corresponds to a data block, a dict of record classes (one per channel group) Each record class contains a list of recordchannel class representing the structure of channel record.

Attributes

fid (io.open) file identifier
pointerToData (int) position of Data block in mdf file

Methods

addRecord(record) Adds a new record in DATA class dict
read(channelList, zip=None) Reads data block
loadSorted(record, zip=None, nameList=None) Reads sorted data block from record definition
load(nameList=None) Reads unsorted data block, not yet implemented
addRecord(record)

Adds a new record in DATA class dict

Parameters:

record class

channel group definition listing record channel classes
load(nameList=None)

not yet implemented

loadSorted(record, zip=None, nameList=None)

Reads sorted data block from record definition

Parameters:

record class

channel group definition listing record channel classes

zip : bool, optional

flag to track if data block is compressed

channelList : list of str, optional

list of channel names
Returns:

numpy recarray of data
read(channelList, zip=None)

Reads data block

Parameters:

channelList : list of str, optional

list of channel names

zip : bool, optional

flag to track if data block is compressed
mdf3reader.arrayformat3(signalDataType, numberOfBits)

function returning numpy style string from channel data type and number of bits Parameters —————- signalDataType : int

channel data type according to specification
numberOfBits : int
number of bits taken by channel data in a record
Returns:

dataType : str

numpy dtype format used by numpy.core.records to read channel raw data
mdf3reader.datatypeformat3(signalDataType, numberOfBits)

function returning C format string from channel data type and number of bits

Parameters:

signalDataType : int

channel data type according to specification

numberOfBits : int

number of bits taken by channel data in a record
Returns:

dataType : str

C format used by fread to read channel raw data
mdf3reader.expConv(data, conv)

apply exponential conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
mdf3reader.formulaConv(data, conv)

apply formula conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value

Notes

Requires sympy module

mdf3reader.linearConv(data, conv)

apply linear conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
mdf3reader.logConv(data, conv)

apply logarithmic conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
class mdf3reader.mdf3(fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True, filterChannelNames=False)

Bases: dict

mdf file version 3.0 to 3.3 class

Attributes

fileName (str) file name
VersionNumber (int) mdf file version number
masterChannelList (dict) Represents data structure: a key per master channel with corresponding value containing a list of channels One key or master channel represents then a data group having same sampling interval.
multiProc (bool) Flag to request channel conversion multi processed for performance improvement. One thread per data group.
convertAfterRead (bool) flag to convert raw data to physical just after read
filterChannelNames (bool) flag to filter long channel names from its module names separated by ‘.’
author (str)
organisation (str)
project (str)
subject (str)
comment (str)
time (str)
date (str)

Methods

read3( fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True) Reads mdf 3.x file data and stores it in dict
getChannelData3(channelName) Returns channel numpy array
convertChannel3(channelName) converts specific channel from raw to physical data according to CCBlock information
convertAllChannel3() Converts all channels from raw data to converted data according to CCBlock information
write3(fileName=None) Writes simple mdf 3.3 file
convert3(channelName)

converts specific channel from raw to physical data according to CCBlock information

Parameters:

channelName : str

Name of channel
Returns:

numpy array

returns numpy array converted to physical values according to conversion type
convertAllChannel3()

Converts all channels from raw data to converted data according to CCBlock information Converted data will take more memory.

convertChannel3(channelName)

converts specific channel from raw to physical data according to CCBlock information

Parameters:

channelName : str

Name of channel
getChannelData3(channelName)

Returns channel numpy array

Parameters:

channelName : str

channel name

Notes

This method is the safest to get channel data as numpy array from ‘data’ dict key might contain raw data

read3(fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True)

Reads mdf 3.x file data and stores it in dict

Parameters:

fileName : str, optional

file name

info : mdfinfo3.info3 class

info3 class containing all MDF Blocks

multiProc : bool

flag to activate multiprocessing of channel data conversion

channelList : list of str, optional

list of channel names to be read If you use channelList, reading might be much slower but it will save you memory. Can be used to read big files

convertAfterRead : bool, optional

flag to convert channel after read, True by default If you use convertAfterRead by setting it to false, all data from channels will be kept raw, no conversion applied. If many float are stored in file, you can gain from 3 to 4 times memory footprint To calculate value from channel, you can then use method .getChannelData()
write3(fileName=None)

Writes simple mdf 3.3 file

Parameters:

fileName : str, optional

Name of file If file name is not input, written file name will be the one read with appended ‘_new’ string before extension

Notes

All channels will be converted to physical data, so size might be bigger than original file

mdf3reader.polyConv(data, conv)

apply polynomial conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
mdf3reader.processDataBlocks(Q, buf, info, dataGroup, channelList, multiProc)

Put raw data from buf to a dict L and processes nested nBit channels

Parameters:

Q : multiprocessing.Queue, optional

Queue for multiprocessing

buf : DATA class

contains raw data

info : info class

contains infomation from MDF Blocks

dataGroup : int

data group number according to info class

channelList : list of str, optional

list of channel names to be processed

multiProc : bool

flag to return Queue or dict
Returns:

Q : multiprocessing.Queue

updates Queue containing L dict

L : dict

dict of channels
mdf3reader.rationalConv(data, conv)

apply rational conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
class mdf3reader.record(dataGroup, channelGroup)

Bases: list

record class lists recordchannel classes, it is representing a channel group

Attributes

recordLength (int) length of record corresponding of channel group in Byte
numberOfRecords (int) number of records in data block
recordID (int) recordID corresponding to channel group
recordIDsize (int) size of recordID
dataGroup (int:) data group number
channelGroup (int) channel group number
numpyDataRecordFormat (list) list of numpy (dtype) for each channel
dataRecordName (list) list of channel names used for recarray attribute definition
master (dict) define name and number of master channel
recordToChannelMatching (dict) helps to identify nested bits in byte
channelNames (list) channel names to be stored, useful for low memory consumption but slow

Methods

addChannel(info, channelNumber)  
loadInfo(info)  
readSortedRecord(fid, pointer, channelList=None)  
readUnsortedRecord(buf, channelList=None)  
addChannel(info, channelNumber)

add a channel in class

Parameters:

info : mdfinfo3.info3 class

channelNumber : int

channel number in mdfinfo3.info3 class
loadInfo(info)

gathers records related from info class

Parameters:info : mdfinfo3.info3 class
readSortedRecord(fid, pointer, channelList=None)

reads record, only one channel group per datagroup Parameters —————- fid : float

file identifier
pointer
position in file of data block beginning
channelList : list of str, optional
list of channel to read
Returns:

rec : numpy recarray

contains a matrix of raw data in a recarray (attributes corresponding to channel name)

Notes

If channelList is None, read data using numpy.core.records.fromfile that is rather quick. However, in case of large file, you can use channelList to load only interesting channels or only one channel on demand, but be aware it might be much slower.

readUnsortedRecord(buf, channelList=None)

Not implemented yet, no reference files available to test it

class mdf3reader.recordChannel(info, dataGroup, channelGroup, channelNumber, recordIDsize)

recordChannel class gathers all about channel structure in a record

Attributes

name (str) Name of channel
channelNumber (int) channel number corresponding to mdfinfo3.info3 class
signalDataType (int) signal type according to specification
bitCount (int) number of bits used to store channel record
nBytes (int) number of bytes (1 byte = 8 bits) taken by channel record
dataFormat (str) numpy dtype as string
CFormat (struct class instance) struct instance to convert from C Format
byteOffset (int) position of channel record in complete record in bytes
bitOffset (int) bit position of channel value inside byte in case of channel having bit count below 8
RecordFormat (list of str) dtype format used for numpy.core.records functions ((name,name_title),str_stype)
channelType (int) channel type
posBeg (int) start position in number of bit of channel record in complete record
posEnd (int) end position in number of bit of channel record in complete record

Methods

__init__(info, dataGroup, channelGroup, channelNumber, recordIDsize) constructor
__str__() to print class attributes
mdf3reader.tabConv(data, conv)

apply Tabular conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
mdf3reader.tabInterpConv(data, conv)

apply Tabular interpolation conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value
mdf3reader.textRangeTableConv(data, conv)

apply text range table conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns:

converted data to physical value

Table Of Contents

Previous topic

mdfreader module documentation

Next topic

mdfinfo3 module documentation

This Page

Navigation

© Copyright 2015, Aymeric Rateau. Created using Sphinx 1.2.3.