Internal Code Reference¶

The code detailed here is not for users of the module, but as an aid to development of the module itself.

For regular usage, see Section API Reference .

biorad1sc_reader.parsing¶

Low-level routines to parse a Bio-Rad 1sc file. Intended to be used internally only.

biorad1sc_reader.parsing.fix_wordsize_zero(field_payload_regions, byte_offsets, data_key_total_bytes)¶

Fix Datakey data when Field Type 100 doesn’t list word_size

In certain 1sc files, the word_size sub_field of Field Type 100 can be 0. This function detects regions in field_payload_regions dict for word_size==0 and changes word_size to the appropriate number of bytes based on data_type codes.

field_payload_regions is modified in place.

Parameters:	field_payload_regions (dict) – dict containing region info byte_offsets (list) – all starting byte offsets for all regions data_key_total_bytes (int) – total number of bytes in data container that field_payload_regions is defining

biorad1sc_reader.parsing.is_ascii(byte_stream)¶

Determine if all bytes in a bytes object are “good” ASCII

If all bytes are normal ascii text with no control characters other than NULL, LF, CR, TAB, then return True, else False.

Parameters:	byte_stream (bytes) – arbitrary length
Returns:	True if all bytes are printable ASCII codes or one of the following ASCII codes: 0 (null), 9 (Tab), 10 (LF), 13 (CR); False otherwise.
Return type:	bool

biorad1sc_reader.parsing.process_data_region(region, payload, field_ids, field_types, visited_ids)¶

Process one region of one data container field.

Parameters:

region (dict) – info from datakey about the format of this region
payload (bytes) – bytes of the payload just for this region
field_ids (dict) – keys are Field IDs, items are dicts containing all data for that Field instance
field_types (dict) – explanation of each Field Type from ‘items’ returned from process_payload_type101
visited_ids (list) – uint32 Field IDs of fields that have been visited

Returns:

comprised of the following structure:

region = {
    'raw': <bytes raw data from payload>
    'proc': <various unpacked/decoded numbers/strings from raw data>
    'interp': <various interpreted version of proc data, or None
                if no interpretation possible. can also be list
                of another field's regions if region data is a
                reference to another field>
}

Return type:

dict

biorad1sc_reader.parsing.process_payload_data_container(field_info, field_types, field_ids, visited_ids)¶

Process the payload of a 1sc data container field.

Process the payload of a 1sc Field Type > 102, (a data container field,) returning the relevant data to a dict.

Parameters:

field_info (dict) – contains info about current field
field_types (dict) – explanation of each Field Type from ‘items’ returned from process_payload_type101
field_ids (dict) – keys are Field IDs, items are dicts containing all data for that Field instance
visited_ids (list) – keeps track of all Field IDs that have been processed into the hierarchical output data

Returns:

regions, where each item of list is a dict of the form:

region = {
    'raw': <bytes raw data from payload>
    'proc': <various unpacked/decoded numbers/strings from raw data>
    'interp': <various interpreted version of proc data, or None
                if no interpretation possible. can also be list
                of another field's regions if region data is a
                reference to another field>
}

Return type:

list

biorad1sc_reader.parsing.process_payload_type100(field_payload, data_key_total_bytes, field_ids=None)¶

Process the payload of a 1sc Field Type 100

Process the payload of a 1sc Field Type 100, a description of the format of a particular Field Type of data container field. Return dict of dict containing region info.

Parameters:

field_payload (bytes) – all the contents of a Field Type 100 after the header bytes
field_ids (dict) – keys of Field IDs (uint32 numbers) and items which are dicts containing all information on that field instance

Returns:

info dict, with the form:

info = {
    'regions':<dict regions>
}

where dict regions is in the form of:

regions = {
    <number1>:<dict region1>,
    <number2>:<dict region2>,
    ...
}

where each key <number> is a number from 0 to Total Regions - 1 where each dict <region> is in the form of:

region = {
    'data_type': <uint16 number coding for data type of region>,
    'label': <str name of region>,
    'index': <int index that orders data regions>,
    'num_words': <int number of words in region>,
    'byte_offset': <int byte offset from start of Data Container payload>,
    'word_size':<int number of bytes in each word>,
    'ref_field_type':<uint16 Field Type of ref. if data_type is ref.>,
}

Return type:

dict

biorad1sc_reader.parsing.process_payload_type101(field_payload, field_ids=None)¶

Process the payload of a 1sc Field Type 101

Process the payload of a 1sc Field Type 101, a summary of every type of Data Container type available in this Data Collection, and return information dict.

Parameters:

field_payload (bytes) – all the contents of a Field Type 101 after the header bytes
field_ids (dict) – keys of Field IDs (uint32 numbers) and items which are dicts containing all information on that field instance

Returns:

dict containing a summary all data_container item types possible for the associated Collection in the form:

collection_item_definitions {
    'items':<int tot_items>,
    <Field1 Type>:<dict item_info1>,
    <Field2 Type>:<dict item_info2>,
    ...
}

where each key <Field Type> is the uint16 Field Type of a Data Container field that is possibly found after this definition in the next Data Block. Each <item_info> gives a summary of how to process a possible future data container field:

item_info {
    'num_regions': <int number of regions>,
    'data_key_ref': <uint32 Field ID of a Field Type 100>,
    'total_bytes': <int total bytes in region>,
    'label': <str name of item>,
}

Return type:

dict

biorad1sc_reader.parsing.process_payload_type102(field_payload, field_ids=None)¶

Process the payload of a 1sc Field Type 102

Process the payload of a 1sc Field Type 102, a Collection definition, and return and informational dict.

Parameters:

field_payload (bytes) – all the contents of a Field Type 102 after the header bytes
field_ids (dict) – keys of Field IDs (uint32 numbers) and items which are dicts containing all information on that field instance

Returns:

collection_info with the form:

collection_info {
    'collection_num_items': <int num of items in collection>,
    'collection_label': <str name of collection>,
    'collection_ref': <uint32 Field ID of a Field Type 101>
}

Return type:

dict

biorad1sc_reader.parsing.unpack_double(byte_stream, endian='<')¶

Return list of doubles, (either endian) from bytestring.

Unpack a bytes object into list of double-precision floating-point numbers. Each 8 input bytes decodes to a double.

Parameters:	byte_stream (bytes) – length is a multiple of 8 endian (char, optional) – “<” means little-endian unpacking, and “>” means big-endian unpacking
Returns:	unpacked double numbers
Return type:	list

biorad1sc_reader.parsing.unpack_string(byte_stream)¶

Return decoded ASCII string from bytestring.

Decode a bytes object via UTF-8 into a string

Parameters:	byte_stream (bytes) – arbitrary length
Returns:	UTF-8 decoded string
Return type:	string

biorad1sc_reader.parsing.unpack_uint16(byte_stream, endian='<')¶

Return list of uint16s, (either endian) from bytes object.

Unpack a bytes object into list of 16-bit unsigned integers. Each 2 input bytes decodes to a uint16.

Parameters:	byte_stream (bytes) – length is a multiple of 2 endian (char, optional) – “<” means little-endian unpacking, and “>” means big-endian unpacking
Returns:	unpacked uint16 numbers
Return type:	list

biorad1sc_reader.parsing.unpack_uint32(byte_stream, endian='<')¶

Return list of uint32s, (either endian) from bytes object.

Unpack a bytes object into list of 32-bit unsigned integers. Each 4 input bytes decodes to a uint32.

Parameters:	byte_stream (bytes) – length is a multiple of 4 endian (char, optional) – “<” means little-endian unpacking, and “>” means big-endian unpacking
Returns:	unpacked uint32 numbers
Return type:	list

biorad1sc_reader.parsing.unpack_uint64(byte_stream, endian='<')¶

Return list of uint64s, (either endian) from bytes object.

Unpack a bytes object into list of 64-bit unsigned integers. Each 8 input bytes decodes to a uint64.

Parameters:	byte_stream (bytes) – length is a multiple of 8 endian (char, optional) – “<” means little-endian unpacking, and “>” means big-endian unpacking
Returns:	unpacked uint64 numbers
Return type:	list

biorad1sc_reader.reader¶

Main reader module for Bio-Rad 1sc files. Includes public API class Reader.

class biorad1sc_reader.reader.Reader(in_file=None)¶

Object to manage reading a Bio-Rad 1sc file and extracting data from it, including image.

Assumes the 1sc file does not change while this instance has it open.

Instantiation:

Args:

in_file (str or file-like obj): filepath (str) or file-like: object, 1sc file to read with this instance

Raises:

BioRadInvalidFileError if file is not a valid Bio-Rad 1sc file

__init__(in_file=None)¶

Initialize Reader class

Parameters:	in_file (str or file-like obj) – filepath (str) or file-like object, 1sc file to read with this instance
Raises:	BioRadInvalidFileError if file is not a valid Bio-Rad 1sc file

__weakref__¶: list of weak references to the object (if defined)

_first_region(item, region_name)¶

Fetch data from first region in item with name region_name

Convenience function to fetch data for the first region with label region_name in item

Parameters:	item (list) – list of regions from get_metadata() or get_metadata_compac() region_name (str) – region label to search for
Returns:	whatever is contained in ‘data’ key of region dict
Return type:	various

_get_img_size()¶

Get img_size x and y, load into instance

Determine image size from metadata of 1sc file and set internal instance attributes self.img_size_x and self.img_size_y

_get_next_data_block_end(byte_idx)¶

Given a byte index, find the next Data Block end, return byte at start of the following Data Block

Parameters:	byte_idx (int) – file byte offset to search for the end of the next Data Block
Returns:	(block_num, end_idx) where block_num is the Data Block that ends at end_idx-1
Return type:	tuple

_make_compact_item(item)¶

Given an Item from metadata data collection, return a compact version of it for use in get_metadata_compact()

Remove everything except ‘label’ and most-interpreted form of ‘data’ available

Parameters:	item (dict) – item_dict from get_metadata(), e.g. collections[<num>][‘data’][<num>]
Returns:	compact representation of input dict for get_metadata_compact()
Return type:	dict

_parse_file_header()¶

Read and process the start of the file (header)

Raises:	BioRadInvalidFileError if file is not a valid Bio-Rad 1sc file

_process_field_header(byte_idx)¶

Parameters:	byte_idx (int) – file byte offset, start of the field to read header
Returns:	(field_type, field_len, field_id) where field_type is uint16 Field Type, field_len is int length in bytes of field, field_id is uint32 Field ID
Return type:	tuple

_read_field_lite(byte_idx)¶

Parameters:	byte_idx (int) – file byte offset, start of the field to read
Returns:	(file_byte_offset_next_field, field_info) where field info is a dict: { 'type':<uint16 Field Type> 'id':<uint32 Field ID> 'start':<byte offset of start of field> 'len':<total length in bytes of field> 'payload':<field payload bytes> }
Return type:	tuple

get_img_data(invert=False)¶

Return image_x_size, image_y_size, and list containing image data. Also ability to invert brightness.

Parameters:	invert (bool, optional) – True to invert the brightness scale of output image data compared to 1sc image data (black <-> white)
Returns:	(xsize, ysize, image_data) where xsize and ysize are integers specifying the size of the image. image_data is a list of uint16 numbers comprising the image data starting from upper-left and progressing to lower-right.
Return type:	tuple

get_img_summary()¶

NOTE: Safer to use get_metadata() or get_metadata_compact()

Read from Data Block 7, containing strings describing image.

Returns:

dict containing data from strings in Data Block 7:

{
    'Scanner Name':'ChemiDoc XRS'
    'Number of Pixels':'(<x pix size> x <y pix size>)'
    'Image Area':'(<x float size> mm x <y float size> mm)'
    'Scan Memory Size': '<size in bytes>'
    'Old file name': '<orig file name>'
    'New file name': '<new file name>'
    'path':'CHEMIDOC\Chemi'
    'New Image Acquired':'New Image Acquired'
    'Save As...':'Save As...'
    'Quantity One':'Quantity One <version> build <build number>'
}

Return type: dict

get_metadata()¶

Fetch All Metadata in File, return hierarchical dict/list

Returns:

collections where each item in list collections is a dict:

collection_dict = {
    'data':<list items>
    'label':'<str name of collection>'
}

where items is a list of dicts, each with the structure:

item_dict = {
    'data':<list regions>
    'id':<uint32 Field ID>
    'label':'<str name of item>'
    'type':'<int Field Type>'
}

where regions is a list of dicts, each with the structure:

region_dict = {
    'data': <dict data_of_region>
    'dtype': <str written type of data>
    'dtype_num': <int data type code of data>
    'key_iter': <??>
    'label': <str name of region>
    'num_words': <int number of words in data>
    'region_idx': <int 1sc-given index>
    'word_size': <int number of bytes per word of data>
}

where data_of_region has the structure:

data_of_region = {
    'raw': <bytes raw bytes, unconverted data>
    'proc': <various unpacked/decoded data from bytes>
    'interp': <various 'interpreted' data>
}

data_of_region[‘interp’] can also be another item_dict, if this: region contained a reference to another field, creating a hierarchical structure.

e.g. collections[0]['data'][0]['data'][0]['label'] = 'array'

Return type: list

Raises: BioRadParsingError – if there was an error in parsing the file

get_metadata_compact()¶

Fetch All Metadata in File, return compact version of hierarchical dict/list

Convert dict(list()) of Collections, Items to dict(). Leave Regions as list, because they are not guaranteed to have unique labels.

Remove everything except ‘label’ and most-interpreted form of ‘data’ available.

Returns:

collections:

collections = {
    '<collection name1>':<dict collection_dict1>
    '<collection name2>':<dict collection_dict2>
    ...
}

where each collection_dict is:

collection_dict = {
    '<name of item1>':<list regions1>
    '<name of item2>':<list regions2>
    ...
}

where regions is a list of dicts, each with the structure:

region_dict = {
    'data': <various most interpreted version possible of data>
    'label': <str name of region>
}

region_dict[‘data’] can also be another regions list, if this: region contained a reference to another field, creating a hierarchical structure.

e.g. collections['Overlay Header']['OverlaySaveArray'][0]['label] = 'array'

Return type: dict

open_file(in_filename)¶

Open file and read into memory.

Raises Errors if File is not valid 1sc file.

Parameters:	in_filename (str) – filepath to 1sc file to read with object instance
Raises:	BioRadInvalidFileError if file is not a valid Bio-Rad 1sc file

read_stream(in_fh)¶

Read file-like object into memory.

Raises Errors if File is not valid 1sc file. Give it object returned by: open(<filename>, ‘rb’)

Parameters:	in_fh (byte stream) – filehandle to 1sc filedata to read with object instance. e.g. result from open(<filename>, ‘rb’)
Raises:	BioRadInvalidFileError if file is not a valid Bio-Rad 1sc file

refresh()¶: Reset and refresh all internal state using same input 1sc file.

reset()¶: Reset all internal state. (Must load file afterwards.)

save_img_as_tiff(tiff_filename, invert=False)¶

Save image data as TIFF image

Also ability to invert brightness

Parameters:	tiff_filename (str) – filepath for output TIFF file invert (bool, optional) – True to invert the brightness scale of output TIFF image compared to 1sc image data (black <-> white)

save_img_as_tiff_sc(tiff_filename, imgsc=1.0, invert=False)¶

Save image data as TIFF image, with brightness dynamic range expanded

Also ability to invert brightness

Parameters:

tiff_filename (str) – filepath for output TIFF file
imgsc (float, optional) –
Expand brightness scale. Value of 1.0 means that dynamic range of output TIFF will be maximum, with brightest pixel having value 65535 and darkest pixel having value 0.

imgsc > 1.0 will cause the brightness dynamic range to be expanded less than imgsc=1.0, and imgsc < 1.0 will cause the dynamic range to be expanded more than the imgsc=1.0 case.

For non-inverted images, the pixel with the minimum brightness will always be 0. For inverted images, the pixel with the maximum brightness will always be 65535.
invert (bool, optional) – True to invert the brightness scale of output TIFF image compared to 1sc image data (black <-> white)

biorad1sc_reader.reader.save_u16_to_tiff(u16in, size, tiff_filename)¶

Save 16-bit uints to TIFF image file

Since Pillow has poor support for 16-bit TIFF, we make our own save function to properly save a 16-bit TIFF.

Parameters:	u16in (list) – u16int image pixel data size (tuple) – (xsize, ysize) where xsize and ysize are integers specifying the size of the image in pixels tiff_filename (str) – filepath for the output TIFF file

Internal Code Reference¶

biorad1sc_reader.parsing¶

biorad1sc_reader.reader¶

biorad1sc_reader

Navigation

Related Topics