5.1. gridfix package

5.1.1. Main Classes (gridfix.model)

class gridfix.model.FixationModel(fixations, regionset, dv_type='fixated', features=None, feature_labels=None, chunks=[], progress=False, exclude_first_fix=False, normalize_features=False)

Bases: object

Combines Features and Fixations to create predictors and R source for GLMM

chunks

list – list of data columns that define chunks (e.g., subjects or sessions)

comp_features

dict – dict of labelled feature comparisons. Predictors will be replicated for each feature in a comparison so that features can serve as fixed or random factor

exclude_first_fix

boolean – if True, first fixation index was set NaN (e.g.,, fixation cross)

features

dict – dictionary of feature objects and feature groups

predictors

DataFrame – model predictors for GLMM

regionset

RegionSet – attached RegionSet

runtime

float – time in seconds for most recent update of predictor matrix

normalize_features

bool – if True, feature values are normalized to 0..1 range

add_comparison(features, codes=None, label=None)

Add a feature comparison group to the model.

This generates a long-style predictor matrix for the specified features, needed to compare e.g. different saliency maps in their relative effects.

Parameters:

• features (list) – list of Feature objects to combine into a group
• codes (list) – numeric codes to use in “dummy coding”, e.g. [0, 1, 2]
• label (str) – label and output column name for this feature group

add_feature(feature, label=None)

Add a feature to the model.

Parameters:

• feature (Feature) – Feature object to add
• label (str) – label and output column name for this feature

predictors

Model predictor matrix, updated if necessary

r_source(datafile='gridfix.csv', comments=True, scale=True, center=True, optimizer=None, fixed=None, random=None, random_slopes=False)

Generate R source code from current feature settings.

Parameters:

• datafile (str) – predictor matrix file name (for R import via read.table())
• comments (boolean) – if True, add explanatory comments and headers to source
• scale (boolean) – if True, add code to scale (normalize) feature predictors
• center (boolean) – if True, add code to center (demean) feature predictors
• optimizer (str) – optional optimizer to pass to R glmer()
• fixed (list) – list of column names (strings) to add as fixed factors
• random (list) – list of column names (strings) to add as random factors
• random_slopes (boolean) – also add random slopes to generated R code

Returns:

R source code as string

save(basename, sep='\t', pred=True, pred_pickle=False, src=True, src_comments=True, precision=10, optimizer=None, fixed=None, random=None, random_slopes=False)

Saves the predictor matrix to a CSV text file.

Parameters:

• basename (str) – base filename to save to, without extension
• sep (str) – item separator, default TAB
• pred (boolean) – if True, output predictor matrix as CSV
• pred_pickle (boolean) – if True, also save predictors to Pickle object
• src (boolean) – if True, output r source code file for lme4
• src_comments (boolean) – if True, add comments to source code
• precision (int) – number of decimal places for CSV (default: 10)
• optimizer (str) – optional optimizer to pass to R glmer()
• fixed (list) – list of column names (strings) to add as fixed factors
• random (list) – list of column names (strings) to add as random factors
• random_slopes (boolean) – also add random slopes to generated R code

update(progress=False)

Update predictor matrix from features (this may take a while).

Parameters:progress (boolean) – if True, print model creation progress

class gridfix.model.Fixations(data, imageset=None, offset=(0, 0), imageid='imageid', fixid='fixid', x='x', y='y', fixstart=None, fixend=None, numericid=False)

Bases: object

Dataset of fixation locations.

Fixation locations are assumed to be one-indexed in input, e.g. 1..800 if the image is 800 pixels wide, and converted internally to Pythons zero-indexed array convention.

data

DataFrame – DataFrame of raw fixation data

has_times

boolean – True if fixation times have been loaded

imageids

list – all unique image IDs represented in the dataset

imageset

ImageSet – if present, the associated ImageSet

input_file

str – file name of fixation data file

num_samples

int – number of fixation samples

num_vars

int – number of columns / variables in dataset

offset

tuple – optional offset from raw (x,y) positions

shape

tuple – dimensions of .data, i.e. same as Fixations.data.shape

variables

list – list of all variables loaded from input file

count_map(imageid=None, size=None)

Map of fixation counts for each image pixel

Parameters:

• imageid (str) – optional image ID to create map for one image only
• size (tuple) – image dimensions, specified as (width, height).

Returns:

2d ndarray of pixel fixation counts

dur_map(imageid=None, size=None)

Map of total fixation durations for each image pixel

Parameters:

• imageid (str) – optional image ID to create map for one image only
• size (tuple) – image dimensions, specified as (width, height).

Returns:

2d ndarray of fixation durations at each pixel

location_map(imageid=None, size=None)

Binary ndarray of fixated and non-fixated pixels within image area

Parameters:

• imageid (str) – optional image ID to create map for one image only
• size (tuple) – image dimensions, specified as (width, height).

Returns:

2d boolean ndarray, True where fixated, otherwise False

plot(imageid=None, select={}, on_image=True, oob=False, plotformat='wo', durations=False, image_only=False, ax=None)

Plot fixations for selected imageid, either alone or on image

Parameters:

• imageid (str) – optional image ID to plot fixations for
• select (dict) – dict of additional filter variables (see select_fix())
• image (bool) – if True, superimpose fixations onto image (if ImageSet present)
• oob (bool) – if True, include out-of-bounds fixations when plotting
• plotformat (str) – format string for plt.pyplot.plot(), default: white circles
• durations (bool) – if True, plot duration of each fixation next to marker
• image_only (boolean) – if True, return only image content without labels
• ax (Axes) – axes object to draw to, to include result in other figure

Returns:

matplotlib figure object, or None if passed an axis to draw on

select_fix(select={})

Return a subset of fixation data for specified imageid.

Args:.

select (dict): dict of filter variables, as {column: value}

Returns:New Fixations object containing selected fixations only

set_offset(offset)

Set a constant offset for eye x/y coordinates.

If image coordinates are relative to image center, use (-width/2, -height/2) (GridFix uses a coordinate origin at the top left).

Parameters:offset (tuple) – 2-tuple of (hor, ver) offset values in pixels

class gridfix.model.ImageSet(images, mat_var=None, size=None, imageids=None, sep='t', label=None, normalize=None, norm_range=None, preload=False)

Bases: object

Set of images of equal size for masking and Feature creation.

info

DataFrame – table of image metadata (filenames, size, type...)

imageids

list – All unique image IDs in the set

label

str – optional label to distinguish between ImageSets

mat_var

str – name of MATLAB variable name if imported from .mat

mat_contents

list – list of all variable names in .mat if applicable

normalize

boolean – True if image data was normalized to 0..1 range

preload

boolean – True if images were preloaded into memory

size

tuple – image dimensions, specified as (width, height)

image(imageid)

Get image by imageid, loading it if not preloaded

Parameters:imageid (str) – valid imageid from set Returns:ndarray of raw image data

plot(imageid, cmap='gray', image_only=False, ax=None)

Display one of the contained images by imageid using matplotlib

Parameters:

• imageid (str) – valid imageid of image to show
• cmap (str) – name of a matplotlib colormap to use
• image_only (boolean) – if True, return only image content without labels
• ax (Axes) – axes object to draw on, to include result in other figure

Returns:

matplotlib figure object, or None if passed an axis to draw on

5.1.2. Region Sets (gridfix.regionset)

class gridfix.regionset.GridRegionSet(size, gridsize=None, label=None, region_labels=None)

Bases: gridfix.regionset.RegionSet

RegionSet defining an n-by-m regular grid covering the full image size.

cells

list – list of bounding box tuples for each cell, each formatted as (left, top, right, bottom)

gridsize

tuple – grid dimensions as (width, height). If unspecified, gridfix will try to choose a sensible default.

label

string – optional label to distinguish between RegionSets

class gridfix.regionset.RegionSet(size, regions, region_labels=None, label=None)

Bases: object

Base class for sets of image regions of interest.

RegionSets can be used to spatially group fixations, create Feature objects for a FixationModel and split an image into parts. Classes inheriting from RegionSet may specify functions to create regions.

info

DataFrame – table of region metadata (labels, bboxes, number of pixels...)

imageids

list – list of all imageids associated with this RegionSet

is_global

bool – True if regions are global (non-image-specific)

label

str – optional label to distinguish between RegionSets.

memory_usage

float – memory usage of all binary masks (kiB)

size

tuple – image dimensions, specified as (width, height).

apply(image, imageid=None, crop=False)

Apply this RegionSet to a specified image.

Returns a list of the image arrays “cut out” by each region mask, with non-selected image areas in black. If regionset is not global, _imageid_ needs to be specified!

Parameters:

• image (ndarray) – image array to be segmented.
• imageid (str) – valid imageid (to select image-specific regions if not a global regionset)
• crop (bool) – if True, return image cropped to bounding box of selected area

Returns:

If crop=False, a list of ndarrays of same size as image, with non-selected areas zeroed. Else a list of image patch arrays cropped to bounding box size.

count_map(imageid=None)

Return the number of regions referencing each pixel.

Parameters:imageid (str) – if set, return map for specified image only Returns:2d ndarray of image size, counting number of regions for each pixel

coverage(imageid=None, normalize=False)

Calculates coverage of the total image size as a scalar.

Parameters:

• imageid (str) – if set, return coverage for specified image only
• normalize (bool) – if True, divide global result by number of imageids in set.

Returns:

Total coverage as a floating point number.

export_patches(image, imageid=None, crop=True, image_format='png', rescale=False)

Apply this RegionSet to an image array and save the resulting image patches as files.

Saves an image of each image part “cut out” by each region mask, cropped by default. If the RegionSet is not global, imageid needs to be specified!

Parameters:

• image (ndarray) – image array to be segmented.
• imageid (str) – imageid (to select image-specific regions if not a global regionset)
• crop (bool) – if True, return image cropped to bounding box of selected area
• image_format (str) – image format that PIL understands (will also be used for extension)
• rescale (bool) – if True, scale pixel values to full 0..255 range before saving (e.g., for saliency maps)

export_patches_from_set(imageset, crop=True, image_format='png', rescale=False)

Save all sliced image patches from an ImageSet as image files.

Saves an image of each image part “cut out” by each region mask, cropped by default. If the RegionSet is not global, only images with valid region masks will be processed.

Parameters:

• imageset (ImageSet) – a valid ImageSet containing images to slice
• imageid (str) – imageid (to select image-specific regions if not a global regionset)
• crop (bool) – if True, return image cropped to bounding box of selected area
• image_format (str) – image format that PIL understands (will also be used for extension)
• rescale (bool) – if True, scale pixel values to full 0..255 range before saving (e.g., for saliency maps)

fixated(fixations, imageid=None, count=False, exclude_first=False)

Returns visited / fixated regions using data from a Fixations object.

Parameters:

• fixations (Fixations/DataFrame) – fixation data to test against regions
• imageid (str) – imageid (to select image-specific regions if not a global regionset)
• count (bool) – if True, return number of fixations per region instead of boolean values
• exclude_first (bool) – if True, first fixated region will always be returned as NaN

Returns:

1D ndarray (float) containing number of fixations per region (if count=True) or the values 0.0 (region was not fixated) or 1.0 (region was fixated)

imageids

Return list of imageids for which region maps exist

is_global

Return True if a global map is defined (key ‘*’)

mask(imageid=None)

Return union mask of all regions or regions for specified image.

Parameters:imageid (str) – if set, return mask for specified image only Returns:2d ndarray of image size (bool), True where at least one region references the corresponding pixel.

memory_usage

Calculate size in memory of all regions combined

plot(imageid=None, values=None, cmap=None, image_only=False, ax=None, alpha=1.0)

Plot regions as map of shaded areas with/without corresponding feature values

Parameters:

• imageid (str) – if set, plot regions for specified image
• values (array-like) – one feature value per region
• cmap (str) – name of matplotlib colormap to use
• image_only (boolean) – if True, return only image content without axes
• ax (Axes) – axes object to draw to, to include result in other figure
• alpha (float) – opacity of plotted regions (set < 1 to visualize overlap)

Returns:

matplotlib figure object, or None if passed an axis to draw on

plot_regions_on_image(imageid=None, imageset=None, cmap=None, fill=False, alpha=0.4, labels=False, image_only=False, ax=None)

Plot region bounding boxes on corresponding image

Parameters:

• imageid (str) – if set, plot regions for specified image
• imageset (ImageSet) – ImageSet object containing background image/map
• cmap (str) – name of matplotlib colormap to use for boundin boxes
• fill (boolean) – draw shaded filled rectangles instead of boxes
• alpha (float) – rectangle opacity (only when fill=True)
• labels (boolean) – if True, draw text labels next to regions
• image_only (boolean) – if True, return only image content without axes
• ax (Axes) – axes object to draw to, to include result in other figure

Returns:

matplotlib figure object, or None if passed an axis to draw on

region_map(imageid=None)

Return map of region numbers, global or image-specifid.

Parameters:imageid (str) – if set, return map for specified image only Returns:2d ndarray (int), containing the number (ascending) of the last region referencing the corresponding pixel.

5.1.3. Image Features (gridfix.features)

class gridfix.features.CentralBiasFeature(regionset, imageset, measure='gaussian', sig2=0.22, nu=None, label=None)

Bases: gridfix.features.Feature

Models central viewer bias as the distance to image center for each region.

The exact model of distance depends on the “measure” argument: if set to ‘distance’ (the default), the image transformation step does nothing and euclidean distance is returned. If set to ‘gaussian’, anisotropic Gaussian distance based on Clarke & Tatler, 2014, Vis Res is used and transform() returns the corresponding Gaussian map.

apply(imageid=None, normalize=False)

Apply central bias to image, returning region distance values.

Parameters:

• imageid (str) – for consistency, ignored for central bias (same for all images)
• normalize (boolean) – if True, scale output to range 0...1 (default: False)

Returns:

1D numpy.ndarray of feature values, same length as regionset

class gridfix.features.Feature(regionset, imageset, trans_fun=None, comb_fun=None, label=None)

Bases: object

Base class for image features defined for each region in a set

A Feature can be described as a two-step “recipe” on a given ImageSet and RegionSet: First, a transform() is executed upon each image array, e.g. a filter operation. Second, a combine() operation yields a single value per region. The result is combined into a feature vector of len(regionset).

regionset

RegionSet – the associated regionset object

imageset

ImageSet – a set of images or feature maps to process

length

int – length of feature vector, i.e. number of regions

apply(imageid, normalize=False)

Apply feature to a single image from associated ImageSet.

Parameters:

• imageid (str) – valid ID from associated ImageSet
• normalize (bool) – if True, scale output to range 0...1 (default: False)

Returns:

1D numpy.ndarray of feature values, same length as regionset

apply_all(normalize=False)

Apply feature to every image in the ImageSet and return a DataFrame.

Parameters:normalize (bool) – if True, scale output to range 0...1 (default: False)

Returns: DataFrame similar to RegionSet.info with region feature values

combine(image, region, fun=<function mean>)

Combine all selected pixel values in selection using specified function.

In the base Feature class, this simply returns the mean of all pixels. This function may be overloaded in subclasses or replaced by the comb_fun= argument.

Parameters:

• image (np.ndarray) – 2D feature image
• region (np.ndarray) – binary mask array defining a region
• fun (function) – function to apply to selection. Must return a scalar.

Returns:

Scalar value depending on the specified function.

plot(imageid, what='both', cmap='gray', image_only=False)

Display feature map and/or feature values.

Parameters:

• imageid (str) – valid ID from associated ImageSet
• what (str) – show only feature ‘values’, ‘map’ or ‘both’
• cmap (str) – name of matplotlib colormap to use
• image_only (boolean) – if True, return only image content without labels

Returns:

matplotlib figure object, or None if passed an axis to draw on

transform(image)

Apply image transform to specified image.

In the base Feature class, this simply returns the input image as-is. This function may be overloaded in subclasses or replaced by the trans_fun= argument on construction time.

Parameters:image (ndarray) – image / feature map array to transform Returns:np.ndarray of image data

class gridfix.features.LumContrastFeature(regionset, imageset, label=None)

Bases: gridfix.features.Feature

Feature based on local luminance contrast in each region

class gridfix.features.LuminanceFeature(regionset, imageset, label=None)

Bases: gridfix.features.Feature

Models mean image luminance in each region

class gridfix.features.MapFeature(regionset, imageset, stat=<function mean>, label=None)

Bases: gridfix.features.Feature

Feature to apply a statistical function to each region in feature maps

stat

function – the statistics function to apply to each region

class gridfix.features.SobelEdgeFeature(regionset, imageset, label=None)

Bases: gridfix.features.Feature

Feature based on relative prevalence of edges within each region