Welcome to FoxMask’s documentation¶

https://travis-ci.org/edevost/foxmask.svg?branch=master

You will find on theses pages information related to the installation and usage of FoxMask. It is intended to be read by users as well as developers.

Source code on GitHub

Getting Started¶

Installation¶

We provide two methods for installing and using FoxMask:

Standalone installation on Linux
Installation through a virtual machine (Any platform)

Standalone installation on Linux¶

This is the recommended method for production.

This method will install FoxMask on your Linux computer. We provide an automated install script, which will take care of installing all the dependencies needed to run FoxMask as well as the software itself. It is made for Ubuntu 16.04. The file is located in install-scripts/install-ubuntu16.sh of the FoxMask repository. You can also download the file directly sh an run it. To execute the script, simply run the following command as a standard user. Note that you will need to have sudo rights.

$ sh install-Ubuntu16.sh

Note

You can issue the install command from any directory, the script will download all the needed software in your home directory. Please do not run the script prefixed by sudo.

Once the installation if finished, you will have a foxmask directory in your home folder where you will find the software and a set of images to test the software. Congratulation, you are now ready to start using FoxMask on your Linux Box ! Consult our Usage page to get started.

This installation have been successfully tested on Ubuntu 16.04, on bare metal computers as well as on GCE (Google Cloud Engine) and Virtual-Box 5.0.

Installation through a virtual machine¶

We do not recommend this installation for production.

We provide a complete virtual environment to run and use FoxMask. The main advantages of this installation is that you can rapidly get FoxMask running on any platform. However, there are important drawbacks to consider:

Poor performances compared to standalone installation
A bit heavy to manage (Vagrant + Virtual-Box + share USB)

Despite theses drawbacks, we consider that using our FoxMask box can be very useful for first time users wanting to test the software, or for development purposes.

Note

Again, let me emphasize that this method is suitable to rapidly get FoxMask up and running on another platform than Ubuntu. However, anyone serious about implementing an automated image analysis pipeline with FoxMask should consider installing it on bare metal hardware running a Linux OS, or on a virtual infrastructure running libvirt. We provide a fully automated installation script, successfully tested on Ubuntu 16.04. Please see Standalone installation on Linux to install FoxMask on you Linux machine.

To run the FoxMask virtual machine on your Linux, Mac or Windows computer, you will need the following three open source software:

Follow the installation procedure for your platform. Once theses software are installed, you are ready to install FoxMask by issuing the following command:

git clone https://github.com/edevost/foxmask.git

This will fetch all needed components to run the virtual machine. Once the clone command is completed, you can start the virtual machine by issuing the following commands:

cd foxmask
vagrant up

This will boot the virtual machine. You can also manage you virtual machine through Virtual-Box software, and start or stop your machine from there. The start process will bring you to the Ubuntu desktop. The default user and password on this virtual environment is vagrant vagrant. Once the machine is ready, you can proceed to our Usage section to launch FoxMask on the provided set of images.

Testing on your own set of images¶

To test FoxMask on your own set of images, you will have to make your images available to the virtual machine. To do so, we recommend saving your image set on an USB drive, and make the drive available to Virtual-Box. There are many good tutorials on the web explaining the process:

Making this work will demand a bit of work and reading, but nothing out of reach of any computer users with minimal computer knowledge. The main steps to get your FoxMask virtual machine to have access to an USB drive plugged on your host computer are the following:

Shutdown your FoxMask virtual machine if it is running
Plug your USB drive on your computer (host)
Install Virtual Box 5.0 extension pack
Activate USB controller on Virtual-Box
Add your drive

Once your drive is added, boot your FoxMask virtual machine and check if there is a link to your drive on the desktop. If everything went well, you will have direct access to your USB drive on your virtual machine ! You can then proceed to our Usage section to start using FoxMask on your set of images.

Usage¶

FoxMask program is shipped with a sample of images located in the images directory, where you will find two other directories (example-1 and example-2) containing JPEG images of Arctic fox. We encourage you to test FoxMask on theses samples first. The full analysis takes around 2 minutes on an Intel core i5.

To run FoxMask, simply invoke it from the command line.

$ foxmask
Usage: foxmask [OPTIONS] SRCDIR RESULTSDIR

Error: Missing argument "srcdir".

Without argument, the software will exit and print it’s usage and what argument is missing. The first argument [SRCDIR] is the directory where all your directories containing images are. The second argument [RESULTSDIR] is the directory where you want the results to be created. For example:

$ foxmask images .

Will launch the software and run the analysis for all directories present in the images directory and create FoxMaskResults in your current directory (.). If you are launching the application in the foxmask directory, the command will launch the analysis process on the default set of images distributed with FoxMask and write it’s results in FoxMaskResults.

As the code run, you will see images being analyzed and mask created. At the end of the process, a CSV file is exported with the image names and the presence of detected objects (0 or 1). If required, masks and images with detected objects are also copied in sub directories.

Running the code on your own set¶

To run the code on your own set of images, simply specify where the directory containing the images are located. For example:

foxmask /media/reconyximages/2015  analysis/results

This will launch the analysis on all directories present in /media/reconyximages/2015. There is no limits in the number of directory’s that can be analyzed. Just keep in mind that the more directories, the more the analysis will take time. We suggest testing the software on just one directory to see how it behaves on your images, and learn how FoxMask is functioning. The results will be written in the analysis/results directory.

Note

You do not need to be in the foxmask directory to run FoxMask. Just make sure to specify to path to your images directory relative to where you are launching the foxmask command, or specify an absolute path as in the example above. The same applies for where you want the results to be written.

Constraints¶

The are a few minor constraints concerning the directory structure containing the images to be analyzed and the format of the images.

Directories naming and structure: All directories to be analyzed should have a descriptive name, as FoxMask will store the results using the names of these directories. Also, every directory should contains images and no sub directory.
Images naming and format: Your images should be in the JPEG format, with the proper extension (e.g. image_0001.jpg, or image_0002.JPG). The numbering is crucial: it is mandatory to use a 4 digit number prefixed by an underscore. The case does not matter, as well as the name of the image. A future version of foxmask will be able to analyze other image format. See issue number 45

Note

We believe that it is the user’s responsibility to format their images in an uniform way before feeding them to an analysis software.

Configuration¶

There are a few values that can be parametrized in the parameters.py file. However, the defaults should be sane enough for allowing testing the software on your own set of images.

maxgap: The maximum time gap (in seconds) between two consecutive images to be considered as part of the same sequence. This parameter is set by default to 5 seconds. Under conditions where the background is dynamic (e.g. movements in static objects occurs), the maxgap parameter should not be raised very high. However, under conditions where the background is very static, higher values might be best.
minsize: Minimum size of moving objects to be considered as an animal. This value is set by default to 500 pixels. You will need to test this parameters on your images to determine the best value. Keep in mind that with lower values, the risk of getting false positive is higher and with higher value, the risk of getting false negative is higher.

Image analysis pipeline¶

Setting up an image analysis pipeline should be done by using code under version control (e.g. git). It should be 100% automated and documented, even for small jobs. It’s the only way to use efficiently any analysis software.

The FoxMask team will implement such a pipeline in the near future, and will make it freely available as a real word example of how to implement FoxMask in a fully automated image analysis pipeline.

Contribution¶

Contributions to the FoxMask project is highly welcome and encouraged ! You can contribute in many ways. Note that you need a GitHub account to contribute to the project.

Report issues¶

You can report any issues with the project in the source code repository on GitHub. These issues can be of any kind, wrong documentation, issues with installation or usage of FoxMaks. Please write as much information as possible when submitting your issue.

Contribute to the documentation¶

You can visit the documentation pages directly on GitHub and edit the documentation page you wish modify. You can then submit your modification.

Contribute to the code¶

You can easily contribute to the code by cloning the repository, checkout to a new branch, work and your branch, and push your branch to FoxMask repository. If you get a permission denied on push, please contact a FoxMask developer to grant you access. Example workflow:

git clone https://github.com/edevost/foxmask.git
git checkout -b my-new-feature
[work on your feature]
git add *your modified files*
git commit -m 'meaningfull message concerning your modification'
git push origin my-new-feature

You can then submit a pull request on GiHub. Thank you for contributing !

FoxMask module¶

Evaluate if an image contains moving objects (animals).

The main routine of the code will iterate through every folders present in the images directory and analyze found images, looking for moving objects.

The following module level attributes are passed via the command line interface.

foxmask.srcdir¶: str – The directory containing folders (one or many) with images to analyze.

foxmask.resultsdir¶: str – The location where to create the FoxMaskResults directory.

class foxmask.Getimagesinfos(folder)[source]¶

Build data structure of images to be analyzed.

getimageslist()[source]¶

Generate a list of all images under folder

Returns:	imglist. All images under the folder being analyzed.
Return type:	list

getimagesmeta()[source]¶

Generate a list of all images time of creation.

The time of creation is taken from the DateTimeOriginal key present in every image metadata. If the method can not read the DateTimeOriginal key, it will look in the Comment key trying to find the time of creation. Method will gracefully exit if not time of creation can be found.

Example of format returned:

>>> datetime.datetime(2014, 8, 6, 16, 5, 55)

Returns:	timeofcreation. Datetime objects representing the exact time of creation of each image.
Return type:	list

getimpg()[source]¶

Group images based on their time of creation

This method will group images based on the differences in time between each shot.

Parameters:	maxgap (int) – This represent the maximum gap, , in seconds for two consecutive images to be considered as being part of the same group.
Returns:	impg. Number of images in each group.
Return type:	list

sortimages()[source]¶

Sort images based on their time of creation.

This method makes sure images are appropriately sorted. We do not want to rely on images file name to sort them.

Parameters:	timeofcreation (date) – A date object representing the time of creation of each image.
Returns:	sortedimglist. Sorted time of creation of all images to analyze.
Return type:	list

class foxmask.Imagesanalysis(folder)[source]¶

Analyze images to detect moving objects.

Parent:: Getimagesinfos

bgfgestimation(sortedimglist, impg, srcdir)[source]¶

Estimate background model and perform foreground segmentation.

The method will iterate over each item of the impg list, and performs the analysis on each groups. The trcd value, which is influencing significantly the outcome of the analysis, is pass to the ForegroundSegmentation code. To pass this value to the cpp code, it is written to a file /tmp/params.txt which is then read by the cpp code at runtime.

Images to analyze are resized (for performances issues on non server grade hardware) and then saved to a temporary directory that is deleted after each run. The results of this method are black and white masks that are written to disks in srcdir/MasksResults.

Note

This code will only run successfully if the input images are named as the following: whatever-name_[4 digits].jpg. It is a requirement of the ForegroundSegmentation code. We do not think that FoxMask should handle the naming of the images to be analyzed. This should be done beforehand, by a prepossessing task.

trcd : Threshold for cosine distance. This value is fed to the ForegroundSegmentation code. It is a static value, but it could be made more dynamic using, for example, the average light in the image, as shown in the code below:

if avgB[0] < 100.0:
    print 'Low light', avgB
    tfcd.write(str(0.001))
else:
    print 'High light', avgB
    tfcd.write(str(0.005))

Parameters:	sortedimglist (list) – The images to be analyzed impg (list) – Groups of images on which to run the analysis. scrdir (string) – The top level directory of the analysis.

getmaskslist(srcdir)[source]¶

Create a sorted list of generated masks.

This method will create a list of all the images present in srcdir/MasksResults. Theses images are the black and white masks created by the foxmask.Imagesanalysis.bgfgestimation() method. Before creating the list, masks with the prefix EstBG are removed.

Parameters:	srcdir (str) – Top level directory containing directories of images.
Returns:	maskslist. All masks to analyze.
Return type:	list

masks_analysis()[source]¶

Analyze masks to detect moving objects

This method will analyze created masks, taking the masks list to analyze from foxmask.Imagesanalysis.getmaskslist(). The area of all white objects in each masks are calculated. If the area is smaller than parameters.minsize, the object is not considered as an animal.

Returns:	resultslist. Results of the masks analysis.
Return type:	list

writeresults(item, resultsdir)[source]¶

Write results of the mask analysis to file

Parameters:	item (str) – The name of the class analyzed, representing the name of the folder analyzed. This is used to name the table in which the results will be written. resultsdir (str) – The top directory where to write the final results.

foxmask.getfolders(srcdir)[source]¶

Get the list of all folders in srcdir

FoxMask needs a list of folders containing images to analyze. Each folder must strictly contain a set of images. No sub folders are allowed. Code will gracefully exit if the srcdir argument does not exist.

Parameters:	srcdir (str) – Top level folder containing all folders to analyze.
Returns:	folderslist. Folders to analyze.
Return type:	list

foxmask.makeresultsfolder(resultsdir)[source]¶

Make all needed folders to store the final results.

Parameters:	resultsdir (str) – Top level folder for storing results.
Returns:	None

Gathering folders to analyze¶

foxmask.getfolders(srcdir)[source]

Get the list of all folders in srcdir

FoxMask needs a list of folders containing images to analyze. Each folder must strictly contain a set of images. No sub folders are allowed. Code will gracefully exit if the srcdir argument does not exist.

Parameters:	srcdir (str) – Top level folder containing all folders to analyze.
Returns:	folderslist. Folders to analyze.
Return type:	list

Creating results folders¶

foxmask.makeresultsfolder(resultsdir)[source]

Make all needed folders to store the final results.

Parameters:	resultsdir (str) – Top level folder for storing results.
Returns:	None

Getting images information¶

FoxMask will create a dictionary of classes containing all methods and attributes needed to analyze images in each found directory. Thus, the foxmask.Getimagesinfo class will be stored in a dictionary, one class for each directory being analyzed. This class is the parent of foxmask.Imagesanalysis

class foxmask.Getimagesinfos(folder)[source]

Build data structure of images to be analyzed.

__init__(folder)[source]¶

Initialize attributes used across methods.

Parameters:	folder (str) – Actual folder containing images to analyze. imglist (list) – Images to analyze. timeofcreation (list) – Images time of creation.

getimageslist()[source]

Generate a list of all images under folder

Returns:	imglist. All images under the folder being analyzed.
Return type:	list

getimagesmeta()[source]

Generate a list of all images time of creation.

The time of creation is taken from the DateTimeOriginal key present in every image metadata. If the method can not read the DateTimeOriginal key, it will look in the Comment key trying to find the time of creation. Method will gracefully exit if not time of creation can be found.

Example of format returned:

>>> datetime.datetime(2014, 8, 6, 16, 5, 55)

Returns:	timeofcreation. Datetime objects representing the exact time of creation of each image.
Return type:	list

getimpg()[source]

Group images based on their time of creation

This method will group images based on the differences in time between each shot.

Parameters:	maxgap (int) – This represent the maximum gap, , in seconds for two consecutive images to be considered as being part of the same group.
Returns:	impg. Number of images in each group.
Return type:	list

sortimages()[source]

Sort images based on their time of creation.

This method makes sure images are appropriately sorted. We do not want to rely on images file name to sort them.

Parameters:	timeofcreation (date) – A date object representing the time of creation of each image.
Returns:	sortedimglist. Sorted time of creation of all images to analyze.
Return type:	list

Analyzing images¶

The following class constitue the core code of the image analysis. It uses two external cpp libraries, coded by Vikkas Reddy:

The two libraries have been modified to be intragrated with FoxMask and to be able to run successful analysis on a small set of images (e.g. 5). The EstimateBackground code was performing well on a small set, but we could not get the ForegroundSegmentation code, which does it’s own background estimation, to perform well on small sets. So we used the EstimateBackround code to feed the ForegroundSegmentation in it’s background estimation. We accomplish that simply by duplicating the detected backround and configured the ForegroundSegmentation code to use theses images as training images.

class foxmask.Imagesanalysis(folder)[source]

Analyze images to detect moving objects.

Parent:: Getimagesinfos

bgfgestimation(sortedimglist, impg, srcdir)[source]

Estimate background model and perform foreground segmentation.

The method will iterate over each item of the impg list, and performs the analysis on each groups. The trcd value, which is influencing significantly the outcome of the analysis, is pass to the ForegroundSegmentation code. To pass this value to the cpp code, it is written to a file /tmp/params.txt which is then read by the cpp code at runtime.

Images to analyze are resized (for performances issues on non server grade hardware) and then saved to a temporary directory that is deleted after each run. The results of this method are black and white masks that are written to disks in srcdir/MasksResults.

Note

This code will only run successfully if the input images are named as the following: whatever-name_[4 digits].jpg. It is a requirement of the ForegroundSegmentation code. We do not think that FoxMask should handle the naming of the images to be analyzed. This should be done beforehand, by a prepossessing task.

trcd : Threshold for cosine distance. This value is fed to the ForegroundSegmentation code. It is a static value, but it could be made more dynamic using, for example, the average light in the image, as shown in the code below:

if avgB[0] < 100.0:
    print 'Low light', avgB
    tfcd.write(str(0.001))
else:
    print 'High light', avgB
    tfcd.write(str(0.005))

Parameters:	sortedimglist (list) – The images to be analyzed impg (list) – Groups of images on which to run the analysis. scrdir (string) – The top level directory of the analysis.

getmaskslist(srcdir)[source]

Create a sorted list of generated masks.

This method will create a list of all the images present in srcdir/MasksResults. Theses images are the black and white masks created by the foxmask.Imagesanalysis.bgfgestimation() method. Before creating the list, masks with the prefix EstBG are removed.

Parameters:	srcdir (str) – Top level directory containing directories of images.
Returns:	maskslist. All masks to analyze.
Return type:	list

masks_analysis()[source]

Analyze masks to detect moving objects

This method will analyze created masks, taking the masks list to analyze from foxmask.Imagesanalysis.getmaskslist(). The area of all white objects in each masks are calculated. If the area is smaller than parameters.minsize, the object is not considered as an animal.

Returns:	resultslist. Results of the masks analysis.
Return type:	list

writeresults(item, resultsdir)[source]

Write results of the mask analysis to file

Parameters:	item (str) – The name of the class analyzed, representing the name of the folder analyzed. This is used to name the table in which the results will be written. resultsdir (str) – The top directory where to write the final results.

Cli module¶

This module is the entry point of the FoxMask package. When foxmask is invoke from the command line, this module is executed. The entry point is specified in the setup.py file, as follows:

        entry_points='''
        [console_scripts]
        foxmask=cli.cli:main

cli.main()¶

Exectution of the foxmask module.

Parameters:	srcdir (str) – Top level directory where the module can find images to analyze. resultsdir (str) – Top level directory where to store the results of the analysis.