Setting Up Multiple GigE Cameras with Spinnaker

Applicable Products

All FLIR Machine Vision GigE cameras

Preparing for use

Before you use your camera, we recommend that you are aware of the following resources available from our website:

  • Camera Reference for the camera—HTML document containing specifications, EMVA imaging, installation guide, and technical reference for the camera model. Replace <PART-NUMBER> with your model's part number:
    http://softwareservices.flir.com/<PART-NUMBER>/latest/Model/Readme.html
    For example:
    http://softwareservices.flir.com/FFY-U3-16S2M-DL/latest/Model/Readme.html
  • Getting Started Manual for the camera—provides information on installing components and software needed to run the camera.
  • Technical Reference for the camera—provides information on the camera’s specifications, features and operations, as well as imaging and acquisition controls.
  • Firmware updates—ensure you are using the most up-to-date firmware for the camera to take advantage of improvements and fixes.
  • Tech Insights—Subscribe to our bi-monthly email updates containing information on new knowledge base articles, new firmware and software releases, and Product Change Notices (PCN).

Overview

When choosing to do a multi-camera system setup, there a number of parameters necessary to reivew to ensure a successful setup.  Specifically:

  • Using multiple GigE host adapters or a switch will determine the overall bandwidth limitation seen by a camera.  Using multiple host adapters means each host adapter provides full interface bandwidth; using hubs/switches may limit total camera bandwidth to the interface bandwidth limit, but they can also be used universally by desktop, laptop and embedded system devices.
  • DeviceLinkThroughputLimit is a camera setting that detrmines the total bandwidth each camera can take up.  When using multiple cameras on one bus/host adapter, it is important to reduce this to ensure that the combined bandwidth of all of the cameras do not exceed the bandwidth limitations of the interface, while maintaining a certain value to achieve your application required resolution/framerate combination.
  • System Components are important to determine the processing performance of a computer/host system.  It's important to know the specifications necessary to run multiple cameras, and what kind of performance you can expect for a given hardware system.

This article provides details on system setups we have tested using multiple cameras, the camera settings used to achieve that bandwidth, and things to look out for when configuring your own multi-camera system.

Configuring Network Adapters and Cameras

Configuring the Network Adapter

The Adapter Config Utility is installed with the Spinnaker SDK; it allows you to control the configuration of your network adapter. The default location for this .exe file is C:\Program Files\Point Grey Research\Spinnaker\bin64\vs2015 if you downloaded the Spinnaker 64x Windows and selected to use Visual Studio 2015 in the installation process.

image001.png

The Adapter Config Utility lists your network adapters and allows you to access the following:

  • Adapter IP address
  • Subnet mask
  • Default gateway
  • Adapter receive buffers
  • Transmit buffers
  • Jumbo packets

image002.png

Setting the Adapter IP Address, Subnet Mask, and Default Gateway

You can either fill in the IP address manually or use “Auto Fill All” which assigns the IP address for the network adapter automatically. The camera and network adapter must be on the same subnet for the camera to show up in SpinView.

image003.png

Setting the Adapter Advanced Properties

The receive buffer is the size of system memory that the adapter can use for receiving packets, which can be increased to help improve the performance of incoming network traffic.

On PCs with limited RAM (<4GB), setting the receive buffer size to the maximum may have a negative impact as buffers consume system memory. On most systems the maximum setting can be applied without reducing available memory.

The adapter uses the transmit buffer to send packets.

The jumbo packet size influences the number of interrupts generated which affects CPU usage. The larger the packet size, the fewer the interrupts for the same amount of data.  To minimize CPU usage, increase the packet size. 

The upper limit depends on your host adapter, your Ethernet switches (if used), and the camera. Usually, the limit is 9000 or 9014.

Select 9014 for Jumbo Packets. If your adapter does not support such a large packet (or MTU) size, then you will experience slightly higher CPU usage.

image004.png

Configuring the Camera's IP Address

The camera's IP address can be assigned in three ways:

Persistent—Both the adapter and the camera have a fixed IP address that will not change. Generally the address is within a closed network range of 192.168.X.X. The adapter and the camera must be on the same subnet.

Note: Persistent IP addressing is recommended for multiple cameras on multiple ports.

Dynamic (DHCP)—Both the camera and the adapter are set to automatically obtain an IP address. This means that the IP address will dynamically change (within a range) every time the camera or computer is restarted. It may take up to one minute for the IP address to resolve and the camera to enumerate.

Note: In order to use DHCP, you need to have a DHCP server running; this article does not go over how to set this up.

Default (LLA)—Both the camera and the adapter use a default IP address from the link-local address block 169.254.x.x.

The cameras assign its current IP address in the following sequence:

  1. Persistent - Uses the defined IP address. If not available, then;
  2. DHCP – Attempts to find a dynamic IP address. If not available, then;
  3. LLA – Uses the default IP address

Use SpinView to set the camera's IP address.

image006.png

Auto Force IP

SpinView can automatically force an IP address refresh. This detects the IP address of the network adapter and automatically sets the camera’s IP address relative to the adapter. If a camera’s IP address is not properly set to match the host adapter the camera is attached to, a red circle with a white exclamation mark inside of it is shown beside the camera.

image007.png

Using Multiple Host Adapters

When setting up multiple cameras using multiple host adapters, it is important to put each host adapter on a unique subnet.

For example:

Subnet A IP Address Subnet Mask
Host Adaptor 1 192.168.100.1 255.255.255.0
Camera 1 192.168.100.2 255.255.255.0
Camera 2 192.168.100.3 255.255.255.0
     
Subnet B IP Address Subnet Mask
Host Adaptor 2 192.168.200.1 255.255.255.0
Camera 3 192.168.200.2 255.255.255.0
Camera 4 192.168.200.3 255.255.255.0

Allocating Bandwidth

The User Datagram Protocol (UDP) used by the GigE Vision standard provides no guaranteed transmission or fixed timing mechanism. Therefore, bandwidth must be managed by adjusting Device Link Throughput Limit based on desired resolution and frame rate.

Calculating Required Bandwidth

The maximum bandwidth for GigE interface is 125 MB. This includes image data, control data and image resends, which occur when frames are dropped. Each image and packet has a certain amount of overhead that uses some bandwidth; therefore, when calculating your bandwidth requirements, you should not attempt to use the full maximum of 125 MB.

To calculate your bandwidth requirements:

Determine your required resolution, frame rate and pixel format (bytes per pixel) as follows:

Height x Width x Frame Rate x Bytes per Pixel = Bandwidth in MB

As an example, for an image that is VGA, 82 FPS, in Mono8:

640 (H) x 480 (W) x 82 (FPS) x 1 (BPP) = ~ 25 MB

Understanding your required bandwidth helps you determine what kind of configuration you need for setting up multiple cameras.

Once you have calculated your required bandwidth, you can allocate an amount to each camera by adjusting the Device Link Throughput Limit. Allocating a specific amount to each camera helps to avoid dropped packets due to a data burst. You would do this in a set up with multiple cameras, or in a situation where the system bandwidth might be limited or shared due to hardware architecture.

To allocate 25 MB ~ 20% of bandwidth:

Device Link Throughput Limit = 25000000

To allocate 55 MB ~ 45% of bandwidth:

Device Link Throughput Limit = 55000000

image008.png

Setting up Multiple Cameras

Multiple Cameras connected to a Single Ethernet Port through a Switch

With a single Ethernet port, multiple cameras are connected to the host adapter via a switch. This setup combines the bandwidth from each camera onto a single cable; therefore, the sum of bandwidth used by all cameras cannot exceed 125 MB.

If you see frames being dropped on an individual camera, reduce the Device Link Throughput Limit for each camera to increase the packet delay. This helps reduce the total bandwidth.

Note: If possible, the switch should be configured to accept large packet sizes. You may need to adjust the camera’s packet size to suit the switch’s capabilities. The switches we use as examples in this article do not run into this issue.

Any method of IP addressing (Persistent, Dynamic, or LLA) can be used for this set up.

image009.png

Component Description
CPU Intel Core i7-8700 @ 3.20GHz
Motherboard Asus Prime Z370-A II
RAM 16 GB
Network adapter GIGE-PCIE2-2P02 (Intel 82574L controller)
Network switch Netgear GS105 5-Port Gigabit Switch
Cabling Cat5e
Operating system Windows 10 x64
Driver GigE Filter Driver (PgrLwf) 2.7.3.111
Software Spinnaker Release 1.23.0.27 x64
Camera firmware 1810.7.3.0

Example A – Two cameras connected to a switch

Camera Model Pixel Format Resolution Frame Rate Packet Size (Bytes) Device Link Throughput Limit (Bytes/s) Bandwidth (MB/s)
BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 34 9000 54000000 54
BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 34 9000 54000000 54
Total Bandwidth 108 MB/s

Example B – Four cameras connected to a switch

Camera Model Pixel Format Resolution Frame Rate Packet Size (Bytes) Device Link Throughput Limit (Bytes/s) Bandwidth (MB/s)
BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 15 9000 25000000 25
BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 15 9000 25000000 25
BFS-PGE-16S2M-CS Mono8 1440 x 1080 15 9000 25000000 25
BFS-PGE-16S2M-CS Mono8 1440 x 1080 15 9000 25000000 25
Total Bandwidth 100 MB/s

Multiple Camera Connected to Multiple Ethernet Ports

With multiple Ethernet ports, multiple cameras connect directly to a dedicated port. In this set up, each camera is able to use the full 125 MB bandwidth.

The host can have:

  • One adapter with multiple ports: If you have a multi-port Ethernet adapter, each port allows a maximum of 125 MB bandwidth. However, if the adapter has a single port bandwidth then the cameras connected share 125 MB bandwidth.
  • Multiple adapters each with a single port

Persistent IP addressing is recommended for this set up. Each camera should have a unique IP address on the subnet of its port. Each port should be on a different subnet.

image010.png

Example A – Two cameras each connected directly to a network adapter

Port Camera Model Pixel Format Resolution Frame Rate Packet Size (Bytes) Device Link Throughput Limit (Bytes/s) Bandwidth (MBytes/s)
A BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 63 9000 100000000 100
B BFS-PGE-16S2C-CS BayerRG8 1440 x 1080 63 9000 100000000 100
Total Bandwidth 200 MB/s

Multiple Cameras Connected to Multiple Ethernet Ports and Multiple Switches

Using multiple Ethernet ports and multiple switches, you can increase the number of cameras connected on the host system. For this setup each Ethernet port allows a maximum of 125 MB bandwidth, which is shared by the cameras that are connected to the port through a switch.

Note: Jumbo packets need to be enabled for the switch to allow the camera to send larger image packets.

Troubleshooting

Solving Enumeration Problems

If your camera is not showing up in SpinView, consider the following:

Is the camera powered?

  • Check that the camera LED is on
  • Power is provided via GPIO on Flea3 GigE and Grasshopper2 GigE. Power is provided via PoE or GPIO on Blackfly S or Blackfly

Is there an IP conflict?

  • Ensure the camera is using a unique IP address
  • The camera and host adapter must be on the same subnet

Is the firewall blocking Spinnaker?

  • Add Spinnaker to your firewall program’s exceptions list or disable the firewall

Is other network traffic blocking your camera?

  • Use a dedicated adapter for the camera and a separate adapter for the Internet or other devices

Do you see a red exclamation mark next to your camera?

image011.png

This is usually caused by unmatched subnet address between the camera and the network adapter.

  • Auto-force IP and Refresh

       or

  • Manually change the IP address of the camera to match the subnet of the adapter

Eliminating Image Consistency Errors

Image consistency errors are often caused by dropped packets. Ethernet is asynchronous in nature and capable of bursting data to peak bandwidth. A sudden burst in data transfer can create a packet collision and lead to image consistency errors.

If you’re experiencing image consistency errors, consider the following:

Is the correct driver installed?

  • Install the Point Grey Lightweight Filter Driver to reduce the packet load on the CPU.
  • image012.png
  • If you do not see “Point Grey Lightweight Filter Driver” in your Network Connection Properties window, then install it by clicking “Install” button -> Service -> Add -> PointGrey -> Point Grey Lightweight Filter Driver.

    Are all network components able to handle the packet size?

    • Increase the packet size to 9014 for the host adapter and the camera. You can increase the camera’s packet size in SpinView and increase the host adapter’s packet size from the Network and Sharing Center (Windows) or Adapter Config Utility.
    • Ensure any switches in the network are also able to handle the packet size.

    Is the packet delay too low?

    • Reduce the Device Link Throughput Limit to increase the packet delay. This slows down the average camera transfer rate and it can help with slower networks, less capable host computers, or for multiple cameras.

    Is the computer able to handle the data?

    • Free up system resources by ending unnecessary programs or processes.