This troubleshooting guide catalogs common issues, organized by symptom, within three categories: setup, jobs, and data collection. For each symptom, the likely causes and the resolutions to each of those causes are described.
If you are already aware of the cause of an issue and are just looking for the steps for fixing it, see the Common Procedures section.
This guide assumes familiarity with Linux commands. See the manual pages for details about individual Linux commands.
During the ItemSense installation, the script displays "cannot start service itemsense" or "encountered errors while bringing up the project". The script may continue to run, but it eventually exits with a status 1. The most likely cause is a conflict on one of the ports that ItemSense requires, as specified by the port requirements in the Resource Requirements guide. For example, if an Apache web server is running, it is consuming port 80.
Use the following command to see which port is being consumed:
sudo netstat -tlnp | grep ':80\W\|:3010\W\|:443\W\|:5672\W'
Stop the process that is consuming the port and uninstall the package.
Go to the directory where you extracted the ItemSense archive file. Rerun the ItemSense installation script in the tarfile directory. For specific instructions, follow the steps beginning at step 5 of the Install Instructions in the Installation Guide.
With Octane Firmware 5.10 and greater, Network Time Protocol (NTP) settings cannot be modified without first disabling the NTP service.
Use the following command to disable the NTP service:
config network ntp disable
Networking issues may cause reader provisioning failures. Although ItemSense may be able to communicate with the readers, the readers may not be able to communicate with ItemSense.
When the readers are provisioned, the IP address or hostname used to access the ItemSense Management Console (IMC) is written to the reader as the ItemSense address. If this is an IP address or hostname that the reader has trouble connecting to, then the reader will have connectivity issues as it tries to communicate back to the IMC.
If the ItemSense administrator password has been lost, it is possible to reset it back to its default value "admindefault".
Note: This will reset ALL configuration settings
Connect to ItemSense server using the Secure Shell (SSH) protocol.
Use the following command to edit the admin.json file:
sudo vi ~/containers/itemsense/home/itemsense/data/configstore/users/admin.json
Replace the contents of the file with the following text:
Reboot the ItemSense server, and log in to the IMC using "admindefault" as the password.
When ItemSense is stopped, one or more error messages are displayed. For example, you may see a message such as driver "overlay" failed to remove root filesystem. You must not restart ItemSense without addressing the issues.
Address the issues that were indicated by the error messages. If you are unsure how to proceed, contact Impinj Support.
Verify that no containers are running:
sudo docker ps
When ItemSense is stopped and then restarted, all configuration data is lost.
Contact Impinj Support.
If the ItemSense hostname changes, you will get an "invalid user/password combination" when attempting to access the IMC.
Recreate the docker containers:
sudo docker-compose -f itemsense-prod-3.yml down
sudo docker-compose -f itemsense-prod-3.yml up -d
The overall health status of a reader is either NOT_RESPONDING or IDLE with last check-in reporting null, as shown in the following screenshot. A reader should check in with ItemSense at 15-second intervals. The most likely cause is that the reader is no longer paired with this instance of ItemSense or the reader is disconnected from the network.
There may be several causes of this issue.
Contact Impinj Support.
NTP time-synchronization issues are the most common cause of tag-read issues. Readers must be synchronized with each other and with the ItemSense server. The following situations may prevent this synchronization from happening:
Network issues or a lack of sufficient disk space may also cause tag-read issues.
Check the job configuration and verify that no filters are preventing tags in the field of view from being read.
Visually inspect the LED lights of the readers to verify that the jobs are running and that they have not been rebooted since the start of the job.
Verify that NTP is working properly:
This command displays information about the peer servers specified in the NTP configuration file. Sample output is shown below. A
+ symbol at the beginning of the line indicates that the peer can be used for synchronization. If no peer is marked by a
* or a
+, NTP is not operating properly.
If NTP is working properly, proceed to the next step; otherwise, set up the ItemSense server as an NTP server and configure the readers to use the ItemSense server as their NTP server, as described in the ItemSense Installation document.
Confirm that there is enough disk space on the ItemSense server. See the Server Requirements section in the ItemSense documentation for resource requirements.
Prune the ItemSense logs to reclaim disk space.
If problems persist, contact Impinj support.
The RabbitMQ broker will quit after an hour of inactivity (no tag reads), and the connected app will not receive any notification that the connection is broken.
No resolution is necessary — this behavior is by design.
If a job stops running before it completes or if a job is unable to start and the IMC displays the following message, the problem may be that there is not enough disk space on the ItemSense server.
If ItemSense is stopped while a job is running, it should be able to continue to receive data from the reader once it restarts. Data may be lost if reader’s buffer fills before ItemSense is started.
If the network is temporarily disconnected from a reader while the reader is running a job, ItemSense does not display an error message to indicate that it has lost contact with the reader.
Although ItemSense may be able to communicate with the readers, the readers may not be able to communicate with ItemSense. Use the following steps to verify that the readers can communicate with ItemSense.
NOTE: The ports used for reader communication to ItemSense are different from the ports used for ItemSense communication to a reader. For information about the network ports used by ItemSense and the readers, see the Network Requirements section in the ItemSense documentation for resource requirements.
Log on to the ItemSense server.
ifconfig command to see which network interface is being utilized for connectivity between ItemSense and the readers.
tcpdump command to test for reader-to-ItemSense connectivity issues.
For example, if eth1 is the network interface, use the
tcpdump command as follows:
To test for basic reader-to-ItemSense connectivity, specify port 443:
tcpdump -i eth1 port 443 -w tcpdumpfile.443.pcap
To test reader-agent provisioning, specify port 51505:
tcpdump -i eth1 port 51505 -w tcpdumpfile.51505.pcap
To test SSH commands sent to the reader, specify port 22:
tcpdump -i eth1 port 22 -w tcpdumpfile.22.pcap
To test NTP communication from the reader to ItemSense, specify port 123:
tcpdump -i eth1 port 123 -w tcpdumpfile.123.pcap
tcpdump command fails or completes, close
tcpdump and load the resulting .pcap file into your preferred network traffic analyzer.
ItemSense logs are contained within the home directory of the user who installed it. Each ItemSense service will have its own directory within the ItemSense parent directory. Generally, log files are only created when ItemSense is being used. In other words, no logs are created if ItemSense isn't being used.
Log into the ItemSense server using the ItemSense installer’s login with an SSH or FTP client.
Delete the largest logs from the general ItemSense logs directory.
Delete the largest logs from the API service ItemSense logs directory.
Delete the largest logs from the coordinator ItemSense logs directory.
Perform the following steps when a reader needs to be provisioned:
Delete the reader configuration from the IMC if it exists.
Connect to the reader using SSH.
Remove the reader’s CAP file:
config image removecap
Close the SSH client, and use the IMC to re-add the reader.
You can determine the instance of ItemSense with which a reader is paired by examining the reader's config.json file. Use the following steps to retrieve this file from a reader.
Connect to the reader via secure shell:
ssh root@<reader hostname>
Once connected, use the config network RShell command to enable FTP on the reader:
config network ftp enable
Exit out of RShell and connect to the reader via an FTP client:
ftp <reader hostname or reader IP address>
Change to the reader's cust directory, download the config.json file, and exit out of your FTP client. The commands may vary slightly, depending on the FTP client that you use. The following commands are an example:
cd / cd cust get config.json quit
Display the contents of the reader's config.json file by using the cat command:
The contents of the baseurl field in the .json file specify the instance of ItemSense that the reader is paired with.