Historically, TrapTagger used to work on the assumption that all data was uploaded correctly and thus did not facilitate the management of the uploaded data thereafter. Modifications like deleting or moving files and folders may sound simple enough on the surface, but there are many underlying factors that complicate things – such as our use of low-cost archival cloud storage which inherently doesn’t allow for the renaming or moving of folders, whilst also requiring 48 hour notice on operations. Moreover, modifying the files associated with a site or station has a lot of knock-on effects such as requiring re-clustering which in turn requires the re-evaluation of AI species labels, and even human labels – and all that’s without mentioning the plethora of edge cases that inevitably crop up. Thankfully, such mistakes were mostly caught early on and thus could be simply solved by deleting the survey and starting afresh. However, as the platform has grown, it has become simply infeasible for us to manually assist users with cleaning up already-annotated surveys – resulting in us undertaking this effort.
File Management
The primary feature is the addition of the new file-management tool, shown in Figure 1, that is accessed as a tab on a ‘manage files’ window that replaces the old ‘add files’ option for a survey. Since it also functions as a way to view your survey structure, it also replaces the survey-structure viewer that used to appear under the edit survey options.
Figure 1: The file-management tool with a singe file and folder scheduled for deletion. The highlighted camera name has also edited, and the highlighted folder has been moved to a different camera.
In essence, this tool displays each individual folder you have uploaded (identified by path), the number of images and videos it contains, and how it relates to the structure of your survey in terms of your sites/stations and cameras. From here you will have the following options:
- Rename cameras and sites/stations by simply clicking on their name and typing a new one.
- Move folders to different cameras or sites/stations – both existing and new – by clicking on the move icon.
- Delete folders by simply clicking on the associated bin icon.
- View folder contents by clicking the folder icon as shown in Figure 2.
- Here the individual files are listed along with their timestamp, and their import date.
- Files can be selected for deletion using their associated bin icon.
- File selections can be reversed using the arrow icon.
- You can filter contents by both timestamp and import date, which can be used in conjunction with the select all option to find and delete files that a particular upload session or from outside the survey period.
- You can also search file names to help narrow your search.
Figure 2: The file-contents viewer with some files selected for deletion.
All your modifications will be accumulated on the main edit file tab and displayed as shown in Figure 1 for ease of use. On confirmation, you will be presented with a summary of the changes you have requested, which you must then confirm before they are executed. Note that these changes might take some time to process depending on the extent of the changes requested. As usual, users will need write permission to make modifications, and delete permissions to delete files.
Please Note: Folders are treated discretely in all operations – in other words sub folders are ignored. So for example if you have a folder of images that contains a sub folder with additional images and you want to delete all the images, you will need to select both folders.
Upload Cancellation
A second part of this update is the addition of the option to cancel an ongoing upload. This is for when you started adding the wrong files to the incorrect already-existing survey. This feature, shown in Figure 3, will delete all the additional images that were added by the aborted upload and safely revert the survey back to the state it was previously. This option can be accessed by pausing the download and then clicking the cancel button. It can only be used by organisational members with delete permission on the survey in question, and is unavailable to users whilst somebody else is actively continuing the upload.
Figure 3: The new cancel upload option and its confirmation message.
Additional-Image Clustering
With our update to the clustering algorithm last year, we decided that it would be most prudent to fully re-cluster sites where images were added since this could potentially affect clustering, which in turn would affect the context available for animal identification. Therefore, nothing and unknown labels were dropped when files were added to a survey to allow users to double check these clusters to ensure that they still apply. However, in cases where non-chronologically overlapping images were added to a site, this could present a nuisance to users have a large number of empty and unknown clusters. Therefore, we have now modified the clustering technique to not drop these labels in such a scenario.