Expanding ZeroFOX’s Computer Vision Capabilities with Weapon and Credit Card Detection

Investing in AI and Computer Vision 

Over the last 12 months, ZeroFOX has invested heavily in Artificial Intelligence, Computer Vision and Machine-Learning tools to accurately identify a broad range of digital threats at scale. Consuming the internet is no small feat and cannot be done by human analysis alone. Tools like Optical Character Recognition (OCR) to identify harmful text within images, negative sentiment analysis to more accurately understand if posts are threatening, and malicious link detection increase the speed and accuracy at which the ZeroFOX Platform and our analysts are able to identify and take action against top digital threats. Today, ZeroFOX expands those capabilities further, going from text analysis, to text in image analysis, to now detecting threatening objects within images.

ZeroFOX is excited to announce the launch of Object Detection to identify weapons and credit cards within images. Continuing our investment in building industry leading AI and computer vision technology, weapon detection is a natural fit to support an important use case for corporate security, violence and physical threats, and location protection.  Expanding on the ability to detect generic objects like cars or books within images as some vendors do, ZeroFOX is acutely focused on detecting objects which are more relevant to security use cases, beginning with threatening objects – guns and knives. To bring these new capabilities into plain view, we’ve included visualizations within the Platform’s Alert View, overlaying the images with bounding boxes and labels to accurately show what was detected alongside any text content, so you can view and determine what is a directed threat vs. a harmless picture of a hunting trip.

Protecting Executives and Locations

There’s no question a threat of violence made online alongside images of guns or knives can be highly suggestive of a real attack, and something to take seriously.  Particularly when there are mentions of high profile executives and VIPs, it is worth taking a careful look at any comments to see if there may be an implied threat that may have otherwise gone undetected through traditional text analysis. Alternatively, it could be a harmless social post of a hunting trip with a protected exec, however, most security teams would err on the side of caution.

Weapon detection can be very useful when it comes to protecting locations as well.  For both corporate security and event protection, knowing if there are social posts including images of weapons near stadiums, conference centers or other site locations, which could be posted live or near live shots, is critical to informing and adjusting security posture for physical security teams. Understanding potential threats before they are actualized helps corporate security teams gain situational awareness and early warning.

In the ZeroFOX Platform

The platform experience is designed to capture the full nature of the threat, giving users clear, actionable context for why ZeroFOX generated the alert and why it is important. For each Weapon Detection alert, the offending content is displayed to immediately show the threat posed. Bounding boxes overlayed on analyzed images to show the type of object detected by our AI model with its associated confidence level, the score outputted by our AI model indicating the likelihood that the associated object is labeled correctly. Security teams can configure the threshold to alert within the policies page, giving them more control over when the rule alerts.  Content text is displayed alongside the image to help users identify if there is a serious threat within the post associated with the image.

For alerts associated with a specific protected location, ZeroFOX prominently displays the associated location so corporate security teams have necessary context. Location alerts display an interactive map, showing the protected location with radius, and specifically pinpoint where the content originated. This is particularly useful context during live events with large groups of people where the threat of violence or disturbance may be high.

Like all the capabilities within ZeroFOX’s robust AI toolkit, weapon detection is a service available for use in other rules, including custom rules, and in conjunction with other AI services such as our negative sentiment classifier and optical character recognition (OCR).  This gives businesses full flexibility to add the weapon detection tool to their larger security program or customize logic to suit individualized needs.

Expanding Computer Vision and Object Detection Capabilities to Credit Card Detection

Finally, we are expanding our object detection to other objects, again with a focus on top security use cases. One of the top threats facing industries like financial services includes the illegal buying and selling of credit cards and money flipping scams across social media, surface, deep and dark web. We will be releasing credit card detection within images in support of our financial clients, where use of credit cards in images tend to be a sign of inadvertent information exposure or carding scams. Stay tuned for more information soon!

See the capabilities in action

Want to see these Computer Vision capabilities in the platform? We created a quick video that shows two object detection examples focused on identifying a weapon and credit card.