Finding that ONE picture among millions

I’ve been so taken with the photos of Megan Rapinoe lately. That confident, fill-your-heart, absolutely epic photo really resonates with me.

This is one of my favorite sports pictures of this decade. Thanks for being such an inspiration to my daughter @mPinoe - you were unbelievable today.

 — @BillSimmons

I had a hard time finding that exact photo and tracing it back to it’s source. I think it’s on Getty Images somewhere, but I gave up trying to find it.

It’s that exact kind of difficulty that led me to Zorroa several years ago. I got the UX job of a lifetime to think about how people can find what they need, aided by machine learning.

You can picture it in your mind…

Sometimes you know exactly what you need (that specific picture of Megan Rapinoe). When that happens you need search tools that map to your mental model. With data exploding, there is so much to sift through — how can that thing in your head be translated to what a computer can find? And how can you do it when you don’t have the text to explain the image in your mind?

This is why we’ve spent time exploring search widgets. Search widgets are tiny blocks that map to different mental models.

And sometimes, when our clients are searching for something they need to use all the time we get to work with them to create search models that work with their specific data.

When you only know a few things…

Let’s take for example the epic photo of Megan Rapinoe. How can Zorroa Visual Intelligence help me find it? Let me first tell you what I know. I don’t follow soccer, so I didn’t know her name or really anything about her other than she has great hair and the best power pose I’ve seen.

Since I’m not that familiar with the subject matter, I’d start by narrowing down my search. With databases of millions of images, we need to find the eye of the needle in the haystack. So we take the few things I know and build the search.

First I’ll narrow down by date, these databases can contain a hundred years of data or more. I think it was in the last month, but let’s be safe and search May — July, 2019.

I also know it was women’s soccer. So let’s say that our database has been analyzed (either by an existing model or using our new Domain Categorization feature) to categorize groups of images by type of sport. I’d use a facet widget and select Women’s Soccer.

From there I’d pop up another Facet Widget with player names. Since I’ve already filtered down, I’ll only get the names of players who meet the first two criteria. I can sort alphabetically or by popularity — since she’s a super star there are lots of photos of her.

Zorroa Search Widgets

There she is! Now I have all the photos of her in our database.

Great. Now, this is where I get to put on my future UX hat and say, but how do I find that one exact photo (or the group of photos that is closest so I’m only choosing from a handful)? This is where a conceptual widget comes in. A body pose widget.

Conceptual mental model—body pose search widget

In this conceptual widget you’d be able to choose which areas to focus on. I don’t know what she was doing with her feet, but that arm position is in my mind. I’d grab the drag points and pull that pose together.

Zorroa Pose Search Widget (Conceptual)

And like that, a subset of like-images would show up. All of that in 2 minutes or less.

Once you have the assets you are looking for our our clients are doing all sorts of things — using the images (and videos and documents) to create sizzle reels, merchandising, reports, or generally glean the information they need to make decisions and move forward.

We’d love to help you source the images, video, and other assets you need to build things like the absolutely breathtaking “I Believe” Nike ad.

https://medium.com/media/9b7ccee3666b4d4d6c0c315b69ea604c/href

What could you do with a system that applies the latest in machine learning to your massive repository of visual assets? What could you do if you had the help of the latest AI algorithms to organize, categorize, and make millions of assets searchable and useful in a matter of seconds? Want to find out?

H/T to HeroForge for helping me hatch the figure pose idea. I’ve been building a few models to use as personas.

Thanks to Juan Buhler for his help and input on this article.


Finding that ONE picture among millions was originally published in Zorroa on Medium, where people are continuing the conversation by highlighting and responding to this story.

artificial-intelligence, data-science, product-design, ux / 11 Jul 2019

“From Oscar to AI: Mining Visual Assets for Fun and Profit”

Unstructured data, such as videos, images, and Word and PDF documents, largely lives outside of structured databases, which can make it tough to corral. Instead of using SQL or a NoSQL query language to interrogate it, unstructured data can’t be directly queried. To get value out of unstructured data, one typically has to run it [...]

Media Mention / 22 Jan 2019

8 million videos analyzed by Google Machine Learning to create an Intelligent Archive - A race to the finish

The developers at Zorroa are always up for a challenge - the bigger the better. So when the team at Google approached us about using our Zorroa Visual Intelligence platform to demonstrate their latest and greatest video analysis algorithms, we said absolutely. The conversation went sort of like this: Google:     Can Zorroa help us [...]

Partners, Zorroa Platform / 10 Oct 2018

Zorroa is partnering with the Google Machine Learning Team at IBC 2018

Zorroa will be demonstrating our Intelligent Video Archive Search solution developed with the Google Machine Learning team in the Google booth at IBC 2018 in Amsterdam.  Using the YouTube-8M Dataset, Intelligent Video Search demonstrates how the Zorroa Visual Intelligence (ZVI) framework and UI put search in the hands of the customers of Google’s end-to-end video [...]

Zorroa Platform / 13 Sep 2018

Visual Search Innovator Zorroa Receives $7M Funding From Gradient Ventures And Strategic Enterprise Partners

BERKELEY, Calif., July 24, 2018 /PRNewswire/ -- Zorroa Corp. the company behind the first search and analysis platform able to automatically extract intelligence from visual assets and apply insights to data-driven business processes, has closed a $7M funding round. The round was led by Gradient Ventures, Google’s AI-focused, early-stage venture fund with participation from a [...]

Press Release / 24 Jul 2018

An Example Processor

In previous posts I did quick introductions to the Zorroa Platform, and to the Curator, the platform's web based front end. Today I'll do a quick overview of a Zorroa Processor. Processors are the building blocks of the import pipeline. They are basically plugins that can be written in Java or Python, and they can [...]

Technical, Zorroa Platform / 14 Dec 2017

Media Mention: AI in Movies, Entertainment, and Visual Media – 5 Current Use-Cases

Zorroa was recently reviewed in the Search Optimization section of AI in Movies, Entertainment, and Visual Media – 5 Current Use-Cases. The article had this to say: Zorroa offers a platform for managing visual assets that reportedly integrates machine learning algorithms to allow users to perform content searches within large databases. Documents are imported into what [...]

Media Mention / 21 Nov 2017

An introduction to the Zorroa Platform

Many, many images and videos It is a common problem: You have hundreds of thousands, or millions of images, videos and documents in general. They might be neatly organized into folders, but most often they aren't. Search in a database of that size is hard enough, let alone freely browsing, and "discovery" of value within [...]

Technical / 24 Oct 2017