Finding that ONE picture among millions

I’ve been so taken with the photos of Megan Rapinoe lately. That confident, fill-your-heart, absolutely epic photo really resonates with me.

This is one of my favorite sports pictures of this decade. Thanks for being such an inspiration to my daughter @mPinoe - you were unbelievable today.

 — @BillSimmons

I had a hard time finding that exact photo and tracing it back to it’s source. I think it’s on Getty Images somewhere, but I gave up trying to find it.

It’s that exact kind of difficulty that led me to Zorroa several years ago. I got the UX job of a lifetime to think about how people can find what they need, aided by machine learning.

You can picture it in your mind…

Sometimes you know exactly what you need (that specific picture of Megan Rapinoe). When that happens you need search tools that map to your mental model. With data exploding, there is so much to sift through — how can that thing in your head be translated to what a computer can find? And how can you do it when you don’t have the text to explain the image in your mind?

This is why we’ve spent time exploring search widgets. Search widgets are tiny blocks that map to different mental models.

And sometimes, when our clients are searching for something they need to use all the time we get to work with them to create search models that work with their specific data.

When you only know a few things…

Let’s take for example the epic photo of Megan Rapinoe. How can Zorroa Visual Intelligence help me find it? Let me first tell you what I know. I don’t follow soccer, so I didn’t know her name or really anything about her other than she has great hair and the best power pose I’ve seen.

Since I’m not that familiar with the subject matter, I’d start by narrowing down my search. With databases of millions of images, we need to find the eye of the needle in the haystack. So we take the few things I know and build the search.

First I’ll narrow down by date, these databases can contain a hundred years of data or more. I think it was in the last month, but let’s be safe and search May — July, 2019.

I also know it was women’s soccer. So let’s say that our database has been analyzed (either by an existing model or using our new Domain Categorization feature) to categorize groups of images by type of sport. I’d use a facet widget and select Women’s Soccer.

From there I’d pop up another Facet Widget with player names. Since I’ve already filtered down, I’ll only get the names of players who meet the first two criteria. I can sort alphabetically or by popularity — since she’s a super star there are lots of photos of her.

Zorroa Search Widgets

There she is! Now I have all the photos of her in our database.

Great. Now, this is where I get to put on my future UX hat and say, but how do I find that one exact photo (or the group of photos that is closest so I’m only choosing from a handful)? This is where a conceptual widget comes in. A body pose widget.

Conceptual mental model—body pose search widget

In this conceptual widget you’d be able to choose which areas to focus on. I don’t know what she was doing with her feet, but that arm position is in my mind. I’d grab the drag points and pull that pose together.

Zorroa Pose Search Widget (Conceptual)

And like that, a subset of like-images would show up. All of that in 2 minutes or less.

Once you have the assets you are looking for our our clients are doing all sorts of things — using the images (and videos and documents) to create sizzle reels, merchandising, reports, or generally glean the information they need to make decisions and move forward.

We’d love to help you source the images, video, and other assets you need to build things like the absolutely breathtaking “I Believe” Nike ad.

What could you do with a system that applies the latest in machine learning to your massive repository of visual assets? What could you do if you had the help of the latest AI algorithms to organize, categorize, and make millions of assets searchable and useful in a matter of seconds? Want to find out?

H/T to HeroForge for helping me hatch the figure pose idea. I’ve been building a few models to use as personas.

Thanks to Juan Buhler for his help and input on this article.

Finding that ONE picture among millions was originally published in Zorroa on Medium, where people are continuing the conversation by highlighting and responding to this story.

artificial-intelligence, data-science, product-design, ux / 11 Jul 2019