Recognizing the growing trend toward providing resources to the open-source community, GIPHY, an online provider of GIFs and animated stickers, has launched “The GIPHY Celebrity Detector.”

Developed by the GIPHY R&D team, the free platform is a neural network that uses deep learning model to detect more than 2,300 celebrity faces found on GIPHY, including film and television stars, athletes, musicians, and politicians with an accuracy rate of 98 percent. It can also discern and make predictions on identities when there are multiple faces that appear simultaneously in GIFs and videos.

Building on its Entertainment Offering

GIPHY has always been about entertainment and popular culture. Approximately half of the search queries on GIPHY are related to the entertainment industry.

Users already enjoy all the GIFs, but it’s often the celebrity ones that get the most attention, according to GIPHY. Because of the demand for this type of GIF,  GIPHY Studios regularly partners with celebrities to create stickers, ads, and reactions.

The Celebrity Detector also helps the company to annotate this entertainment content to help locate specific GIFs within the company’s internal search engine.

How it Works

The celebrity detection process includes face detection and face recognition. When a GIF or image is submitted to the classifier, it uses a pre-trained model called MTCNN. Each face is submitted to this neural network, based on Resnet-50, and trained using the existing dataset mentioned above.

They also use a GMM algorithm to cluster each face by its vector representation. From there, each cluster receives an aggregate prediction, which is computed for all the faces within the cluster to yield one or more celebrity names along with a confidence score. The model’s final output combines these predictions across all clusters.

To train the deep learning platform, GIPHY’s research and development team took celebrity names from the top 50,000 searches across all of its platforms, including its website, mobile apps, integration with Slack, and social media integrations with Facebook and Twitter.  Also, to validate its celebrity detection accuracy, they used the open source labeled faces and a crowdsourced labeled dataset with approximately 1,000 popular Giphy celebrities.

Share and Learn

There are other platforms that offer something similar but they involve fees, which reduces access for machine learning and open source communities. By making the performant code available, anyone can leverage the platform for projects and apps.

GIPHY has also provided more technical information on how to use it in their Github open source repository.  Additionally, the GIPHY Engineering team will use the platform in-house to add metadata to their GIFs catalog. They also plan to open-source the model to the machine learning community for others to build off their work or integrate it into their projects

To try it out, you can go to their demo page where they offer 3D projection and a complete list of all celebrities available on the Celebrity Detector.