Reputation: 331
Automatically giving tags to image for a search function
I play a game which uses skins, around 200'000 of them and the players struggle to find them since there isn't really a search function other than random names. Each skin has a name chosen by the users who upload them and that's it, others don't know about it untill they see it used by someone else. I made some tools to make it easier and I wanted to implement a search by tag function, for example you search "dog" and it gives you the skins containing dogs. Is there a program able to automatically give tags to 200k images?
Game: https://Gota.io
My "tool"(badly made website) https://specy.it/SkinGallery.html
And the extension for the game: https://www.dropbox.com/s/36w6dkz22pxksrz/Extension.js?dl=0
Upvotes: 0
Views: 76
Answers (1)
Reputation: 17481
This is not my specialty and it's an evolving field, but for the time being there are "image recognition as a service" services such as Google Vision whose rest API you can use to have the machine tag your image according to what it sees.
That being said, I'm not sure if your skins are suit for a good result. If we try it on the "dead mickey" skin, we get:
Well yeah, there's an animal, I guess. If you look at the "labels" tab you'd see it describes mostly colors, but it also mentions "fictional character" in the 12th place or so.
Depending on the skin, different contexts could suit you better. For example, if you use Sage Mode Naruto, Objects, Labels and Logos bring very little information, but Web tab says:
How would you handle the response of the rest service? Well, it's a JSON with the form:
{
"cropHintsAnnotation": {
"cropHints": [...]
},
"fullTextAnnotation": {
"pages": [
{
"blocks": [...],
"height": 512,
"property": {
"detectedLanguages": [
{
"confidence": 0.88,
"languageCode": "en"
}
]
},
"width": 512
}
],
"text": "SUH DUDE\n.\n"
},
"imagePropertiesAnnotation": {
"cropHints": [...],
"dominantColors": {
"colors": [...]
}
},
"labelAnnotations": [
{
"description": "Illustration",
"mid": "/m/01kr8f",
"score": 0.8927454,
"topicality": 0.8927454
},
{
"description": "Cartoon",
"mid": "/m/0215n",
"score": 0.8583309,
"topicality": 0.8583309
},
{
"description": "Circle",
"mid": "/m/01vkl",
"score": 0.70432633,
"topicality": 0.70432633
},
{
"description": "Art",
"mid": "/m/0jjw",
"score": 0.6253477,
"topicality": 0.6253477
},
{
"description": "Clip art",
"mid": "/m/03g09t",
"score": 0.5876344,
"topicality": 0.5876344
},
{
"description": "Fictional character",
"mid": "/m/02h7lkt",
"score": 0.52791756,
"topicality": 0.52791756
}
],
"localizedObjectAnnotations": [...],
"logoAnnotations": [...],
"description": "Los Angeles Rams",
"mid": "/m/06x76",
"score": 0.5958965
}
],
"safeSearchAnnotation": {
"adult": "VERY_UNLIKELY",
"medical": "VERY_UNLIKELY",
"racy": "VERY_UNLIKELY",
"spoof": "VERY_UNLIKELY",
"violence": "VERY_UNLIKELY"
},
"textAnnotations": [...],
{
"boundingPoly": {
"vertices": [
{
"x": 182,
"y": 40
},
{
"x": 295,
"y": 40
},
{
"x": 295,
"y": 111
},
{
"x": 182,
"y": 111
}
]
},
"description": "SUH DUDE\n.\n",
"locale": "en"
},
{
"boundingPoly": {
"vertices": [
{
"x": 182,
"y": 44
},
{
"x": 222,
"y": 43
},
{
"x": 223,
"y": 66
},
{
"x": 183,
"y": 67
}
]
},
"description": "SUH"
},
{
"boundingPoly": {
"vertices": [
{
"x": 237,
"y": 42
},
{
"x": 294,
"y": 40
},
{
"x": 295,
"y": 64
},
{
"x": 238,
"y": 66
}
]
},
"description": "DUDE"
},
{
"boundingPoly": {
"vertices": [
{
"x": 194,
"y": 101
},
{
"x": 196,
"y": 100
},
{
"x": 200,
"y": 111
},
{
"x": 198,
"y": 111
}
]
},
"description": "."
}
"webDetection": {
"bestGuessLabels": [
{
"label": "cartoon"
}
],
"pagesWithMatchingImages": [
{
"pageTitle": "edit #anime #naruto #foryou #foryoupage #narutouzumaki ... - TikTok",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://www.tiktok.com/@sakura._.haruno_editz/video/6777103057055517958?lang=en"
},
{
"pageTitle": "Originalton - edix_official TikTok Music Videos • TikyToky",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://tikytoky.com/music/originalton-6768700410359302918"
},
{
"pageTitle": ".... Wanna gain tons of followers? Follow @dragonballexclusives ...",
"partialMatchingImages": [
{
"url": "https://scontent-frt3-1.cdninstagram.com/vp/cdb9be9aa9707b90d55564a965668b7c/5E881986/t51.2885-19/s150x150/71524634_2513191889001976_2699215275693178880_n.jpg?_nc_ht=scontent-frt3-1.cdninstagram.com"
}
],
"url": "https://www.mystalk.net/detail/1917580489812869720_7212599026"
},
{
"pageTitle": "edit #anime #naruto #foryou #foryoupage #narutouzumaki ... - TikTok",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://www.tiktok.com/@sakura._.haruno_editz/video/6777103057055517958?region=KR&langCountry=pa&source=h5_t&lang=ar"
},
{
"pageTitle": "edit #anime #naruto #foryou #foryoupage #narutouzumaki ... - TikTok",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://www.tiktok.com/@sakura._.haruno_editz/video/6777103057055517958?region=KR&langCountry=pa&source=h5_t&lang=en"
},
{
"pageTitle": "edit #anime #naruto #foryou #foryoupage #narutouzumaki ... - TikTok",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://www.tiktok.com/@sakura._.haruno_editz/video/6777103057055517958?region=JP&langCountry=ja"
},
{
"pageTitle": "edit #anime #naruto #foryou #foryoupage #narutouzumaki ... - TikTok",
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
}
],
"url": "https://www.tiktok.com/@sakura._.haruno_editz/video/6777103057055517958?lang=de"
}
],
"partialMatchingImages": [
{
"url": "https://p16.muscdn.com/obj/tos-maliva-p-0068/b27c516d2faee4174d0e4a26b7a3d466"
},
{
"url": "https://scontent-frt3-1.cdninstagram.com/vp/cdb9be9aa9707b90d55564a965668b7c/5E881986/t51.2885-19/s150x150/71524634_2513191889001976_2699215275693178880_n.jpg?_nc_ht=scontent-frt3-1.cdninstagram.com"
}
],
"visuallySimilarImages": [
{
"url": "https://previews.123rf.com/images/lenm/lenm1703/lenm170300080/73206609-sketchy-illustration-featuring-a-woman-looking-fondly-at-her-baby-strapped-to-a-carrier.jpg"
},
{
"url": "https://media.gettyimages.com/vectors/cartoon-soccer-football-girl-character-vector-id487299807"
},
{
"url": "https://pbs.twimg.com/profile_images/1062176350797066241/UhaIkhRy_400x400.jpg"
},
{
"url": "https://img.favpng.com/1/13/23/vertebrate-line-art-cartoon-clip-art-png-favpng-2LN24w4JHvqm4VfaF0gq56thE_t.jpg"
},
{
"url": "https://f1.pngfuel.com/png/337/556/27/smile-human-cartoon-character-pink-m-happiness-behavior-purple-png-clip-art-thumbnail.png"
},
{
"url": "https://img.favpng.com/8/3/13/clip-art-illustration-dance-hula-vector-graphics-png-favpng-jPQ56XsgP7Gi2cMLzhxbFApeA_t.jpg"
},
{
"url": "https://img.favpng.com/16/4/23/clip-art-pre-school-illustration-human-behavior-png-favpng-VEBwHZmyxNijn756GP0YUW6BV_t.jpg"
},
{
"url": "https://cdn.imgbin.com/11/5/20/imgbin-blond-girl-woman-blonde-haired-s-vPKgK1yAr3Z1vhLcyBms7tCL0.jpg"
},
{
"url": "https://thumbnail.imgbin.com/3/13/22/imgbin-homo-sapiens-woman-cartoon-gipsy-nCHD5ktXjAryu06DKrqc4cZmR_t.jpg"
},
{
"url": "http://clipartmag.com/images/shocked-clipart-1.jpg"
},
{
"url": "https://i.pinimg.com/236x/10/89/7a/10897abb60aeafad127c8e80c5732001--doodle-kids-muslim-fashion.jpg"
},
{
"url": "https://art.pixilart.com/thumb/0721b071d8ec62a.png"
},
{
"url": "https://img.favpng.com/0/4/24/detective-cartoon-png-favpng-7t1LwLmVSieejTCmm39qDLgQV_t.jpg"
}
],
"webEntities": [
{
"description": "Hinata Hyuga",
"entityId": "/m/04b5nh",
"score": 0.32295
},
{
"description": "Naruto Uzumaki",
"entityId": "/m/02sp46",
"score": 0.31364998
},
{
"description": "Sakura Haruno",
"entityId": "/m/037b28",
"score": 0.30705
},
{
"description": "Naruto",
"entityId": "/m/01hryy",
"score": 0.2758
},
{
"description": "Illustration",
"entityId": "/m/01kr8f",
"score": 0.2729761
},
{
"description": "Cartoon",
"entityId": "/m/0215n",
"score": 0.25171754
},
{
"description": "TikTok",
"entityId": "/g/11f555cn8l",
"score": 0.2241
},
{
"description": "Anime",
"entityId": "/m/0jxy",
"score": 0.207
},
{
"description": "Text",
"entityId": "/m/07s6nbt",
"score": 0.19760928
},
{
"description": "Music video",
"entityId": "/m/0mdxd",
"score": 0.187
},
{
"description": "Dragon Ball",
"entityId": "/m/0k2dm",
"score": 0.1776
},
{
"description": "Character",
"entityId": "/m/0k5v2",
"score": 0.1657
},
{
"description": "Fiction",
"entityId": "/m/02xlf",
"score": 0.1247
}
]
}
}The image above lists the terms under webDetection -> webEntities.
However, there's no one size fits all. Those terms are irrelevant in mickey's skin), whose tags, albeit loose, should be the ones under localizedObjectAnnotations or labelAnnotations. It would need human moderation to provide real value.
Of course, if you have no tags whatsoever, adding some groups from the response without human moderation would be better than nothing.
Other services from big players you can look at:
Upvotes: 1
Related Questions
- How to the get the Wordpress Media Search to include tags?
- How to implement multiple tag search across my image gallery with JavaScript?
- Tag images in the image itself? HOW-TO
- Finding similar images via tags
- Adding tags to images in Wordpress
- Photo Tagging in Javascript
- Rails: Search all images that are tagged with a certain tag
- Tagable Image In Browser
- Image tagging on website (like Facebook) where tags are stored in database
- How can I add a search feature to a website that returns images?