Giter VIP home page Giter VIP logo

emoji-data's Introduction

emoji-data - Easy to consume Emoji data and images

NPM version NPM downloads FOSSA Status

This project provides easy-to-parse data about emoji, along with a spritesheet-style images for use on the web.

The current version supports Emoji version 15.1 (Sept 2023)

You can see a catalog of the emoji data here: http://projects.iamcal.com/emoji-data/table.htm

Installation

The git repo is pretty big (almost 4GB), but contains everything. If you want to use npm, you can:

npm install emoji-datasource

This will only install the 32px full-fidelity spritesheets (with fallback images). If you want different size sheets (16, 20 or 64px), quantized sheets (128 or 256 color), non-fallback (clean) sheets, or the individual images (at 64px) then you'll need to install additional npm modules:

npm install emoji-datasource-apple
npm install emoji-datasource-google
npm install emoji-datasource-twitter
npm install emoji-datasource-facebook

You can also use it without downloading via jsDelivr CDN (different sizes here).

Using the data

The file you want is emoji.json. It contains an array of entries for emoji that look like this:

[
    {
        "name": "WHITE UP POINTING INDEX",
        "unified": "261D-FE0F",
        "non_qualified": "261D",
        "docomo": null,
        "au": "E4F6",
        "softbank": "E00F",
        "google": "FEB98",
        "image": "261d.png",
        "sheet_x": 1,
        "sheet_y": 2,
        "short_name": "point_up",
        "short_names": [
            "point_up"
        ],
        "text": null,
        "texts": null,
        "category": "People & Body",
        "subcategory": "hand-single-finger",
        "sort_order": 170,
        "added_in": "1.4",
        "has_img_apple": true,
        "has_img_google": true,
        "has_img_twitter": true,
        "has_img_facebook": false,
        "skin_variations": {
            "1F3FB": {
                "unified": "261D-1F3FB",
                "image": "261d-1f3fb.png",
                "sheet_x": 1,
                "sheet_y": 3,
                "added_in": "6.0",
                "has_img_apple": true,
                "has_img_google": false,
                "has_img_twitter": false,
                "has_img_facebook": false,
            }
            ...
            "1F3FB-1F3FC": {
                ...
            }
        },
        "obsoletes": "ABCD-1234",
        "obsoleted_by": "5678-90EF"
    },
    ...
]

The meaning of each field is as follows:

Fields Description
name The offical Unicode name, in SHOUTY UPPERCASE.
unified The Unicode codepoint, as 4-5 hex digits. Where an emoji needs 2 or more codepoints, they are specified like 1F1EA-1F1F8. For emoji that need to specifiy a variation selector (-FE0F), that is included here.
non_qualified For emoji that also have usage without a variation selector, that version is included here (otherwise is null).
docomo, au,
softbank, google
The legacy Unicode codepoints used by various mobile vendors.
image The name of the image file.
sheet_x, sheet_y The position of the image in the spritesheets.
short_name The commonly-agreed upon short name for the image, as supported in campfire, github etc via the :colon-syntax:
short_names An array of all the known short names.
text An ASCII version of the emoji (e.g. :)), or null where none exists.
texts An array of ASCII emoji that should convert into this emoji. Each ASCII emoji will only appear against a single emoji entry.
category, subcategory Category and sub-category group names.
sort_order Global sorting index for all emoji, based on Unicode CLDR ordering.
added_in Emoji version in which this codepoint/sequence was added (previously Unicode version).
has_img_* A flag for whether the given image set has an image (named by the image prop) available.
skin_variations For emoji with multiple skin tone variations, a list of alternative glyphs, keyed by the skin tone. For emoji that support multiple skin tones within a single emoji, each skin tone is separated by a dash character.
obsoletes, obsoleted_by Emoji that are no longer used, in preference of gendered versions.

Understanding the spritesheets

For each image set (Apple, Google, etc) we generate several different "sprite sheets" - large images of all emoji stitched together.

Every emoji image in the sheet has a 1 pixel transparent border around it, so the 64px sheet is really made up of 66px squares, while the 16px sheet is really made up of 18px squares, etc. You can find the position of any given image on a sheet using the sheet_x and sheet_y properties, as follows:

x = (sheet_x * (sheet_size + 2)) + 1;
y = (sheet_y * (sheet_size + 2)) + 1;

Inside the Git repo you'll find some sheets in the root directory and some in the sheets-indexed-128, sheets-indexed-256 and sheets-clean directories. In the NPM packages, you'll find them under the img/{$set}/sheets* paths. For example:

Git Repo NPM Packages
/sheet_apple_16.png /img/apple/sheets/16.png
/sheets-indexed-128/sheet_apple_16_indexed_128.png /img/apple/sheets-128/16.png
/sheets-clean/sheet_apple_16_clean.png /img/apple/sheets-clean/16.png

In these examples, the image set is from Apple and the images are 16px on a side. The sheets on the top row are 24 bit color, while the sheets in the middle row use an indexed color palette with only 128 colors. This makes the image much smaller, but sacrifices a lot of quality. Both 128 color and 256 color sheets are provided. The sheets on the bottom row do not contain fallbacks for missing images, so the Google sheet only contains Google images (and no Apple fallbacks). This means that some images are replaced with the fallback character (a question mark), but the usage rights are simpler.

Version history

See CHANGES.md

Image Sources

Images are extracted from their sources and this library attempts to track the latest available versions. If you're looking for older versions of Apple or Android images (such as the Hairy Heart) then you'll need to look at previous revisions.

Image Set Source Version Supported Emoji Missing Images
Apple iOS 17.4 beta Emoji 15.1 3
Google Noto Emoji, v2.042 Emoji 15.1 0
Twitter Twemoji (fork), v15.0.3 Emoji 15.0 118
Facebook v9, fetched 2024-02-05 Emoji 15.0 142
  • Apple images, Copyright © Apple Inc., are not licensed for commercial usage.
  • Android/Google/Noto images, are available under the Apache License 2.0.
  • Twitter images are available under the Creative Commons Attribution 4.0 license.
  • Facebook images, © Facebook, Inc., have no clear licensing.

If you use the spritesheet images and are concerned about usage rights, please use the 'clean' versions, which avoid using fallback images for missing emoji (see the spritesheet section above for more details).

Libraries which use this data

emoji-data's People

Contributors

alexmick avatar andersk avatar crissov avatar dannyfeliz avatar ekohilas avatar elronalds avatar envek avatar ericost avatar evertoncunha avatar fossabot avatar iamcal avatar jamesplease avatar joeattardi avatar joshblum avatar juanfran avatar jwheare avatar leplay avatar lukasdrgon avatar maxoumime avatar morozgrafix avatar nerixyz avatar noodlebox avatar oriolbcn avatar rugk avatar simonkberg avatar srcreigh avatar susnux avatar trknhr avatar whrazer avatar xaviershay avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

emoji-data's Issues

Support Android Nougat Noto Color Emoji

I see there is work towards supporting Android Nougat's changes with the Noto Color Emoji under the yifanfwu-update_google_nougat branch. How can I help to get this work completed?

Publish to npm

Hi. It would be great to see this project in npm registry.

Two emoji named "satellite"

The emoij for SATELLITE ANTENNA and SATELLITE both have the short_name satellite.

Emoji at codepoint 1f4e1 should be assigned the short_name satellite_antenna.
Emoji at codepoint 1f6f0 should remain satellite.

Extract apple images using perl

This will allow me to easily ingest new versions of the apple font without having to manually calculate the glyph->name mapping. There are about 1500 currently, so this is essential.

The basics are easy (parsing sbix and cmap tables), but unwinding the ligatures will be pretty tough (morx table).

Outdated images in gemoji

/cc @mroth @javan

This is a gemoji issue really, but only pull requests are accepted on that repo, not issues, and I wanted to see if anyone's looked into this.

♥️♣️♦️♠️

♥️♣️♦️♠️

^ Those changed in iOS7/Mavericks to not have the card background (new unified versions visible in Safari)

Any idea on the best way to regenerate the emoji images so that a pull request can be sent upstream?

There are several other images that got some tweaks, but can't think which ones right now.

You broke the internet!

Hey!
I've seen you have released a new minor version 2.5.x, where you split the emojies to different npm packages.
This is a blessing BUT it's a breaking release, should have been a major update (3.0.0).

Updating my package to this solved the issue, though many people came complaining to me 😭

Unicode 9 support!

Twemoji has been updated to include unicode 9 emoji – is it possible to re-build and include that data now?

New Apple Emojis

Apple just added over 150 new emojis from the Unicode 8.0 standard in iOS 9.1 and El Capitan 10.11.1.

Emoji 4.0+ sequences support

The Unicode standard itself #59 only covers codepoints and their attributes, e.g. Emoji and Emoji_Presentation. All kind of character sequences that shall result in a single emoji glyph rendering are defined in Unicode® Technical Report (soon: Standard) # 51 Unicode Emoji. This includes the following:

  • presentations
    • applicable emoji followed by either Variation Selector 15 (text) or 16 (emoji)
  • flags
    • an Emoji Flag Sequence is a valid pair of Regional Identifier Symbol Letters (RIS), according to CLDR's implementation of ISO 3166-1
    • an Emoji Tag Sequence for ISO 3166-2 code elements (soon)
    • an Emoji ZWJ Sequence (only Rainbow Flag currently)
  • skin tones (Emoji Modifier Sequence)
    • applicable base followed by Fitzpatrick modifier
  • genders and professions (Emoji ZWJ Sequence)
    • profession emoji base followed by gender symbol (Female = Venus or Male = Mars, possibly soon Neutral/Ambiguous/Androgynous = Mercury)
    • human emoji base (Man or Woman, possibly soon Adult) followed by profession symbol

Emoji 4.0 released on 22 November 2016 added several sequences that most implementations now support (including iOS since version 10.2 #69 released in December 2016): emoji-data.txt. It also switched to English CLDR short names (usually in lowercase or with initial caps) that may differ from the immutable character names in Unicode (usually in uppercase or with initial caps), e.g. red heart vs. HEAVY BLACK HEART for ❤️.

Emoji 5.0 will add more sequences around the same time Unicode 10.0 will be released, due in June 2017.

PR #72 changes the Unicode source files accordingly to support Emoji 3.0 and 4.0, but is probably not enough on its own.

Missing flags

There are a bunch more flag glyphs in the 10.10.3 apple font. Once #19 is done, we'll have unicode codepoints for the remaining flags (111 of them!). With those, we can take them into account when building the map, we they'll get included correctly.

Missing emoji

For example: 👩‍❤️‍💋‍👩 (couplekiss_woman_woman), 👨‍❤️‍💋‍👨 (couplekiss_man_man).

Publish your library to npm

Hey,
First of all, great project.

I've been working on a library to display panels of emojies using your data.
As for now, using this library as peer-dependency (because in the future I'll make another libraries like emoji-input and such) and don't won't to include \ pack your data twice.

Peer-dependencies to github pages is not usual, better take it to npm.
Please consider

Optimise image data

The images are currently quite heavy. How about adding some sort compression?

For instance the sheet_google_64.png has a total of 3.44MB. I've run this image through TinyPNG and the result is only 1.00MB. That's a save of 72%!

What do you think?

iOS 10.2 emojis

Would be nice to add the iOS 10.2 emojis (missing the bacon emoji gg)

Duplicated emoji

Thanks for your project!
During the upgrade my emoji library I've found a small bug:

{ char: '👪', name: 'family', x: 14, y: 20 }
{ char: '👨‍👩‍👦', name: 'man-woman-boy', x: 41, y: 9 }

These emojis are completely the same, and family already has man-woman-boy alias

Scan images for missing emoji

There might be emoji we don't have in the catalog, but have images for from one of the sources. Let's scan those to be sure.

[feature] Add age to JSON

Would be cool if you could add the age to the JSON data, i.e. in which unicode version the emoji was introduced (i.e. 🤑 was introduced with Unicode 8.0).

Publish npm

Hi,

Please publish latest version to npm. thanks.

Discrepancies in emoji names?

Hello! I'm not sure if this is an issue with this repo or an issue with emoji in general or what the heck. I am comparing the emoji alias definitions here with gemoji's (https://github.com/github/gemoji/blob/master/db/emoji.json)

and some of them don't match? Eg

:thinking_face: in emoji-data vs :thinking: in gemoji. Or
:face_with_cowboy_hat: in emoji-data vs :cowboy_hat_face: in gemoji

This is causing some funny problems for us because we have an autocomplete on the front end but then parse it using gemoji on the back end.

Any ideas? Feel free to close as a wontfix as needed, I just was confused because I can't find anyone else complaining on the internet about emoji naming discrepancies, and I thought it odd there's not a canonical set of alias rules. (or maybe there are but I couldn't find any!)

(PS - prior to some of the newer emoji releases we didn't have this problem. Like all the classic emoji - :grinning: :grimacing: etc all have the same names on both gemoji and emoji-data)

Padding

Maybe add a tiny bit of padding between the icons in the pre generated 20 and 32 spritesheets (64 works fine pretty much everywhere)? ShoeBox and most other utilities can't single out each icon otherwise (selects blocks of icons upon selection/extraction).

Update emojione

Update emojione sprites to the emojione 2016 Q1 update that was recently made available.

Are there any ios libraries that use this emoji-data set?

This emoji-data, along with js-emoji is working great on the web.

  • always store emojis in db in colon syntax: :smile:
  • use js-emoji to render the emojis easily on any platform
  • (on web) - if users enter unicode themselves via the mac emoji keyboard - use js-emoji to convert the unicode into colon syntax
  • (on native ios app) When a user enters an emoji directly from an ios native application it needs to be translated into the colon syntax. There are a number of swift libraries that do this, but they all seem incomplete and do not seem to use the this emoji-data set. Are there any known swift or objective-c libraries that use this emoji-data set?

Thank you 🙏 😄

Duplicate short name "umbrella"

UMBRELLA (2602) and UMBRELLA WITH RAIN DROPS (2614) both have the short name "umbrella". The second should probably be "umbrella_with_rain_drops".

I see you have plans to check for duplicate short names in general, but I'm happy to make a PR for this if it is helpful.

bug/inconsistency in empty variations encoding

some of the entries in emoji.json have an empty array for the property 'variations' (probably expected behavior), others, for example EARTH GLOBE EUROPE-AFRICA omit the variations property entirely. Not quite sure what's causing this but it makes parsing the file very inconsistent. Ideally standardize on the empty array case for all entries so someone parsing JSON can assume the property will always be present.

Apple emoji license

Hey. Thanks for the useful package!

Do you have any idea how Apple's emojis are licensed? The results of my google searches don't paint a clear picture. I'm wondering if it's allowed to use them in an app (e.g. how Whatsapp provides the same emojis on Android and iOS).

Expectations around variation selector handling

Hi there,

I’m currently implementing some work based upon your emoji data set.

The data set here doesn’t appear to take into consideration variation selector requirements for certain code points. As far as the data here is concerned, U+00AE ® is an emoji just as much as U+00AE-U+FE0F ®️ is.

Despite a “variations” key existing for some emoji, with U+FE0F VARIATION SELECTOR-16 included, this being present does not accurately indicate a character’s default Emoji_Presentation value!

I’d suggest either that this data set should provide the Emoji_Presentation data, or the unified values should include the U+FE0F VARIATION SELECTOR-16 in cases where the character’s Emoji_Presentation is False.

I’ve worked around this limitation by making use of a regular expression which takes variation selectors into account correctly for matching, but this isn’t ideal, as it means my application ultimately has two sources of truth!

For your reference;

Thanks for your work on this invaluable data set! 💜

null values instead of blank strings for codepoints?

Minor point, but currently for values docomo, au, softbank, google a character that is not in the character set is represented with a blank string, e.g. "".

Similar to the text: field, would it make more sense to rather set this to null if not present? I believe most JSON parsing libraries will "do the right thing" with this then?

Missing unicode bytes?

So I'm completely new to the emoji world and trying to get it all worked out :-) We're using this library but finding that we hit an issue where some unicode bytes were missing and thus the emojis weren't found.

For example:
\xF3\xBE\x93\xAC => Russian flag
\xF3\xBE\x93\xA6 => US flag

Hit the problem because they are used in this tweet that came through the app we're using this library in: https://twitter.com/restorationm/status/617365812769353729

I found the missing emoji listed in https://github.com/atilag/skia-b2g-sgs2/blob/master/emoji/gmojiraw.txt.

Is this something intended, just a one off that we have to deal with, or is there another suggestion?

For now, I simply customized my lists to include those unicode bytes aliased to the equivalents already in the library.

Thanks!!
Alan

Add to bower

Hello!

Just wondering if you would be able to publish this repository through bower. Or maybe you have a good reason why you haven't done that already. Any thoughts?

Thanks

Indexed sheets and npm

Is there a reason indexed spritesheets are not a part of the emoji-datasource package? If not, I'll be happy to create PR.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.