#Baby Names!
Fun with the Social Security Administration's baby name data
##Setup
This is a Node.js script, so you should consider downloading Node before attempting to run it.
To download the repo, simply clone it:
git clone https://github.com/TimeMagazine/babynames.git
cd babynames
Then install the dependencies:
npm install
##Data
First, you need to get the raw data from the Social Security Administration. This script will download and unzip it for you with the following command:
./index.js download
###Total babies born each year
There is also a file called extra/totals.json
with data on the total number of babies born (or at least, those issued a SSN) each year, per the SSA. This is useful because the totals are higher than the sum of each name in the name files, which don't include names that occur fewer than five times.
If you want to re-download the data--maybe it's a new year or you suspect there has been a revision--just run ./scripts/total_births.js
, which will scrape the page on the SSA website and overwrite the file in the repo.
###Baby names
The Social Security Administration organizes the baby name data, somewhat inconveniently, as year-by-year text files named yob[year].txt
. The above command extracts those files to a local directory named data/
and then deletes the zip file it downloaded. If you want to keep that zip file for some reason, just pass --cache
to the command.
Once that's done, you can aggregate the data to a per-name basis and store it in a variety of formats:
./index.js store --format=json
First, the script reads every file and stores the data on a per-name basis in memory. For each name, it records both the absolute number of babies with that name in a given year and the percentage of all babies of the same gender with that name. The denominator in that calculation is the gender-specific total number of babies as reported on SSA.gov, NOT the calculated sum of all baby name frequencies (which will be lower than the actual number of children born in the United States, given that the data only counts names that appear at least five times). The years are stored as keys in an object for fast retrieval:
{
"_id": "Lothar-M",
"name": "Lothar",
"gender": "M",
"values": {
"1927": 7,
"1928": 6,
"1929": 10,
"1931": 6,
"1932": 8,
"1935": 8,
"1956": 7,
"1959": 5,
"1964": 6,
"1968": 5
},
"percents": {
"1927": 0.0000062135232896167586,
"1928": 0.000005418551494752584,
"1929": 0.000009301278646775573,
"1931": 0.000005775539435383265,
"1932": 0.000007664935030094451,
"1935": 0.00000768623999707923,
"1956": 0.0000033120134297413128,
"1959": 0.0000023436460007087187,
"1964": 0.000003010349581862443,
"1968": 0.0000028754074452349896
}
}
###Formats
Your choices are:
json
: Each name is stored as an individual JSON file in the/flat/individual/
directory.jsonp
: Each name is stored as an individual JSON-P file in the/flat/individual/
directory. It is wrapped in a callback function namedticallback
by default, which you can override withopts.callback
.csvs
: Each name is stored as an individual CSV file in the/flat/individual/
directory.csv
: All names are packaged into one CSV file and stored in/flat/names.csv/
. This file will be able 30MB if you don't include limiting specifications (below). This preprocessed file is included in this repo.mongodb
: All names are inserted into a MongoDB instance. You are responsible for running a Mongo server atlocalhost:27017
or updating the source to point to your instance. Note: Because this is optional, the mongodb Node module is not listed as a dependency, you you'll need to install it yourself.
##Reducing the size As of 2013, there are 102,691 names that show up in at least one year at least five times. Many users will not be interested in this volume of data. There are several ways to reduce the scope with command line options.
###Limit the years
start
: Don't retrieve years before this year. Ex:--start=1950
. Default is1880
, the first year of the data.end
: Don't retrieve years after this year. Ex:--end=2000
. Default is the present year.
###Exclude uncommon names
min
: Don't include names that don't show up at least this many time in at least one year. Ex:--min=25
. Default is0
.cutoff
: Don't include names that don't show up in at least this many individual years. Ex:--cutoff=50
. Default is0
.
##Analysis
The script comes with several options for basic analysis:
normalize
: Add a third property to each name that is the normalized value for the percentage figures, such that the peak percentage year is 1.peaks
: Find the peak value and year for both raw values and percentsmaxima
: Identify all the local maxima -- points where every value 5 years before and after is lower. Only counts maxima that are at least 25 percent of peak value.pronunications
: See if the name is listed in the CMU Pronouncing Dictionary. Require that younpm install cmudict
manually.dense
: If a name does not appear in a year in the range specified betweenstart
andend
, list that year in the data as0
. Otherwise it is not included at all (a "sparse" format).
##Types
For csv outputs, you can get the data back as either raw numbers of new babies each year with a given name (--type=values
, which is the default) or as a percent (--type=values
). JSON formats return both percents and values.
##Phonemes
You can also pass a special type, --type=phonemes
, to get back a JSON document of phoneme percents for each year for all names. By default, the script examines the first phoneme in each name. You can use --N==TK
to aggregate around the TKth phonemes in the name. Use a negative value to start from the end.
##License
This script is provided free and open-source by Time under the MIT license. If you use it, you are politely encouraged to acknowledge Time and link to this page.
The dictionary file dict/2of12.txt is from the 12 Dicts project, which is in the public domain.