anvaka / map-of-reddit Goto Github PK

Interactive map of reddit

Home Page: https://anvaka.github.io/map-of-reddit/

License: MIT License

JavaScript 65.74% Shell 0.17% HTML 1.93% Vue 31.88% Stylus 0.28%

map-of-reddit's Introduction

Map of reddit

This code renders a map of reddit. Each dot on the map is subreddit. Two dots within the same cluster are usually close to each other if multiple users frequently leave comments on both subreddits.

How is it made?

I processed 176,178,986 unique comments that redditors left in years 2020 - 2021 and computed Jaccard Similarity between subreddits.

Then I treated relationships between subreddits as a graph clustering problem. Once clusters are computed, I treated them as a graph layout problem and created an SVG file. Once SVG file was created I treated it as a dual problem of WebGL rendering and streaming SVG parsing, and created my own webgl renderer and streaming svg parser.

Along the way I strived for simplicity. I wanted to let people edit the SVG file, correct it, and contribute to the development of the map. And I'm happy to say that it is possible to edit the map via anvaka/map-of-reddit-data.

There is a lot of things that I'd love to improve. But I want to know that this effort is not in vain. If you enjoy the work - please consider supporting it on patreon or github. Please share your suggestions and comments with me on twitter.

Some notes

There is no single "right" way to cluster reddit. Communities are very rich and they interact with each other on multiple dimensions. Please don't treat proximity of some countries or even subreddits within the same country as a strong proof of relationship - they could be just an artifact of my clustering/layout algorithm.
It was very hard to develop the website as frequently I got side-tracked by all the amazing content people produce on reddit.
The algorithm decided to group multi-millionaire subreddits in one cluster. One possible way to de-noise such cluster is to click on "Show Related" from the sidebar. This gives focused view on the first and second degree connections.

Running code locally

First - you need to clone/fork this repository and then:

npm install

Now you are ready to serve the website locally:

npm start

To compile a production build:

npm run build

Customize vue configuration

See Configuration Reference.

License

The code in this repository is licensed under MIT license.

map-of-reddit's People

Contributors

Stargazers

Watchers

map-of-reddit's Issues

Feature Suggestion: Label Color Regions

Label each color cluster so it's clear what's in each one. There's too many to click through and figure out one by one. If it's too much work (or too subjective) to manually determine all of them, then you could just label them each by the biggest subreddit (or 2-3) contained within. You could write the label off to the side in some of the empty space in the region or make it pop up on hover/click in the empty space or something.

Bug: Problematic zoom on very small subreddits

I found a 0 member subreddit here. The bubble is very small (so small that I first thought the line that led me to it was ending in nothing) and the text is unreadable. When you search for the subreddit, you get to this view, which bypasses the minimum z=50 that I have otherwise noticed. For me the text also looks quite funky:

Cannot build

I cannot for the world of me build the project. I tried node 14, 16, and 18, and all of them fail with the same error. I tried removing package-lock.json for each version as well. I tried building the project in Docker container as well, and it fails identically.

I would like to at least Google for an error, but I cannot even understand from the error what failed because there are so many things.

This is the Dockerfile that I was using

FROM node:16
COPY . .
RUN npm install

Reproduction:

Clone repo
npm install
Fails with error

STEP 1/3: FROM node:16
Resolved "node" as an alias (/etc/containers/registries.conf.d/shortnames.conf)
Trying to pull docker.io/library/node:16...

Getting image source signatures
Copying blob ad6a3fa7db3b done  
Copying blob e97ef50ee5a8 done  
Copying blob dfb1477a1a0e done  
Copying blob 838447eff6a7 done  
Copying blob b2404786f3fe done  
Copying blob 61f659024e30 done  
Copying blob c78b98c706a4 done  
Copying blob db8137892335 done  
Copying blob 5704b7ac3e4d done  
Copying config b22f8aab05 done  
Writing manifest to image destination
Storing signatures
STEP 2/3: COPY . .
--> 7e1e01aa0b3
STEP 3/3: RUN npm install
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: 3.x is no longer supported
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This loader has been deprecated. Please use eslint-webpack-plugin
npm WARN deprecated [email protected]: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: Switch to 'npm install joi'
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: Moved to 'npm install @sideway/address'
npm WARN deprecated [email protected]: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated [email protected]: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm notice 
npm notice New major version of npm available! 8.19.3 -> 9.5.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.5.0>
npm notice Run `npm install -g [email protected]` to update!
npm notice 
npm ERR! code 2
npm ERR! git dep preparation failed
npm ERR! command /usr/local/bin/node /usr/local/lib/node_modules/npm/bin/npm-cli.js install --force --cache=/root/.npm --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit --include=dev --include=peer --include=optional --no-package-lock-only --no-dry-run
npm ERR! > [email protected] prepare
npm ERR! > npm run build
npm ERR! 
npm ERR! 
npm ERR! > [email protected] build
npm ERR! > del dist && npm run update-license-version && npm run build-umd && npm run build-esm && npm run build-cjs && npm run build-dts && node ./utils/build.js
npm ERR! 
npm ERR! 
npm ERR! > [email protected] update-license-version
npm ERR! > node utils/update-license-version.js
npm ERR! 
npm ERR! 
npm ERR! > [email protected] build-umd
npm ERR! > rollup -c
npm ERR! 
npm ERR! 
npm ERR! Computed sizes of "dist/gl-matrix-min.js" with "umd" format
npm ERR!   bundler parsing size: 198,141 B
npm ERR!   browser parsing size (minified with terser): 50,493 B
npm ERR!   download size (minified and gzipped): 12,805 B
npm ERR! 
npm ERR! 
npm ERR! > [email protected] build-esm
npm ERR! > cross-env BABEL_ENV=esm babel src -d dist/esm
npm ERR! 
npm ERR! Successfully compiled 11 files with Babel (519ms).
npm ERR! 
npm ERR! > [email protected] build-cjs
npm ERR! > babel src -d dist/cjs
npm ERR! 
npm ERR! Successfully compiled 11 files with Babel (666ms).
npm ERR! 
npm ERR! > [email protected] build-dts
npm ERR! > tsc --allowJs --declaration --emitDeclarationOnly --module amd --outFile ./dist/index.d.ts ./src/index.js ./src/types.d.ts && node ./utils/bundle-dts.js && tsc --noEmit ./dist/index.d.ts
npm ERR! 
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,42): error TS1005: ',' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,44): error TS1068: Unexpected token. A constructor, method, accessor, or property was expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,57): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,65): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,66): error TS1109: Expression expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1485,67): error TS1109: Expression expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1490,17): error TS1005: ',' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1490,26): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1494,17): error TS1005: ',' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1494,26): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1498,15): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1498,41): error TS1005: '(' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1504,17): error TS1005: ',' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1504,32): error TS1005: ',' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1504,41): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1504,47): error TS1109: Expression expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1508,17): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1508,43): error TS1005: '(' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1512,26): error TS1005: ';' expected.
npm ERR! node_modules/@types/node/ts4.8/util.d.ts(1514,1): error TS1128: Declaration or statement expected.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN ERESOLVE overriding peer dependency
npm ERR! npm WARN While resolving: [email protected]
npm ERR! npm WARN Found: [email protected]
npm ERR! npm WARN node_modules/rollup
npm ERR! npm WARN   dev rollup@"^2.3.2" from the root project
npm ERR! npm WARN   3 more (rollup-plugin-babel, rollup-plugin-terser, @rollup/plugin-replace)
npm ERR! npm WARN 
npm ERR! npm WARN Could not resolve dependency:
npm ERR! npm WARN peer rollup@"^1.20.0" from [email protected]
npm ERR! npm WARN node_modules/rollup-plugin-size-snapshot
npm ERR! npm WARN   dev rollup-plugin-size-snapshot@"^0.11.0" from the root project
npm ERR! npm WARN 
npm ERR! npm WARN Conflicting peer dependency: [email protected]
npm ERR! npm WARN node_modules/rollup
npm ERR! npm WARN   peer rollup@"^1.20.0" from [email protected]
npm ERR! npm WARN   node_modules/rollup-plugin-size-snapshot
npm ERR! npm WARN     dev rollup-plugin-size-snapshot@"^0.11.0" from the root project
npm ERR! npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
npm ERR! npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated
npm ERR! npm WARN deprecated [email protected]: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm ERR! npm WARN deprecated [email protected]: Please use @jridgewell/sourcemap-codec instead
npm ERR! npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-terser
npm ERR! npm WARN deprecated [email protected]: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-babel.
npm ERR! npm WARN deprecated [email protected]: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm ERR! npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm ERR! npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm ERR! npm WARN deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! 
npm ERR! ./src/index.js → dist/gl-matrix.js...
npm ERR! created dist/gl-matrix.js in 974ms
npm ERR! 
npm ERR! ./src/index.js → dist/gl-matrix-min.js...
npm ERR! created dist/gl-matrix-min.js in 1.5s
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm ERR! code 2
npm ERR! npm ERR! path /root/.npm/_cacache/tmp/git-clonevP2FUC
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command sh -c -- npm run build
npm ERR! 
npm ERR! npm ERR! A complete log of this run can be found in:
npm ERR! npm ERR!     /root/.npm/_logs/2023-02-17T15_04_13_403Z-debug-0.log

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2023-02-17T15_04_08_650Z-debug-0.log
Error: error building at STEP "RUN npm install": error while running runtime: exit status 2

Citing the "Map of Reddit" Data for Academic Research

I'm currently working on a thesis focused on Reddit and wish to reference the "Map of Reddit" data.
(https://github.com/anvaka/map-of-reddit-data)

Could you guide me on the proper way to cite this resource?

Feature Suggestion: List Connected Subreddits on Click

When user clicks on a subreddit, pop up a list somewhere off to the side of all the subreddits whose connections to it light up. Would be much more convenient to see them all in one place than zooming in and out and panning around to see all of them.

Feature suggestion : Find connections between a group of subreddits

We know that the combination of ideas can be very fertile for innovation or inspiration (artistic, scientific research, etc...).

E.g. : In science, the researchers that invented simulated annealing (a metaheuristic) took inspiration from metallurgy ! Biomimetics could be another example. In art, think about a Bach prelude played with jazz idioms, etc...

In a similar way, I think your map could greatly benefit from the following feature :

Select two or more subreddits.
Show the connections between the chosen subreddits, possibly ranking them by distance (the notion of distance could be discussed).

Thanks for considering this idea !

Feature Suggestion: Add option to use old.reddit.com

This could just be placed below the text in the infobox as a on/off switch. It should change all the links to subreddits and theads to use https://old.reddit.com instead of https://reddit.com.

Error: WebGL is not enabled

Hey! I'm getting this error. I thought I'd report in case you didn't know.

"WebGL is not enabled :(
This website needs WebGL to render a map of reddit.

You can try another browser. If the problem persists - very likely your video card isn't supported."

is it possible to map quarantined subs?

thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.