ran4 / argus Goto Github PK
View Code? Open in Web Editor NEWFetches public personal information from Wikipedia
Fetches public personal information from Wikipedia
I have written methods of handling some common environments for date of birth (and age). The below should all parse perfectly well:
{{birth date|1993|2|4|df=yes}}
{{dob|1993|2|4|df=yes}}
{{birth date and age|1993|2|4|df=yes}}
{{bda|1993|2|4|df=yes}}
Yet there remains more obscure birth date environments, and death date environments as can be seen here: http://en.wikipedia.org/wiki/Template:Birth,_death_and_age_templates
These need to be taken care of accordingly.
We need to be able to fetch more interesting statistics from the database.
Examples of cool stuff: Things that can be fitted to graphs and tested against hypotheses.
We've done initial testing on parsing attribute values and achieved a rugged functionality. Sooner or later, we are going to have to revert to using mixed upper- and lower-case characters in order to be able to display the information neatly. This will have to be fixed before the attribute values are sent to be parsed.
We are still going to want attribute keys to be lower-case only, though.
It would be better to save it little by little (perhaps a few thousand lines at a time?).
Consider if we should read category information from Wikipedia markup language in xmlwikiparser2.py.
This stuff should be presentable. Currently, the project looks like a mess. We could start by putting things in separate subdirectories - readme files and similar in the root directory, code in one folder, input in one folder, output in one (or the two latter in the same), and so on.
We don't want pictures in our attribute values. We can identify pictures by having certain file extensions, but we must remove them in a smart way so that we might eventually still salvage the rest of the attribute value, if there is more than just an image.
Examples of what we want to remove:
[[File:Andrei Tarkovsky.jpg|240px]]
Jacques-Louis David - The Emperor Napoleon in His Study at the Tuileries - Google Art Project.jpg
One attribute value might hold several lists. Empirically, this has been shown to occur with a significant frequency. We are going to need to handle this either by splitting up these values before we send them to be parsed, or at parse time.
Examples of lists ending and beginning with tags:
{{plainlist}}
{{endplainlist}}
How do these store attributes?
What do we do with these environments when there is no end tag (apparently displays OK in Wikipedia, seen for example in the Abraham Lincoln article)
The Wikipedia list environment "plainlist" can apparently also be invoked as "plain list", and so on. We need to find thorough documentation on this matter - apparently the template articles on Wikipedia are insufficient.
Alternatively, we can find these by trial and error.
The "convert" environment
Example: {{convert|550|ft|m|0}}
The convert environment displays and converts values in different metric and imperial units. We need to parse all cases which might appear in biographical infoboxes.
"oldstyledate" environment
Displays date both in old and new style.
"army" environment
Displays flag and name of army.
"post-nominals" environment
Displays orders from British Empire.
"linktext" environment
Displays list of links.
"air force" environment
Displays flag and name of air force.
"flag" environment
Displays flag and country name.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.