abdelrahman-badary / c-tokenidentfier Goto Github PK

-TextParser class removes all unnecessary characters like new lines and trims each line of the code with a ";" , and returns an array with each element containing a statement

-TokenIdentifier class takes each statement of code and extracts the tokens out of it then it categories each of the into one of the following :
-keywords .
-operators .
-special characters
-other tokens which is also categorised in the following :
-identifiers .
-string literals .
-int literals .