sicroatgit / pb-codearchiv-rebirth Goto Github PK

I've added to my PureBASIC Archives repo a reviewed link of this project:

https://github.com/tajmone/purebasic-archives/blob/master/README.md#purebasic-codearchiv-rebirth

I hope that my review is correct — if anything needs amending, just let me know.

I would like to know something more about PureBasic-CodeArchiv-Rebirth:

• What license will it be released under? Same as found on PureArea?
• Will it contain PureBASIC 5 codes only?
• Is the repo meant for building a website?
• What is the current state of the project?

Thanks

Hi @SicroAtGit, I have a proposal: I would like to write an app that can generate the required HTML pages to create a website for PureBasic-CodeArchiv-Rebirth (via GitHub Pages project's website). I would take care of all the coding, so it won't burden you any further, and when it's ready — and if you like it — you could then merge it in.

Here is a general sketch of what I have in mind, and I'd like to create a system that doesn't add a burder in the project maintainance, but actually slims it down.

HTML Files and Website Structure

• Set GitHub Pages to publish from master branch, so that the actual webpages are HTML files inside the repo, alongside with the code files.
• Have single index.html per folder, containing auto-generated resume cards for all the source files inside that folder (and navigation links).
• Have a single assets folder in the root, for CSS stylesheets and other shared assets, to prevent pollution of the project with lots of unrelated files.

Contents from Source Comments

The app should generated a resume card for each souce code file or project, by parsing the comments in the file. There would be very little changes (if any) to comments as they are now: the app's parser should be smart to detect what is what by looking for "<key>: <value>" structures. At the most, for special cases like multiline verbatim blocks (as in a license text), a single unobstrusive character might be added after the comment delimiter in order to allow the parser to know if that comment line is significant. For example:

;| The pipe after ";" could be a non-intrusive special char for the parser.
;{| Between the comment delimiter and the special char there might be a space
; | or one of the special PB comments chars "{ } -", so that code folding and
; | indexing might be preserved.
;}
;@ Different chars can be used to tell the parser what is what.

Special needs could be handled by machine generated comments at the end of the file.

Because of the "<key>: <value>" structure, the current comments system is already simple to parse, and intuitive to use for coders who want to add some code :

;   Description: Threadsafe FIFO-BufferQueue
;            OS: Mac, Windows, Linux
; English-Forum: 
;  French-Forum: 
;  German-Forum: http://www.purebasic.fr/german/viewtopic.php?f=8&t=27824
; -----------------------------------------------------------------------------

Oviously, the new system will impose strict rules on the entries structure and naming convention, but this is inline with the project's goals anyhow.

The system should be simple enough that contributors can handle it from within PB IDE, but it would be good to provide a dedicated IDE tool for validating the comments (check that all required keys are present, and that their format is correct).

The HTML Generator App

The app for generating the resume cards and HTML pages will be written in PureBASIC (I can reuse code I've written for the PB Archives CMS I'm working on). If needed, it might rely on some external tools, like pandoc for format conversion — possibly, standalone binary tools only. But I believe that the required HTML will be simple enough to handle it from PB code.

The app could be hosted inside the assets folder, along with sytlesheet and images, to avoid creating more folders. So, we won't be needing separate developers branches: everything needed to generate the HTML pages will be kept in the repo's master branch, without polluting it.

Keep in mind that, beside adding an online website to the PureBasic-CodeArchiv-Rebirth project, this will also make it browsable locally, and simplify the user experience of navigating through its contents.

Furthermore, the same data extracted from the comments and used to build the website pages could also be used (in the future) to build a database of the project's contents. We could then create a UI app that enables to quickly search and browse the project's contents using criterias such as OS compatiblity, categories, dependencies, tags, etc. So there is actually more potential in this idea then just a website.

Now, if you think the ideas is worth a try, I shall be working on it locally, and when a usable prototype is ready for testing I could create a testing ground repository so we can fine tune it.

@SicroAtGit,

could you please add a PayPal 💳 donation button to the project?

This is the only PureBasic project truely alive and offering good resources. At least, when supporters of the project can't find the time to contribute code/time they can at least donate to it — time, time! ⏳ it's always a good in shortage. I wish we all had more time to share.

• https://hc.weebly.com/hc/en-us/articles/205737658-Accept-Donations-with-PayPal
• https://mcahalane.com/how-to-create-a-great-paypal-donate-link/

The donation button is also a good way to measure the pulse of love 💗 for the project. I mean, people can donate even just 5 euros, which is like treating a friend for a coffee ☕, but it still makes a difference, especially in terms of motivation (it also counters the bitterness of critiques 😡, which usually come free, and plentyful ).

I'm thinking about a new format for the code metadata.

The current format is too complicated:

• One file code:

  CodeName[Win].pb, CodeName[Win,Lin].pb, CodeName.pb (all OS)

  Code file header:

  ;   Description: 
  ;            OS: Windows, Linux, Mac
  ; English-Forum: 
  ;  French-Forum: 
  ;  German-Forum: 
  ; -----------------------------------------------------------------------------
  

• More files code:

  CodeDirectoryName[Win], CodeDirectoryName[Lin,Mac], CodeDirectoryName (all OS)

  CodeDirectoryName/CodeInfo.txt:

  ;   Description: 
  ;            OS: Windows, Linux, Mac
  ; English-Forum: 
  ;  French-Forum: 
  ;  German-Forum: 
  ; -----------------------------------------------------------------------------
  

How I imagine the new format at the moment:

• One file code and more files code:

  CodeDirectoryName/PackageInfo.json:

  {
    "packageDescription": "This is an example package",
    "packageAuthors": "Author1, Author2",
    "packageVersion": "1.0 Beta 1",
    "packageUpdateDate": "02.10.2019 16:00",
    "packageLicense": "MIT License",
    "windowsSupport": true,
    "linuxSupport": true,
    "macSupport": true,
    "forumThread_DE": "https://www.purebasic.fr/german/...",
    "forumThread_EN": "https://www.purebasic.fr/english/...",
    "forumThread_FR": "https://www.purebasic.fr/french/...",
    "files": [
      {
        "fileName": "Logo.png",
        "authors": "Author1, Author2",
        "version": "1.0",
        "updateDate": "01.10.2019 10:30",
        "license": "CC-BY-4.0"
      },
      {
        "fileName": "Music.mp3",
        "authors": "Author",
        "version": "2.0",
        "updateDate": "04.09.2019 14:18",
        "license": "CC-BY-4.0"
      }
    ]
  }

As you can see, the new format now also supports metadata for non-code files.

The JSON can easily be read into a structure:

Structure CodeInfoFilesStruc
  fileName$
  authors$
  version$
  updateDate$
  license$
EndStructure

Structure CodeInfoStruc
  packageDescription$
  packageAuthors$
  packageVersion$
  packageUpdateDate$
  packageLicense$
  windowsSupport.i
  linuxSupport.i
  macSupport.i
  forumThread_DE$
  forumThread_EN$
  forumThread_FR$
  List files.CodeInfoFilesStruc()
EndStructure

[...]

ExtractJSONStructure(JSONValue(json), @codeInfo, CodeInfoStruc)

Hi @SicroAtGit,

I was thinking of waht you said, about the fact that the CodeArchiv should only host code that runs with the latest version of PureBasic. I totally agree with this, and I've come to the conclusion the best way to handle release tags would be as follows:

• release tags should matche the PureBasic version, eg: v5.61
• whenever a new version of PB is releases, the current state of master branch is tagged with the previous PB release.

This means that each tag will represent the latest commit for any given version of PB. After a new version of PB has been released (and a project release created) then all the work of checking and adapting the code against the new PureBasic version begins. Any incompatible code is either updated or removed (temporarily or permanently), and so on until the next PB version/repo tag.

The idea is that release tags would allow to quickly switch to any given PB version number and access all the files of the project as they were at the time they were working for that version. This could become useful in the years, for example if I'm looking for some code no longer availabe with new PB versions (for some reason or another), or because I need a specific version (eg: Scintilla was updated and the code is incompatible, ecc.).

It would also make it easier to port old code which was abbandoned.

I had the same problems with my PB Archives project: because of the mixture of different codes it contained, the only release tags that would make sense were either these or arbitrarily making a release a specific points in time (eg: every 1st of January, and create yearly snapshots; or every so many months).

In theory there is total freedom in how tags are used, and one could adopt both systems. But in practice, consistency is best and the only true reference in this context is the PB version for which the codes are granted to work with (which of course, forces to create tags at the end of a PB version cycle, rather than at its beginning).

What's you view on this?

Hi Sicro,

I'm really impressed with your new Lexer, well done!

I've run-tested it and quickly sifted through the code, really interesting.

RegEx for PB Numerals

I only have some reservations about the numerals RegEx of the PB Lexer example:

      regEx$ = "[0-9]+(?:\.[0-9]+(?:e[0-9]+)?)?" + ; Integers and decimal numbers
               "|" +
               "%[01]+" + ; Binary numbers
               "|" +
               "\$[0-9A-F]+" + ; Hexadecimal numbers
               "|" +
               "'.*?'" ; 'a'

The floats RegEx doesn't seem to cover the way PureBasic defines floating point numbers. Although PureBasic handles floats according to IEEE 754, when it comes to the actual notation of floats the PB compiler has a rather unorthodox approach to what constitutes a valid float number notation. This will compile and work fine in PB:

; Who said IEEE 754 can't be fun?

float1 = 1.575e2bananas ; --> Yes, "floating bananas" are valid in PureBasic!
float2 = 1.575e2

; Compile me if you don't believe me!

If float1 = float2
  Debug "Test passed!"
EndIf

It looks like PB Compiler at Lexing time simply eats-up and ignore any letters after the float number (which in a scientific notation could be a d, etc.). Your RegEx doesn't seem to contemplate letter suffixes at all.

Also, having written a few syntax highlighter for PureBasic, during extensive tests I've managed to catch a few edge cases that allowed me to refine the various RegExs, and as a rule of thumb numerical constants should be captured in this order when using a single RegEx with piped alteratives:

1. Hex
2. Binary
3. Float
4. Decimal

This is to avoid some edge cases in which a non-decimal gets parsed as a decimal.

Here's the RegEx I've used for Highlight PB syntax (Boost RegEx engine):

Digits=[[ (?x)
        # ============ HEX ============
        # Pascal style ($FF):

          \$[0-9a-fA-F]+\b

        # ============ BINARY ============
        | %[01]+\b


        # ============ FLOAT ============
        # With decimal point separator:

        | \b[-]?\d+\.\d+(?:[eE][\-\+]?\d+)?[a-zA-Z]*\b

        # Without decimal point separator:
        
        | \b[-]?\d+(?:[eE][\-\+]?\d+)[a-zA-Z]*\b


        # ============ DECIMAL ============
        | (?<!\$)\b\d+\b
 
       ]]

This is the RegEx used in Sublime-PureBasic for floats (proprietary RegEx engine):

  num_flt: |
    (?x)
    \b[-]?\d+
    (
        \.\d+ (?:[eE][\-\+]?\d+)?   # With decimal point separator
    |         (?:[eE][\-\+]?\d+)    # Without decimal point separator
    )[a-zA-Z]*\b

PB Syntaxes References and Tests

Hoping they might be of some use, I'm pasting here some links to PB syntaxes definitions I've written for various highlighters, were you can look at the RegExs I've used to circumvent edge-cases:

• Sublime PureBasic
• Highlight
• highlight.js

I'm also working on a Highlight syntaxes test suite, where I've started to create some PureBasic test files:

• Highlight Test-Suite: PureBasic

And there also the Sublime Text test files for PureBasic syntax:

• Sublime PureBasic tests folder

Neither of them is yet mature, but I'll be adding more tests in the course of time, so it might be worth checking them every now and then. I have created quite a lot of PureBasic test files while working on those syntaxes, but I need to clean them up before I can publish them on these test suites. The edge cases I came across are quite numerous, and they can be very tricky to handle via RegExs.