blacktoppstudios / mezz_foundation Goto Github PK

There is a fastest hash table and a fast reduced memory hash table over here: https://github.com/skarupke/flat_hash_map

We need to check if the boost software license is amenable to our use.

Port over the StringTools namespace of functions from the monolithic repo. Implement Tests for them.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/stringtool.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/stringtool.cpp

The robin hood hash library is fast and suitable for gaming workloads: https://github.com/martinus/robin-hood-hashing

It should be checked for compatibility, licensing and investigated for API integration.

Port over the CountedPtr class and its tests from the monolithic repo.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/countedptr.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/countedptrtests.h

Currently all the functors and drains check that ranges use cend and cbegin and check for these. This is a reasonable default because these are guaranteed to be const. Most containers provide a const and non-const version of begin and end, and the Functors and Drains should be able to use those, but presently just static assert for the c versions. These should both be checked and should use whichever is available.

Create a new, and hopefully simple serialization system with a distinct concept of a frontend API for classes to call and backends to emit serialized stuff.

Features and goals:

• The frontend should supply some kind of Node class that can be assembled into hierarchies.
• Serialization Nodes should be able to store at least a list of names and values.
• Serialization Nodes should intelligently collapse to JSON or XML.
• Serialization Nodes should be able be store metadate as a list of names and values.
• Serialization Nodes should be queriable for metadata. Serialization Node Queries (SNQ)
  • SNQs should be possible against a single node.
  • SNQs should be possible against a node and all its parents.
  • SNQs should be possible against a node and all its children.
• It should be possible to remove the XML backend with a CMake switch.
• The Serialization Frontend needs to provide some API for backends to implement.
• An XML backend should included - https://github.com/zeux/pugixml
• An JSON backend should included - https://sourceforge.net/projects/rapidxml/files/latest/download
• The included backends should be possible to remove/disable with CMake options.
• Test serializers with an instance of the Version class.

At the initial time of porting std::exceptions were used in the Base64 namespace because Mezzanine::Exception wasn't yet ported to Mezz_Foundation. Once the exception framework is moved over the exceptions raised by methods in the Base64 namespace should be replaced with potentially more descriptive Mezzanine Exceptions.

Edit: This now applies to a handful of classes in the Foundation. Mostly std::runtime_errors were used in place of Mezz exceptions and they should be properly converted once Mezz exceptions are ready.

In StringTools.h there is a function for dragon/camelcasing a string. 2 actually, one mutably modifies a range and another immutably copies.

Functions like this would be easy to make for other common programmer styles, and I would have used them a few times:

dragon case/pascal case - Every Word Is Upper Case.

camel case - every Word But The First Is Upper Case.

snake case - all_lower_case_with_underscores

kebab case - all-lower-case-with-dashes

screaming dragon or screaming snake case - ALL_UPPER_CASE_WITH_UNDERSCORES

We could write our own or use something like: https://github.com/cameron314/concurrentqueue

Currently we have the SortedVector, which is a vector of objects sorted by a predicate. It only stores the one value_type, which is good for smaller object types. However for larger object types where constructing heavier weight objects may not be the best idea it would be ideal to have an associative version of the SortedVector instead. This would amount to being a SortedVector of pairs that store the key and value separately.

Additionally, having a flat map that has a binary tree to keep insert times low may be appropriate based on the use case.

We should try to add a valgrind test to as many of the CI test builds and possible.

A timeout of some kind should added to builds. In the event something locks up or otherwise doesn't work right the Jenkins cluster shouldn't spin forever. Here is a sample timeout line that should be good to go in the options section of the pipeline in the Jenkinsfile. 10 minutes seems like a reasonable default. Higher or lower might be reasonable.

timeout(time: 600, unit: 'SECONDS')

The whole thing might look like:

    options {
        buildDiscarder(logRotator(numToKeepStr:'30'))
        timeout(time: 600, unit: 'SECONDS')
    }

The GSL has facilities for tracking non-owning pointers and preventing deletion. It also has a ranges implementation and some other minor tools that should be useful. This needs to be considered for inclusion in this package.

https://github.com/Microsoft/GSL

All of the Functors and most of the Drains have modes that accept predicates and return functions. It should be possible to adjust these to do the same when passed containers with data. Here is an example of what that might look like from the FunctionalToolsTests.h

const auto SelectFromOneToTen = Select(OneToTen);
TEST("SelectAutoCurriedOnData<Implicit>(OneToTen)", OneToTenOnlyEven == SelectFromOneToTen(IsEven))

Some Unicode text files have a Byte Order Mark at the beginning of the file. This is usually 4 bytes that indicate if the file was made on a big or little endian machines. The JagatiIndex.cmake file has one that can be used as an Example. Having at least 4 functions on the StringTools seems like a good idea: IsBigEndianBOM(String), isLittleEndingBOM(String), IsBom(String), IsBOMDigit(Character).

Currently we are comparing our own implementation of Base64 encoding and the original implementation it is based on in the same file as the rest of the Base64 tests. As it is a benchmark, it should be separated into it's own Benchmark file.

Additionally, we should switch to using the newer MicroBenchmark utilities in Mezz_Test where applicable.

The old Mezzanine has support for automatically creating exceptions.

https://github.com/BlackToppStudios/Mezzanine/tree/master/Mezzanine/libincludes/common/exceptions

The C++ and Macro part should go into the foundation along with the basic set of exception classes that are intended to be thrown and inherited from the rest of the Mezzanine Packages.

The colony is a container optimized for delete speed. Many games have hard to predict memory use, and often create and destroy many items based on player interaction.

There is a library that implements this already and it should be researched: https://github.com/mattreecebentley/plf_colony

Consider the performance ramifications of strings, HashedString and rope classes. One example of a rop class is the SuperString: https://www.boutglay.com/SuperString/ . It is MIT license but doesn't match the std::string API at all.

Consider adding a UTF-8 compatible string class, Once this is done do away with the unicode repo: https://github.com/BlackToppStudios/UTF8Conv

Consider the CopperSpice CsString, it is BSD licensed and appears to work: https://github.com/copperspice/cs_string

There is a video detailing it here: https://www.youtube.com/watch?v=ysh2B6ZgNXk&index=21&list=PLHTh1InhhwT6bwIpRk0ZbCA0N2p1taxd6

Port over the Trie class and its tests from the monolithic repo.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/trie.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/trietests.h

Given we are stepping more and more into implementing our own containers, we can benefit from creating a test suite to explicitly measure the strengths and weaknesses of each of our containers. For the most part this can be made to be generic, making calls on an stl container interface.

ManagedArray has an initializer list constructor, but would be more complete if it could also accept a range in the form of iterators for insertion.

At the top of the Base64Tests file we have a collection of functions meant to be the basis for benchmarking against our own code. We aren't responsible for this code, nor do we want to alter it. It's there to compare. So any and every possible raised warning is simply not our issue (unless it somehow crashes the tests which it's not at the time of this writing).

It'll be more future proof and convey intent better to suppress all warnings there, rather than the current list that was found by trial and error. Additionally, the standard includes in DataTypes.h could benefit from this.

This is currently not possible because we are compiling for C++14, which does not support inline variables. When we upgrade to C++17 we should make is_static_any_v and is_static_any_decayed_v inline.

When we make a PR two builds are started in our CI. One at the HEAD of the PR branch and one at a new commit with master merged. The Jenkins system is interleaving these builds in an attempt to optimize performance.

However, our Jenkinsfile is setup so that each stage must complete before the next begins. This is causing artificial slowdowns by causing machines that could be working to wait.

This relates to Issue #37 this is the performance side effects of that issue.

Don't forget the stuff outside sources files.

Port over the BinaryBuffer class and Base64 methods and their tests from the monolithic repo. These could be separated into their own distinct files.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/binarybuffer.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/binarybuffer.cpp
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/binarytoolstests.h

The Guideline support library helps prevent certain categories of bugs. It is open source and BSD licensed. We should consider and probably include it or implement it.

It should probably be made available in its own namespace, probably a sub-namespace of the Mezzanine namespace.

Here are some relevant links to kick-start research:

https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md

https://github.com/Microsoft/GSL

Port over the HashedString32 class and its tests from the monolithic repo.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/hashedstring.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/hashedstring.cpp
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/hashedstringtests.h

This was originally written pre-c++11, prior to the existence of std::hash. This may or may not be worthwhile for us to use, as the actual hashing algorithm is implementation defined (as long as it meets certain standardized criteria). So we should evaluate if we want the the std implementation, interface, or none. In the event we decide to port our own hashing implementation these files should be used:

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/Internal/murmurhash3.h.cpp
https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/Internal/murmurhash3.cpp

The docs say these packages are needed for the old Mezzanine.

sudo apt-get install g++ gdb codeblocks qtcreator doxygen texlive-font-utils graphviz git-core pkg-config cmake-gui cmake autoconf pkg-config swig ninja-build mercurial

sudo apt-get install libxaw7-dev libxrandr-dev libglu1-mesa-dev libgles1-mesa-dev libpulse-dev

libgles1-mesa-dev and git-core are no longer packages.

This list should be broken out on a per Jagati package basis and in each package a List of packages for each supported OS should be listed.

Ideally all of this would be in the same doxygen group so that it would all show up together in the final docs.

The newest version of the Jagati, still in PR, has many opinionated features that can simplify certain aspects of CMake creation, and just abou tall of that should apply to this very normal package.

There should be a more Streamlined way for this to create the stream automatically. If I tell it to use a stringstream I shouldn't need to manage the stream and the buffer.

When we push to github it launches two builds in our CI. One at the HEAD of the PR branch and one at a new commit with master merged. The Jenkins system is interleaving these builds causing one of the commits to be code used in both builds.

Port over the SortedManagedArray class and its tests from the monolithic repo.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/sortedmanagedarray.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/sortedmanagedarraytests.h

We should try to add a valgrind test to as many of the CI test builds and possible.

Currently, the timings for the CountedPtr is a mess. Particularly on Windows (7 was tested) with it's horribly low precision timer, getting fine grained performance timings can be difficult or impossible.

To make matters worse, in some tests the Internal reference count implementation is beating out raw pointers in performance...which shouldn't be possible. Other times External beats Internal on some platforms. Either the tests should be reconsidered entirely, or in depth bug hunting needs to occur.

This needs Jenkins, Travis, Appveyor and Codecov set up for this.

Port over the SortedVector class and its tests from the monolithic repo.

https://github.com/BlackToppStudios/Mezzanine/blob/master/Mezzanine/src/sortedvector.h
https://github.com/BlackToppStudios/Mezzanine/blob/master/UnitTests/tests/sortedvectortests.h

Add more bitmask operations for the BitfieldTools. There should be an && and an || that return booleans.

 if( (YetAnotherBitField::A || BitFieldInstance)
    { /* Should exucute if BitFieldInstance has the A bit enabled*/ }