ModuleBase::memoize() returns a finalized hash from BPHash, which is stored as a vector of unsigned chars. In practice we've been converting the hashes as strings when we need to use them. We should just bphash::hash_to_string in the memoize function and return a std::string in the first place.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

When writing modules it's common to have something like:

MyModule() {
   submodules_["Submodule Key"] = std::make_shared<submodule_type>();
}

Describe the solution you'd like
A clear and concise description of what you want to happen.

This could be replaced with a function:

MyModule() {
    add_submodule<submodule_type>("Submodule Key");
}

While not a huge stroke savings, I feel like this syntax is cleaner and it encapsulates the actual registration process (making the shared_ptr) in case we change it.

I have modified this issue to purely focus on terminology. Below are descriptions for concepts key to the SDE and choices for names for those concepts. To vote simply click on the options you prefer. If you pick other please comment on what you want.

1. The self-contained object thingies that provide content to the SDE:

2. The thing that stores said self-contained code thingies (X is winner of poll 1):

Results:

1. module
2. module manager

Opening a separate to start a conversation about the points brought up by @ryanmrichard here

I think the property types do belong in SDE. I see the value of removing them and making the SDE generic if you want to expand the SDE user base outside of molecular simulations. But if we limit the scope to molecular science from the beginning, then we can make certain specializations (incl. PropertyTypes, default data) which make the SDE easier to use for people in our fields - and a one-stop shop for developing molecular science Modules.

I am in favor of switching the dependency order with LibChemist to implement things like system partitioning as Modules. Can all of LibChemist be implemented as Modules?

This is just a todo reminder to change the ready function on a module so that it takes an property type and then uses the property type to determine what inputs could still be set.

Similar to #17, we should establish:

• what library we're going to use?
• what API should classes use?

At the moment I'm under the impression the answers are:

1. cereal
2. The save/load API:

class MyClass {
    void save(T& archive) const {...}
    void load(T& archive) {...}
};

// or non-invasively
void save(T& archive, const MyClass& class2serialize) {...}
void load(T& archive, MyClass& class2serialize2) {...}

This issue is thus largely intended to finalize these decisions.

Same as other API issues. If one builds the documentation in PR #11, and then follows the link on the resulting main page entitled "Module Manager API" one will find the relevant discussion.

Describe the bug
The documentation says that a module can have multiple Property Types, but adding more than one satisfies_property_type() to a module's constructor results in a runtime error "Can't set type twice".

Raised by @robertjharrison on NWChemEx-Project/LibMathematician#1 (and also at PNNL meeting). Specifically:

• how does the SDE automate management of memoized objects?
• how does a user mark a memoized object for deletion?
• how to iterate through cached objects?
• object properties (size, lifetime, number of users, etc.)
• save only some returns from a module
• above with distributed objects

Describe the bug

Can't build tests for ctest.

To Reproduce
Turn BUILD_TESTING in CMakeLists.txt to ON
cmake
make
ctest

Expected behavior
Would expect the tests to run.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Right now if you try to pass an argument of the wrong type to run_as you trip a static assert which provides an archaic error message about types not being convertible. It'd be nice if it was more helpful. At the very least it should print something like the n-th argument wasn't convertible. It'd be great if we could actually get the types into the message somehow, but I haven't found a great way to print type information at compile time.

Title says it all.

• ParallelRuntime #1 (moved to #4)
  • Future
  • Promise
  • Device
  • SerialDevice
• PhysicsRuntime
• ChemicalRuntime #5
  • BasisSetExchange
  • Wikipedia
• CalculationState
  • ModuleInfo
  • Options

📝 We need to pick at least one parallel backend and then wrap its API in that of the Device.

Because std::any requires you to get an object back as the type you put it in as, it does not allow you to get the object back as a base class of its original type. So, an exception is encountered when a module that accepts a base class input is provided with a derived class input.

(Note: Ryan advises using overloads 4 and 5 from https://en.cppreference.com/w/cpp/utility/any/any_cast in resolving this problem, to avoid having to catch exceptions.)

Describe the bug
On MacOS the SDE libraries are generated as .dylib . Python 3 cannot find SDE via .dylib, only via .so

To Reproduce
Compile on a Mac with g++7. Run ctest, it will fail.

Expected behavior
cmake should produce an environment that produces .so files.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: MacOS

Additional context
Haven't been able to find a way to convince cmake that it never should generate a dylib.

Right now if you ask the ModuleManager for a module and the MM doesn't have it, the MM throws an exception. That exception is the generic std::out_of_range exception thrown by the std::map under the hood. It would be nice if the MM did the check itself and returned a more useful error message containing the module key that it couldn't find.

Apparently my tongue-in-cheek name for the instance holding the constants has spurred a bit of confusion. In an effort to avoid this I'd like to recommend the instance be renamed. Although boring, maybe physical_data?

Is your feature request related to a problem? Please describe.
Yes. We don't have a way to parallelize things outside of TAMM.

Describe the solution you'd like
Ultimately, I'd like a solution that doesn't require me to do coding 🙂. To this end it's worth noting that there are a bunch of runtimes out there, including some funded by ECP. That said the real question is "do any of those runtimes suit us?". To that end it helps to have a better idea of what we want in a runtime, which is the main focus of this issue.

I personally have found runtimes like Intel's TBB to very easy to use and capable of good performance. Obviously TBB isn't going to get us across Summit though. For those not familiar with TBB it basically gives you functions that loop over a series of lambdas. You can change the queue used for the loop (and possibly how the lambdas are scheduled). TBB promotes asynchronous computing via the use of continuations. I'd like to find/write something that uses similar coding concepts while supporting parallelization beyond threads. Historically, when I make such a suggestion @evaleev and @robertjharrison usually advocate for something more higher-level than this, but I have to admit, I'm not completely sure what this higher-level looks like---just parallel objects like tensors and STL-like containers?

Finally, I'd be remiss if I didn't include @twindus's suggestion from the conversation that prompted this issue, namely a series of get, put, gather, scatter functions that work with own resource descriptions (as I understand it an MPI-like API that goes over MPI as well as OpenMP and possibly other threading/distribution models). Of course such a layer would be immensely useful for writing a TBB-like layer that deals with mulitiple types of parallelism.

Describe alternatives you've considered
Waiting for the compiler to magically parallelize the code.

Additional context
This issue is necromancy on #1 and #4. Those PRs ultimately never got merged (and it's not clear to me that any of the code in them is worth revisiting). I'll leave to the reader to read the discussions on those PRs (it's not too much). The overall punchline of those PRs was that the TAMM team was going to implement the runtime. That said, I am under the impression that this is no longer the case, rather the TAMM team is going to implement TAMM's runtime and the rest of NWX needs to implement/adopt their own. If this is the case it becomes essential that NWX's runtime plays nicely with TAMM's, which is going to be challenging without a thorough discussion.

consider a module that computes the kinetic energy of the electrons. Right now it's of property type AOIntegral<2>, but there's other modules that are also of AOIntegral<2> (such as the electron-nucleus attraction). While building the core Hamiltonian you can imagine looping over AOIntegral<2> modules, but if you only want the kinetic energy there should be a way to specify that. In OOP fashion you'd have the kinetic energy derive from AOIntegral<2>, then a module would say it satisfies the kinetic energy and the SDE could upcast it to AOIntegral<2> if need be.

It is clear that type-erasure is going to play some role in the SDE.
This issue purposes that we write a class SDEAny to do this type-erasing. In addition to the normal type-erasure features of the std::any class (or the Utilities::Any class) we will need to be able to invoke at least several other members:

• void hash(hasher &)
  • const and non-const versions
  • See issue #17 for more details

• - ~~~`T` is a reference to an archive satisfying cereal's API~~~
  - Silly me, can't have virtual templated methods
  

• pybind11::object pythonatize()
  • Returns wrapped object as Python object
  • Likely want const and non-const versions
  • See issue #19 for more details about C++ Python wrapper

Thus SDEAny can be implemented by copy/pasting Utilities::Any and adding support for those members (as I started Utilities::Any for this use-case it probably makes sense to move it here and change the name instead (possibly to SDE::Any)).

Update: #11 includes a purposed API for this class.

For the most part the declaration of a module (and a property type) is just boilerplate. Basically all you need is the name of the resulting class and the declaration can be generated. To reduce the amount of boilerplate we should write a C preprocessor macro that declares a module/property type given the name of the resulting class.

As I write a lot of the UML stuff I've started wondering if it actually makes sense to pass pubchem and bse around inside the ChemistryRuntime class? I think these two objects are likely only of use for making inputs and will get no use as members of the ChemistryRuntime class.

My proposal would be:

• Remove pubchem and bse members from ChemistryRuntime
• Have tests directly call default_molecules and select the molecule they want.
• The way the apply_basis_set function works would have to change
  • New parameter that defaults to result of default_bases?
• load_basis would need tweaked too
  • Return new basis rather than adding to SDE?

Thoughts?

Right now, the script docs/build_docs.py will parse the C++ tests that serve as tutorials and make a .rst file for inclusion with the documentation. We should extend this functionality to cover the Python tests. I recommend rewriting build_docs.py so that it if it finds a Python comment like:

#TUTORIAL
# 
# This is a comment for inclusion in the documentation. It will be dumped *verbatim* into
#  a ``.rst`` file to generate the tutorial
def function():
    return true

it would generate:

This is a comment for inclusion in the documentation. It will be dumped *verbatim* into
a ``.rst`` file to generate the tutorial

.. code-block:: python
    def function():
        return true

Similarly for C++:

//TUTORIAL
// 
// This is a comment for inclusion in the documentation. It will be dumped *verbatim* into
// a ``.rst`` file to generate the tutorial
bool function() { return true; }

it would generate:

This is a comment for inclusion in the documentation. It will be dumped *verbatim* into
a ``.rst`` file to generate the tutorial

.. code-block:: c++
    bool function(){ return true;}

The use of TUTORIAL helps distinguish tutorial comments from normal comments that should be left in. The C++ tests would need some slight tweaking to work with this (I used a different mark-up style).

It would be convenient to be able to specify defaults to certain PropertyType inputs, for instance the origin of the multipole expansion in the electric multipole integrals.

Can follow similar syntax to modules setting default inputs. Will require run_as<PropertyType> to accept fewer arguments than the full set of inputs when the defaults are desired.

Under the hood the call to a module passes in the type-erased inputs and the type-erased submodules; however, the public API of run/run_asonly lets the user set inputs. The original motivation behind this assumed that the ModuleManager would be the one switching submodules; however, at least for
unit testing purposes it is convenient to be able to just pass in the submodule you want.

While refactoring I have introduced the ModuleOutput class. As it stands this class is little more than a
wrapper class that associates a type-erased value with a description. One interesting idea is to also associate with the returned value the accuracy of the value. This would enable algorithms to automatically adjust to the number of significant digits in their input values or the target output accuracy.

After much deliberation I think I'm happy with the module API. If one builds the documentation in #11 and clicks on Module API at the bottom of the main page of the html documentation they can see the discussion. I am currently recommending the API entitled "Acyclic Visitor API".

Similar to issue #20, I'd be interested in feedback, particularly with respect to:

• Use cases. Are any missing?
• Are there other APIs to consider?
• Are there additional advantages/disadvantages?

Right now if a module is not ready to run when it is called you get not so helpful error messages like "module is not ready" and "inputs are not ready". In the former case we should print out why the module is not ready and in the latter we should print out why the input is not ready.

This is not good and will need to be resolved before people use the code. Otherwise we're going to have to pay people to install the code. This reminds me of how the early ada compilers worked (or didn't). I'm guessing there's some N^2 checking going on.

The purpose of this issue is to solicit feedback regarding the purposed Cache API. To see the purposed API, build the documentation associated with #11 and follow the link Cache API on the main page.

Right now we have the load_mm() functions which look like:

mm.add_module(...);
mm.add_module(...);

This article gave me an idea for how to autoregister modules. Basically in SDE we declare some global map std::map<T, ModuleBase*()> factory (T likely typeid(DerivedClass), but maybe have the user set a string). We then define a macro:

#define REGISTER_MODULE(TypeOfModule, key)\
   struct Register ## TypeOfModule {\
       Register ## TypeOfModule(){ factory[key] = [](){ return new TypeOfModule();};}\
   };\
  Register ## TypeOfModule p

which a user puts in their source file like REGISTER_MODULE(MyModule, "DefaultKey");.

Want to be able to do something like:

mm.add_lambda_module<PropertyType>(key, callback);

To have the ModuleManager generate a facade module for us of the specified PropertyType that wraps the provided callback.

PropertyType<T>::wrap_results takes two arguments the results map that it is adding the results to and the results to add to the map. The motivation for taking the results map is that it supports concatenation of multiple property types' results into a single result map; however, when there is only a single property type (or equivalently it's the call to the first of several property types) this leads to code like auto rv = results() to make the initial result map. We should create an overload of wrap_results that does not require a results_map.

Note: this will likely require SFINAE to pull off as the resulting signature will be fully templated.

Describe the bug

Complains about bphash missing. Might be related to cpp on being able to install?

To Reproduce

cmake -H. -Bbuild -DCMAKE_INSTALL_PREFIX=/home/users/wadejong/NWChemEx-Python/install -DCMAKE_PREFIX_PATH=/home/users/wadejong/NWChemEx-Python/install -DCMAKE_C_COMPILER=gcc-8 -DCMAKE_CXX_COMPILER=g++-8 -DCMAKE_Fortran_COMPILER=gfortran-8 -DCPP_GITHUB_TOKEN=e90570937183608ee6df7dc1ca6a58da21b18584
-- The C compiler identification is GNU 8.3.0
-- The CXX compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/gcc-8
-- Check for working C compiler: /usr/bin/gcc-8 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-8
-- Check for working CXX compiler: /usr/bin/g++-8 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find utilities (missing: utilities_DIR)
-- Could NOT find cereal (missing: cereal_DIR)
CMake Warning (dev) at build/_deps/cereal-src/CMakeLists.txt:2 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

The following variable(s) would be set to empty:

PROJECT_VERSION
PROJECT_VERSION_MAJOR
PROJECT_VERSION_MINOR
PROJECT_VERSION_PATCH

This warning is for project developers. Use -Wno-dev to suppress it.

-- Could NOT find bphash (missing: bphash_DIR)
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:

CMAKE_Fortran_COMPILER

-- Build files have been written to: /home/users/wadejong/NWChemEx-Python/SDE/build

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

This issue is focusing on what we should call a concept I'm currently calling "module type".

An example of how to declare and implement a module is probably the best way to motivate the concept.
There exists a class ModuleImpl, which does all the "hooking-up" of modules. To use it, we derive several APIs from it, one for each generic property we want to compute (things like energy, wavefunction, molecular systems). As a more concrete example, let's assume we agree that energy is one of these generic properties (what these properties are is a whole other discussion). Then we could define a class EnergyDeriv like:

template<size_t N> // N being the derivative order N==0 is the energy
class EnergyDeriv : public ModuleImpl<EnergyDeriv, Tensor, const System&>  { 
    virtual Tensor run_(const System& ) =0; // could actually put finite difference here
};

To implement, say, an SCF one would then do:

template<size_t N>
class SCFEnergy : public EnergyDeriv<N> {
    Tensor run_(const System& sys) override{
        // implementation
       return energy_deriv;
    }
};

Algorithms, like geometry optimizations would then simply ask for two modules, one of type EnergyDeriv<0> and one of type EnergyDeriv<1> (modules capable of generating the energy and gradient respectively). The optimizer would never need to know what the actual algorithm is that implements the energy.

At the moment I'm calling classes like SCFEnergy "modules" and classes like EnergyDeriv "module types". The latter is particularly nasty as it leads to sentences like "the type of the module type"...

@keipertk presented me this link which is for the Atomic Simulation Environment. It's quite close to Simulation Development Environment. Also @keipertk's "market testing", i.e., a talk to computational chemists, suggests that it's not clear that the SDE is designed for chemistry.

The overall question is then: do we want to rename the SDE? If so, suggestions? All thoughts are welcome.

• CMSDE: Chemistry and Material Simulation Development Environment
• ASDE: Atomistic Simulation Development Environment
• ASDF: Atomistic Simulation Development Framework
• MSF Molecular Simulation Framework
• MSF Molecular Software Framework
• MSDF Molecular Software Development Framework
• MSSF Molecular Science Software Framework
• MSSF Molecular Simulation Software Framework
• MDE Model Development Environment
• SiDEn: Simulation Development Environment (also Danish for “since”)
• SiDeF: Simulation Development Framework (also a programming language)
• SiSyEn: Simulation Synthesis Environment (also an online art platform)
• MoCoEn: Model Construction Environment (also a mountain in the Andes)
• MoCoF: Model Construction Framework (also a furniture company)
• MoSyEn: Model Synthesis Environment (also a family name)
• MoSyF: Model Synthesis Framework
• SiREn: Simulation Research Environment
• SimPlE: Simulation Platform and Environment
• DES: Development Environment for Simulations (Though there is a DESi neutrino experiment)

Over time it's likely that module collections will contribute modules with the same key. To disambiguate among the keys it would be nice to have namespaces. Like C++, the namespaces should support arbitrary nesting (it should suffice to concatenate the strings together into one namespace under the hood).

This would require modifying the way the ModuleManager stores modules. If we only require namespaces to be specified when there is ambiguity this should not break anything that currently exists.

@wadejong and I are meeting with Wim (the creator of Cppyy) on Wednesday to discuss possibly transitioning to Cppyy. I started this issue to keep a running list of things we need from Cppyy before we
make the switch:

• memory semantics (support for transferring ownership, taking ownership, copying between C++ and Python)
• Inheriting from C++ classes in Python
• Cross-language lambdas

The purpose of this issue is to solicit feed back regarding the purposed Parameters class hierarchy (design of the actual class will be a separate issue once discussion on this issue has culminated). The purposed Parameters class is the one known as "Pythonic" in the documentation (specifically if you build the HTML documentation in #11 it will be the second Parameters API link under Related Pages).

Talking points:

• Are there missing use cases?
• Are there other APIs to explore?
• Are there pro/cons that should be added to the lists?

This issue is for writing the Cache class using the API in the documentation.

The intent is to write the entirety of the SDE in C++ and then to allow the user to optionally compile Python bindings for the SDE. Said bindings would allow the SDE to be directly manipulated from a Python script.

To me the main topics to discuss are:

• Is this still the plan?
• If so, what library for Python bindings?
• Further considerations?
  1. Restricts template usage at public APIs

Based on the one consideration I came up with it's important to think about this now as it does limit the design space for the SDE.

Describe the bug
A clear and concise description of what the bug is.

I'm trying to define an option for the DFJK builder that would be the name of the fitting basis set. However, I'm getting compiler errors when the value of an Option is a string. This may actually be an issue with SDE::Any or it may be caused by the fact that a string literal (e.g. "A string literal") is not strictly the same thing as an std::string. Regardless of the root cause I think it's reasonable to assume users are going to want to use string literals as if they were std::string so we probably need to handle this.

To Reproduce
Steps to reproduce the behavior:

SDE::Parameters params;
params.insert("Description1", SDE::Option("A string literal", "The description"));

Right now run_as always returns an std::tuple of results, even if the module only returns a single value. This means you have to use structured bindings, i.e., auto [x] = mod.run_as<...>(...) to auto-unpack the tuple. For the user's convenience the run_as function should have a if constexpr check on the size of the return. In the event that it's a single value run_as should call std::get<0> on the tuple, unpacking it for the user.

This issue is intended to house discussion about the question in the title.

https://github.com/NWChemEx-Project/SimulationDevelopmentEnvironment/blob/master/reference_data/generate_basis.py
will read a list of files containing basis sets and parse them into C++ structures for use in:
https://github.com/NWChemEx-Project/SimulationDevelopmentEnvironment/blob/master/SDE/Defaults/DefaultBases.cpp

If we come to some conclusions I will add those defaults in this PR: #13

call_submodule is meant to be analogous to run_as (in fact it's just a thin wrapper around it cleaning up the syntax). In order to better help establish this parallel I purpose we change the name of call_submodule to run_submodule_as.

• Change name of ChemistryRuntime
• Change name of file and class (in UML) to ChemistryRuntime
• Fix types of members
• Add missing IsotopeData member
• Add missing members of ChemistryRuntime

It's becoming clear we're going to need hashing capabilities. This issue needs to address the following:

1. what library are we using/writing for hashing?
2. what API must classes implement to be hashable?
   • support const/non-const versions for saving hash
   • how to combine hashes?
3. is hashing going to work with floating point (FP) instances?

FWIW, Pulsar's solutions were:

1. We used @bennybp's hash library located at https://github.com/bennybp/BPHash. Ultimately it was deemed important that the hash library use an algorithm such that: hash collisions are uber rare and that the hashes should be stable between runs (N.B. std::hash is implementation defined and not guaranteed to satisfy this requirement).

2. The Pulsar hashing API was simple. Each class defined a function:

//hash_type was just a string, hashes joined by string concatenation
hash_type hash() const;

3. FP worries turned out to be a non-issue in Pulsar due to:

• restricting FP inputs to relatively constant things like the AO density matrix, MO coefficients, and system
• the values of these FP quantities were always taken from the cache and thus referred to the same instance
• the system usually comes from a string input and should thus result in the same FP representation for multiple runs
• cache entries are saved in full precision, reloaded cache entries thus have same FP representation

This issue revolves around actually writing the Parameters Class using the API included in the documentation.

Is your feature request related to a problem? Please describe.
The SDE is being billed as agnostic to the details of the package; however, it currently contains a lot of empirical data that, one could argue, is package specific.

Describe the solution you'd like
The contents of the Defaults directory should be moved to the NWChemEx repo.

A function for a PropertyType that takes as input the desired output of a lambda function of that PropertyType and returns said lambda function.

function like:

mm.change_submod("key", "callback", "submod key");

where key is the module whose submodule is being changed, callback is the callback point getting the new submodule, and submod key is the module key (as it is known to the MM) to use for the submodule.