Code contains many Python 2to3 transition imports about doconce HOT 2 OPEN

I created a dedicated Python venv and removed these imports and made the required changes.

Do you mean that you just tested Python 3?

What is the advantage of doing this? Removing Python 2 code would certainly make the code clearer. Although Python 2 is not supported, cleaning the code would likely break DocOnce for Python 2 users. Therefore I am not sure this is something we should pursue. Unfortunately I do not know if there are still users on Python 2.

Tests give an indication that DocOnce works, but they do not provide an exhaustive answer. For example, I think there are commands not running because errors due to missing labels/references. Also, it would be great to process the output so that the reference and produced output files (test.r/test.v) are easy to compare. Finally, I have not really looked into the commands that are run to test DocOnce.

from doconce.

Yes, I only tested a Python 3 env.

Benefits

I am not an expert on the topic of coding, dependencies, etc. But personally I see the main advantages of droping Python 2 support as simplifying and cleaning the code base, and allowing the project to move into the future without continually having to bridge Python 2 and 3.

If I understand correctly, most project that are still alive today that experienced the 2to3 issue have moved to Python 3 only (but still have old versions available for those who might need them).

The future of DocOnce - contributors

The more years go by, the less people will be familiar with Python 2 an the Python2to3 issues that existed a few years back. Thus, people might find it a bit more complicated to read the code, find issues and code a fix for it. More importantly, people will be hesitant or not willing to contribute new functionality to DocOnce because they would have to make it compatible with Python 2 and 3.

The more time that passes the more this will become an issue.

Breaking Doconce for Python 2 users

I find it hard to believe there would be many people who actively work with DocOnce in Python 2. They might work with other code and packages that are written in Python 2, but this does not mean they work exclusively on Python 2. Most likely they create a dedicated Python 2.7 environment to run that code and where possible, their day-to-day work is done in Python 3.

As an end-user, and not a contributor, these individuals don't necessarily know (or care) what is happening under-the-hood; the thing they care about is that the output is properly created. If the solution is to have these people start a venv, installing DocOnce and its dependencies, and run their doconce commands, I don't this is asking too much of them. And for those people for whom this is a big ask, they can continue to work with the version they currently have (which is Python 2/3 compatible). More on this next.

Breaking Doconce for Python 2 users - Doconce versions

With regards to breaking DocOnce for Python2 user, is this not the purpose of freezing working versions and pushing them to PyPi and conda-forge?

The last update to the PyPi version of DocOnce was in 2018, and that version is Python2/3 compatible. And people are installing DocOnce using this method, as indicated by the download history. We could update the PyPi version with the current version of DocOnce and announce that it will be the last version that will be DocOnce 2 and 3 Compatible. For anyone still on Python 2, they can pip install that specific version.

I think the same applies to conda (verion there currently works in Python 2 and 3).

** As an aside, it would be interesting to investigate just how many people are installing doconce via conda; this should be doable using some of the tools anaconda provides.

The flip side: Newer version of Python breaking DocOnce

DocOnce is currently stuck on version 3.6 of Python because a change that was made to in Python 3.7 breaks the code. I can see that making DocOnce Python3.7 compatible is not a priority. But at some point it would be nice to focus on increasing compatibility with new aspects of Python. This will become more important when new formats emerge (the next notebook format, or cool dashboard?, or a replacement for ePub?) since these will most definitely be coded in modern Python. Thus, DocOnce might not be able to incorporate these new developments because it is stuck in the past.

The flip side: dependencies breaking DocOnce

As I have highlighted, at least one packages, ptex2tex, did not make the transition to Python 3. Moving forward, will we run into more problems with compatibility? Will some packages change some of the functions, adopt f-string, etc? It is quite possible that the past packages takes care of this. But for how long will it continue to do this?

Importantly, DocOnce has lots of dependencies, and it is a dependency to no other package. Thus, by updating DocOnce to Python 3 only (and possiblty at some point supporting newer version of Python), the worse that could happen is that users may need to 1) specificy the version of DocOnce they want to install from PyPi or conda-forge or 2) create a dedicated Python 3 virtual env to run the newest version of DocOnce.

As highlighted by Mirco, "Sphinx implementation in doconce is an always open-topic, since Sphinx itself changed drastically the wrappers from version to version, which doconce uses." This is a difficult problem. While not a solution to the issue of having to adapt DocOnce to an ever changing Sphinx wrapper, would it help to have releases of DocOnce that have clear dependencies with their versions pinned. Currently, none of the dependencies are pinned.

Who will do the work?

Allesandro, you doing a great job, and Kristian before you. But the number of active contributors to DocOnce seems to be small. Thus, I can see why you might not want to embark on major changes as it might mean considerably more work for you.

It would be nice to have more people actively involved in DocOnce. This would help spread the workload and create a bit of institutional memory of what happened in the past (and why). Also, as it stands, I get he impression there is no one that is familiar with the entire code base, or people who know well large parts (e.g. who is willing to help update/change/refactor the tests?). I can understand that there might be hesitation in making major changes to the code when the tests might not capture breaking changes, and there is not enough help to think through how such changes will effect the code-base.

I am happy to put in some work here to try and get people interested (or re-interested), and I am happy to put in some time planning or documentation some of these things. There is no rush to make any of the changes I propose; the point of this issue is to get people thinking about the future and the work that will need to be done to move forward. If no one is willing to help, unfortunately DocOnce may not change much and will eventually become less and less usable to new Python users (of which there are many!).

Moving forward...

If people agree, a plan might be to work towards a final release of DocOnce that is Python 2/3 compatible. Various open issues could be resolved (were possible), lingering changed implemented (e.g. moving over to the requests library), and eventually the dependencies verified, updated and pinned. Then this version can be published to PyPi and conda-forge, and it could come with a warning that future versions of DocOnce will no longer support Python 2.

Obviously this is my view and opinion. I am happy to have an open discussion on these topics.

from doconce.