speedyleion / sphinx-c-autodoc Goto Github PK

During sphinx clean-up for 5.0 (specifically this change: sphinx-doc/sphinx@267954e#diff-f2b81eb446ad45461b255667a70a375c8548595c28846185fef603078fc8ab96), the 2nd parameter in prepare_docstring(...) in sphinx/util/docstrings.py was removed.
Consequently, sphinx-c-autodoc throws an exception when calling prepare_docstring:

Traceback (most recent call last):
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/cmd/build.py", line 281, in build_main
app.build(args.force_all, args.filenames)
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/application.py", line 347, in build
self.builder.build_update()
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/builders/init.py", line 310, in build_update
self.build(to_build,
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/builders/init.py", line 326, in build
updated_docnames = set(self.read())
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/builders/init.py", line 433, in read
self._read_serial(docnames)
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/builders/init.py", line 454, in _read_serial
self.read_doc(docname)
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/builders/init.py", line 510, in read_doc
publisher.publish()
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
self.document = self.reader.read(self.source, self.parser,
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/io.py", line 104, in read
self.parse()
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/readers/init.py", line 78, in parse
self.parser.parse(self.input, document)
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/parsers.py", line 78, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 170, in run
results = StateMachineWS.run(self, input_lines, input_offset,
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
context, next_state, result = self.check_line(
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
return method(match, context, next_state)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
self.section(title, source, style, lineno - 1, messages)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
newabsoffset = self.nested_parse(
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
context, next_state, result = self.check_line(
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
return method(match, context, next_state)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
return method(self, expmatch)
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
return self.run_directive(
File "/home/me/venv_test/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
result = directive_instance.run()
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/ext/autodoc/directive.py", line 148, in run
documenter.generate(more_content=self.content)
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx/ext/autodoc/init.py", line 918, in generate
docstrings: List[str] = sum(self.get_doc() or [], [])
File "/home/me/venv_test/lib/python3.8/site-packages/sphinx_c_autodoc/init.py", line 293, in get_doc
return [prepare_docstring(docstring, ignore, tab_width)]
TypeError: prepare_docstring() takes from 1 to 2 positional arguments but 3 were given

I want to start out with saying that this is a great tool and it is making the generation of documentation much better and easier.

My use case for this extension has been to generate documentation automatically from documented source code. The idea being that developers should have to put no effort in beyond commenting their code to get documentation for it. One constraint I have found is that sphinx will emit a warning for duplicate member declarations across different files, which will commonly occur in a c project where the prototypes for functions are declared in header files. If the prototypes are declared within the same file, clang will correctly ignore the prototype, but it will not do this when the prototype is in a separate file. I would suggest adding a feature where prototypes declared within a header file with no documentation should either be skipped completely, or at least made private. This would allow a developer to still use the autocmodule directive to not have to manually define what elements from a header they would want to have to avoid including a prototype.

Below is a simple way of implementing this

class DocumentedFunction(DocumentedObject):
    ...
    def is_public(self) -> bool:
        """
        Functions are public as long as they are not static.
        """
        # Start of changes
        if self.node.location.file.name.endswith(".h") \
                and not self.node.is_definition() \
                and self.doc == "":
            return False
        # End of changes

        if self.node.storage_class == StorageClass.STATIC:
            return False

        return True

This implementation could be expanded to be dependent on a configuration variable being set (I'm not very familiar with writing sphinx extensions and didn't see how to access the environment from this class or I would have included this above, sorry!).

An alternative option to this would be to implement the meta tag features present in sphinx's autodoc to allow the developer to set if something is private within a comment (using :meta private:). While this feature is useful I believe the above feature will be more easily adopted and easier to use by the end developer.

If you would like me to make a PR for this issue just let me know and I can do that. Sorry for all the issues recently! 😆

With #17 it's apparent that the dependencies need to be managed in a more automated fashion:

• #18 to provide a passing CI.
• #19
  • Need to look at python/pip usage in dependabot doesn't seem to be updating It just took longer
• merge dependabot prs
  • For some reason sphinx won't go from 4.3.2 to 5.x with dependabot, even though #24 happened
    I found the issue:
    Because flake8 (4.0.1) depends on importlib-metadata (<4.3) and sphinx (5.3.0) depends on importlib-metadata (>=4.8), flake8 (4.0.1) is incompatible with sphinx (5.3.0). So, because sphinx-c-autodoc depends on both sphinx (5.3.0) and flake8 (4.0.1), version solving failed.
• #28
• add back mypy and pylint to the ci #40 #42
• document deprecation plan. #43

https://www.sphinx-doc.org/en/master/usage/extensions/viewcode.html provides links from the module documentation to a syntax highlighted version of the source code.
Should implement something like this for the C docs.
It should probably not rely on "auto" directives and instead focus on .. c:function:: and similar.

I'm using Sphinx 4.2.0 with sphinx_c_autodoc 1.0.0 and sphinx_c_autodoc.viewcode extensions. I have a particular file whose viewcode is broken and displays as a continuous stream of text rather than being nicely formatted. I've attached a zip file which contains the source code (lights.c), the viewcode file (lights.c.html) and my conf.py file.

viewcodeissue.zip

Could there be something in my source file causing this or is it a bug?

Conversation from #10 ,

One idea I've been trying to consider, is doing like doxygen and combining the documentation from the declaration and the definition. I think it would require an option to let the user specify if the docs go at the declaration or the definition location.

I think adding something like this would be incredibly useful, especially if also integrated with the viewsource extension, which would allow the developer to define the documentation in a header file, have it show up in the source (or header) and have it link to the source code.

The main idea is to consolidate any documentation in the declaration with that of the definition.
Some things to consider:

• As @wynn-rj mentioned viewsource should probably default to linking to the definition.
• If autocmodule is used where should the resultant documentation end up? Doxygen appears to only put it with the header (declaration). The header may be a good initial default.

When parsing a structure like the one below

struct my_struct {
   int array[10]
};

The extension will give the following warning:

/usr/local/lib/python3.7/dist-packages/sphinx_c_autodoc/loader.py:docstring of my_file.h:1: Warning: Error in declarator of parameters
Invalid C declaration: Expected identifier in nested name. [error at 4]
  int [10] array
  ----^

Taking the array portion ([10]) makes the warning go away.

Any file level variable that has a type that is included from a standard include file (such as bool from <stdbool.h> or pthread_mutex_t from <pthread.h>) will cause the following error to be emitted

/usr/local/lib/python3.7/dist-packages/sphinx_c_autodoc-0.3.1-py3.7.egg/sphinx_c_autodoc/loader.py:docstring of example.c:1: WARNING: Error in declarator or parameters
Invalid C declaration: Expected identifier in nested name. [error at 3]

And the documentation generated from them will be incorrect. No type will be listed and the [source] link if using the viewcode extension will not be present.

After a little inspection it seems to be that the issue arises whenever the get_soup_declaration method of the DocumentVariable class returns None. It then defaults to using the name of the variable, which is without a type. A possible place to get the correct type information from would be the node.get_tokens() method, but there may be a better place that I am not aware of

The following bug can be replicated by including the following code into example.c

#include <stdbool.h>

/** 
 * A file level bool variable
 */
bool some_bool_variable = true;

specifying something like

.. autocmodule:: some/root/path/foo.c

Will result in an error as currently the logic is using the python module representation which is some.package.path.module.

Just ran a build with 1.2.0 this morning on a project using sphinx-c-autodoc that yielded a warning:
invalid signature for autocstruct (path/file.h::)

This is caused by this style of unnamed declaration:

typedef struct
{
    int foo;
    int bar;
} struct_def_t;

and not :

typedef struct named
{
    int foo;
    int bar;
} struct_def_t;

The error doesn't disappear with any changes to documentation of the struct, and I can confirm that the issue is not present in further builds using 1.1.0 releases.

Consider the following C code:

/**
 * My Define
 */
#define MY_BITMASK  0x8000U

/**
 * My Define mask
 */
#define MY_MASKED( base_enum ) ((base_enum) | MY_BITMASK )

/**
 * My enum
 */
typedef enum {
    VALUE_0,          /**< my value 0*/
    VALUE_1,          /**< my value 1*/
    VALUE_2 = MY_MASKED( VALUE_0 ),     /**< my value 2*/
    VALUE_3 = MY_MASKED( VALUE_1 ),     /**< my value 3*/
} my_value_e;

When running sphinx with sphinx-c-autodoc it gives the following error:

.../venv_py3/lib/python3.8/site-packages/sphinx_c_autodoc/loader.py:docstring of my.h:1: ERROR: Error in "c:enumerator" directive:
invalid option data: duplicate option "module".
.. c:enumerator:: VALUE_3 = ((VALUE_1) | 32768U)
   :module: hx_cv_frame.h
   :module: hx_cv_frame.h
   my value 2
   my value 3

and the generated HTML doesn't contain VALUE_2 and VALUE_3 in the list of enum members.

Removing VALUE_3 from enum makes the error disappear (and VALUE_2 renders ok in HTML), but in real life, I have a lot of enums using MY_MASKED macro.
Is it possible to add support for multiple entries using MY_MASKED macro?

Thanks

What are the problems for this extension to be Sphinx 3.x compatible?

Installing this extension currently degrades Sphinx from 3.2.1 to 2.4.4. As Sphinx is usually full of bugs and problems, it wise to use always the latest Sphinx version.

/cc @eine

In C, when I document the various options of an enum, sphinx-c-autodoc puts them in a namespace named after the enum. When x-referencing the enumerators, I need to provide the namespace. That is okay I guess for writing the documentation (explicit and all), but the namespace actually ends up in the rendered output. This is contrary to how the languages (C & C++) work; they consider the enumerators to live in the same namespace as the enum itself. C doesn't even have namespaces to begin with.

typedef enum MYAPI_ErrorCode {
    /**
     * No error
     */
    MYAPI_STATUS_SUCCESS = 0,
} MYAPI_ErrorCode ;

/**
 * :returns: :c:enumerator:`MYAPI_ErrorCode.MYAPI_STATUS_SUCCESS`
 */
MYAPI_ErrorCode foobar();

Rendered:

Currently the version info isn't available programmatically it is common for python packages to provide this.

In setup.py, the minimum required version for sphinx is >= 3. However, sphinx-c-autodoc does not work with this version, as it depends on features added in the 3.1.0 version of sphinx. Setting the minimum version to >= 3.1 fixes the issue

Traceback (most recent call last):
  File "/usr/local/bin/sphinx-c-apidoc", line 6, in <module>
    from sphinx_c_autodoc.apidoc import main
  File "/usr/local/lib/python3.7/dist-packages/sphinx_c_autodoc/__init__.py", line 29, in <module>
    from sphinx.ext.autodoc import (
ImportError: cannot import name 'member_order_option' from 'sphinx.ext.autodoc' (/usr/local/lib/python3.7/dist-packages/sphinx/ext/autodoc/__init__.py)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency types-docutils to v0.20.0.20240406
• Update dependency sphinx to v7.3.7
• Update KyleMayes/install-llvm-action action to v2
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Thinking these flags should have a way to be per module. Given that it may be best to create a directive option which is a list of flags and leverage the autodoc-default-options

.. autocmodule:: foo.c
    :compile_options: '-DFoo="My string"' '-DDEBUG'

Is this something on the roadmap for being released soon? We would love to have compability/support with Sphinx7 so we're awaiting this release.

Thanks!

Currently it looks like the c module reference is relative to the current rst file being parsed, not relative to the conf.py.

The initial intent was to be relative to one place and the conf.py seemed like the most reasonable.

_insert_line_anchors does not check to see if key "children" exists before iterating through it's members. This is caused by empty c files or header files that have no normal members. An example would be a header file like this

/* version_list.h */
"version_1",
"version_2",

Which would be included in a c file like

char * supported_versions[] {
    #include "version_list.h"
}

Currently a user has to manually specify each file if trying to build up a list of files to document.
Something such as https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html would allow users to more quickly set up a project for API documentation

It looks like sphinx 3.4 made changes that cause test failures

It looks like only one set of unit tests fail, so it may be that the sphinx test environment changed.
Rolling back to sphinx 3.3 works.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ c:\git\sphinx-c-autodoc\.tox\py37\lib\site-packages\sphinx_c_autodoc\__init__.py:422: in __init__
    super().__init__(directive, name, indent)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sphinx_c_autodoc.CTypeDocumenter object at 0x000001CE7FAAEF08>
directive = <sphinx.ext.autodoc.directive.DocumenterBridge object at 0x000001CE7FAAECC8>, name = 'my_name', indent = ''

    def __init__(self, directive: "DocumenterBridge", name: str, indent: str = '') -> None:
        self.directive = directive
>       self.config = directive.env.config
E       AttributeError: 'NoneType' object has no attribute 'config'

c:\git\sphinx-c-autodoc\.tox\py37\lib\site-packages\sphinx\ext\autodoc\__init__.py:330: AttributeError

Is there any way to detect if the source file is currently processed by the documentation generator, rather than being built properly?

I imagine there could be a macro that is only defined by sphinx-c-autodoc, so that I can do

#ifdef SPHINX_C_AUTODOC
// whatever
#endif

I wanted to try your sphinx-c-autodoc to document one of my zephyr projects. The code is entirely written in c. It works quite well at least for the configuration and building. The rendered code documentation seems not to work as expected.

The source.rst looks like this

main func
=========

.. autocfunction:: main.c::main

file main.c
===========

.. autocmodule:: main.c
   :members:
   :private-members:

it is then rendered like this:

The doxygen docu and @Keywords are not really rendered properly.

versions used:

• sphinx-build: 7.2.6
• sphinx-c-autodoc: 1.3.0
• clang: 14.0.6
• ubuntu 2022.4

Do you have a clue what the issue might be? Many Thanks in advance!
Thomas

sphinx-doc/sphinx#11302

Describe the bug

see this Log:

https://readthedocs.org/api/v2/build/20050480.txt

How to Reproduce

see:

https://github.com/Meisterschulen-am-Ostbahnhof-Munchen/visual-programming-languages-docs/blob/main/docs/conf.py

Environment Information

all Settings public Available:

https://github.com/Meisterschulen-am-Ostbahnhof-Munchen/visual-programming-languages-docs

sphinx-doc/sphinx#11302