How to handle files that need generating? about fusesoc HOT 9 CLOSED

Hi,

I have some grand plans for this, but haven't yet implemented any of them.

First I would like to divide this into two categories. The first one, which seem to be the one you describe, is to generate code from an existing description, like generating RTL or NGC files with CoreGen from cgp and xco files. For that I want new sections in the core file. To allow for several CoreGen files, I think something like the syntax below would work

[coregen.myfancyipcore]
cgp_file=input_file1.cgp
xco_file=input_file2.xco
ngc_file=output_file1.ngc

The other category is when we are free to define the configuration files ourselves. An example of this is the Wishbone interconnect generator in wb_intercon. For those cases I envision cores being able to register themselves as "generators", which mean that they will tell FuseSoc they have the ability to generate code, either from the command-line, or on-the-fly with via a section in the .core file. To examplify this with wb_intercon_gen:

Today we call wb_intercon_gen manually with /path/to/wb_intercon_gen

By adding the following section to wb_intercon.core FuseSoC will be aware that wb_intercon is cabaple of generating code (exact format of the section is not really defined yet)

[generator]
name = wb_intercon_gen
description = Creates a Wishbone Interconnect from a description
config_file = Configuration file
output_file = Output file
module_name = Module name

and we can now generate the code with fusesoc gen wb_intercon_gen .

We can also add a section to the core that wants to use autogenerated code, to build that on-the-fly with something like this

(in toplevel .core file)
[gen.wb_intercon]
generator = wb_intercon_gen
config_file = data/wb_intercon.conf
output_file = rtl/verilog/wb_intercon.v
module_name=wb_intercon

This will hopefully allow us to reuse all code generators that people have built over the years.

Well, these are still just ideas, since I haven't gotten around actually implementing any of it, but I would be interested in knowing if that would be sufficient for your use case

from fusesoc.

Yes, that sounds like what I'm thinking about. At the moment I'm using a build system that I rolled myself (github.com/benreynwar/pyvivado), but I thought it'd be nice to at least make it compatible with what you're doing. I end up needing to generate code quite often, for example if I wanted to generate a tree with the depth specified as a parameter.

The approach I've taken is for each module to subclass a Builder class which is responsible for specifying the necessary filenames for simple modules, and also for generating them for more complex modules (https://github.com/benreynwar/pyvivado/blob/master/builder.py). This makes things nice and flexible, but I think your approach of writing a .core file will be something that people are more comfortable with.

Things start to get complicated when you start having the same generator with the same parameters used in different places in the design, or when one generator has a dependency on another generator where the parameters it passes to the second generator are some function of the ones it has received itself. Or perhaps you have a generic module with a dependency on a generator where the parameters it passes depend on it's generic parameters.

I'm not at all sure on a good way to handle these kinds of situations.

from fusesoc.

After plenty of discussion, this is now moving forward. The approach that was chosen was to use the FuseSoC provder mechanism. This is normally used to download from remote locations, but we can also use it to provide generated files. Two new providers for Xilinx CoreGen and LogiCore IP is currently provided

from fusesoc.

If I have an ASIC design with compiled memories, I'll have one memory file for each (width, depth) pair. To be able to make the design parameterizable I want to wrap all those compiled memories in a single memory module with generic height and width parameters, which then instantiates the appropriate compiled memory depending on those parameters.

I can create a provider for the compiled memory, but when it generates the files it needs to know all the (width, height) combinations for which it will be used. My current plan for this is to add a logging statement into a dummy memory wrapper and elaborate the design using ghdl (or similar). The output could then be parsed, the logging statements extracted, and the set of (width, depth) pairs extracted. This information could then be used by the provider to generate the files.

Ideally this should be an iterative process, so that generated files could also instantiate modules that need to be generated.

It feels totally doable, but would require some serious tinkering with fusesoc. Is this an approach that you would be comfortable with? I'm happy to have a first crack at getting this working.

from fusesoc.

It's great to have more real-world use cases. Please go ahead and see what you come up with and I'm happy to help, but my worry is that the provider API might be too limited for this, and that this might be a good task for the yet-to-be-created generators infrastructure or the use flags support.

Would it work to include all the compiled memory files in a fileset (assuming they are verilog/vhdl files), and then the wrapper would instantate the correct one and discard the others?

It's an interesting problem and I'll keep it in my head

from fusesoc.

from fusesoc.

I've got a solution now that works, although it's not very elegant. I'm keeping it in a separate repo since I think it's unlikely it would make sense to merge into fusesoc.

https://github.com/benreynwar/fusesoc_generators

It adds a section to a core file that associates a python function with the core. The python function has access to the generic parameters of the module, and returns a list of all the files it generated. The inelegant part is related to getting the generic parameters to pass to the python function. At the moment this is done by running the design through GHDL, and including "assert" statements in the module which log the generic parameters that are required.

It's convoluted but it works.

from fusesoc.

Hi Ben,

Looking forward to dig into your code. Hope to find some time for that next week. Even if the whole thing would be hard to get into FuseSoC, perhaps there are bits and pieces that can be useful and we can iterate from this.

from fusesoc.

Support for FuseSoC generators is now pushed to master. Hopefully this will work for most of your cases. If not, reopen or file a new issue

from fusesoc.