Giter VIP home page Giter VIP logo

edoc-gen's Introduction

about edoc-gen

This is a helper script for generating Python and Odoo ERP libraries for the Brazilian electronic fiscal documents using http://www.davekuhlman.org/generateDS.html. It supports all fiscal documents that have XSD schemas, that is: NF-e, CC-e, NFS-e, MDF-e, CT-e, EFD-Reinf, e-Social, GNRE, BP-e...

But why a helper instead of launching generateDS manually?

  • For all these fiscal documents there is a common pattern: the schemas are inside a zip archive that can be downloaded from an official URL.
  • After normalizing the xsd file names, we launch the generateDS.py generator on them. But we also want to keep our libraries small, so edoc-gen enables to specify that only some xsd files should support both export and import.
  • Finally automating all the steps with the proper settings allowed me to quickly re-generate the Python libraries for all the fiscal documents and run the pytests import/export tests on them to ensure any patch in generateDS would would retain real life compatibility (I got some 6 patches merged into generateDS). And mostly it allowed me to the do the same with the Odoo mixin generator plugin: quicky re-generate all Odoo modules and ensure they install and pass the tests. The libraries and Odoo modules are typically distributed as separated packages inside different repos that can get their own bug reports, forks and contributions but it also work to generate everything in the same directory for a quick extensive testing.

usage

Install generateDS somewhere https://www.davekuhlman.org/generateDS.html#how-to-build-and-install-it

Then export its home directory for the script to know where to find it:

export GENERATEDS_HOME=path_to_generateds

Optionnally, if you want to generate Odoo models mixins, you need the Akretion plugin for Odoo. It is available in the generateDS pull request for now: https://bitbucket.org/dkuhlman/generateds/pull-requests/51/d7e954682a90/diff .

Download this Odoo plugin branch and export the ODOO_GEN_HOM environment variable to the odoo plugin directory with (alternatively you can also merge the plugin branch):

export ODOO_GEN_HOM=path_to_odoo_plugin_dir

The script is meant to be run inline from curl so the maintenance can be centralized here without the need to bother packaging the bash scripts. The API is:

curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s schema_name plugin_name version \
'schema_url' \
'files'

  were the arguments mean:
- schema_name: name of the schame, can be nfe, nfse, cce, mdfe...
- plugin_name: python|odoo (Akretion made an Odoo plugin for GenarateDS, similar to the Django plugin)
- version: example v4_00
- schema_url: URL of the zip archive containing the schemas
- files: optionally if you want to support XML import of only some XSD files, list them separated by |. ex: 'file1|file2'

The script has a modular structure were you can override your own scripts in scripts/gen_plugin to target custom plugin. This projects assumes

  • a plugin for a Python library (marshall and unmarshall XML docs according to the provided XSD)
  • a plugin for the Odoo open source ERP that generates Odoo mixins to be injected later properly into Odoo models

For each kind of these Brazilian electronic fiscal documents you can find here the generated lib maintained by Akretion and also the basic command that has been used to generate the Python lib. These scripts are only here to demonstrate how it works (you can run them in an empty directory), but the real Python libraries have additional README.rst, tests and limited overrides maintained manually in their respective repo. We also don't generate Python classes (1000 lines minimum) for very small 20 lines schemas such as communication events as these can be easily dealt with simple Python strings or Jinja2.

NF-e | Nota Fiscal Eletrônica

Python https://github.com/akretion/nfelib
Odoo OCA/l10n-brazil#658
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s nfe python v4_00 \
'http://www.nfe.fazenda.gov.br/portal/exibirArquivo.aspx?conteudo=vdxcmJ2AgTo=' \
'leiauteNFe|leiauteInutNFe'

CC-e | Carta de Correção Eletrônica

Python https://github.com/akretion/ccelib
Odoo OCA/l10n-brazil#659
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s cce python v1_00 \
'http://www.nfe.fazenda.gov.br/portal/exibirArquivo.aspx?conteudo=P/FXaGiLKo0=' \
leiauteCCe

MDF-e | Manifestação do Destinatário Eletrônica

Python https://github.com/akretion/mdfelib
Odoo OCA/l10n-brazil#660
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s mdfe python v3_00 \
'https://dfe-portal.sefazvirtual.rs.gov.br/MDFE/DownloadArquivoEstatico/?sistema=MDFE&tipoArquivo=2&nomeArquivo=PL_MDFe_300_NT022018_v1.02.zip' \
'mdfe|mdfeModalAereo|mdfeModalAquaviario|mdfeModalFerroviario|mdfeModalRodoviario'

CT-e | Conhecimento de Transporte Eletrônico

Python https://github.com/akretion/ctelib
Odoo OCA/l10n-brazil#661
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s cte python v3_00 \
'http://www.cte.fazenda.gov.br/portal/exibirArquivo.aspx?conteudo=xuFsi7DbwBk=' \
'cte|cteModalAereo|cteModalAquaviario|cteModalDutoviario|cteModalFerroviario|cteModalRodoviarioOS|cteModalRodoviario|cteMultiModal'

NFS-e | Nota Fiscal de Serviço padrão nacional ABRASF

Python https://github.com/akretion/nfselib
Odoo OCA/l10n-brazil#662
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s nfse python v2_03 \
'http://www.abrasf.org.br/arquivos/publico/NFS-e/Versao_2.03/schema_xml_nfs-e_%20v2.03.zip' \
nfse

EFD-Reinf | Escrituração Fiscal Digital de Retenções e Outras Informações Fiscais

Python https://github.com/akretion/nfselib
Odoo OCA/l10n-brazil#666
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s efdreinf python v01_04 \
'http://sped.rfb.gov.br/estatico/CA/E40B96DD94D4B54626EDDF0CC3004937AB1597/Pacote%20XSD%20Eventos%20EFD%20Reinf%20v1_04_00.zip'

e-Social | Sistema de Escrituração Digital das Obrigações Fiscais, Previdenciárias e Trabalhistas

Python https://github.com/akretion/esociallib
Odoo OCA/l10n-brazil#665
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s esocial python v02_05 \
'https://portal.esocial.gov.br/manuais/2019-01-29_esquemas_xsd_v02-05-00.zip'

GNRE | Guia Nacional de Recolhimento de Tributos Estaduais

Python https://github.com/akretion/gnrelib
Odoo OCA/l10n-brazil#663
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s gnre python v1_00 \
'http://www.gnre.pe.gov.br/gnre/portal/arquivos/EsquemaLote.zip' \
lote_gnre

BP-e | Bilhete de Passagem Eletrônico

Python https://github.com/akretion/bpelib
Odoo OCA/l10n-brazil#664
curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s bpe python v1_00 \
'https://portal.fazenda.sp.gov.br/servicos/bpe/Documents/PL_BPe_100_NT022018.zip' \
'bpe|eventoBPeTiposBasico'

edoc-gen's People

Contributors

rvalyi avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

edoc-gen's Issues

nfse after_prepare: TODO backup file

This is the nfse after_prepare hook because of the invalid XSD file. I put it here to avoid loosing it by mistake until it's properly committed in akretion/nfselib.

#!/usr/bin/env bash
set -eo pipefail

after_prepare() {
  local schema_name=$1
  local version=$2

  # rename
  mv $ROOT_DIR/schemas/$schema_name/$version/*.txt* $ROOT_DIR/schemas/$schema_name/$version/nfse_v2.03.xsd
  # fix schema typo
  sed -i 's/ xsd:/<xsd:/' "$ROOT_DIR/schemas/$schema_name/$version/nfse_v2.03.xsd"
}

after_prepare $@

MDF-e

Opa seguindo o tutorial de uso desse generate acabei esbarrando em algumas situações.
Primeiramente nunca usei algo do tipo e nunca vi, então não sei bem como funciona, mas estou tentando rsrs.

** Primeiro instalei o generate usando o**

pip3 install generateDS

Logo ele ficou no seguinte diretório

/home/implanti/.local/bin/

Terceiro passo foi usar o seguinte comando

export GENERATEDS_HOME=/home/implanti/.local/bin/

E por quarto passo tentei executa o comando abaixo que esta no tutorial

curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s mdfe python v3_00
'https://dfe-portal.sefazvirtual.rs.gov.br/MDFE/DownloadArquivoEstatico/?sistema=MDFE&tipoArquivo=2&nomeArquivo=PL_MDFe_300_NT022018_v1.02.zip'
'mdfe|mdfeModalAereo|mdfeModalAquaviario|mdfeModalFerroviario|mdfeModalRodoviario'

Estava dando erro o erro abaixo

implanti@ubuntu:/opt/odoo/test$ curl https://raw.githubusercontent.com/akretion/edoc-> gen/master/generate | bash -s mdfe python v3_00
'https://dfe-portal.sefazvirtual.rs.gov.br/MDFE/DownloadArquivoEstatico/?sistema=MDFE&tipoArquivo=2&nomeArquivo=PL_MDFe_300_NT022018_v1.02.zip'
'mdfe|mdfeModalAereo|mdfeModalAquaviario|mdfeModalFerroviario|mdfeModalRodoviario'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3660 100 3660 0 0 4889 0 --:--:-- --:--:-- --:--:-- 4893
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 315 100 315 0 0 334 0 --:--:-- --:--:-- --:--:-- 334
Archive: /tmp/schemas.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of /tmp/schemas.zip or>
/tmp/schemas.zip.zip, and cannot find /tmp/schemas.zip.ZIP, period

E analisando o erro, percebi que url dos esquemas estava errado, primeiro que esta apontando pra um schema desatualizado, segundo que url da sefaz estava incorreta, então fiz a correção e o comando ficou assim

curl https://raw.githubusercontent.com/akretion/edoc-gen/master/generate | bash -s mdfe python v3.00 'https://dfe-portal.svrs.rs.gov.br/MDFE/DownloadArquivoEstatico/?sistema=MDFE&tipoArquivo=2&nomeArquivo=PL_MDFe_300a.zip' 'mdfe|mdfeModalAereo|mdfeModalAquaviario|mdfeModalFerroviario|mdfeModalRodoviario'

Logo acabou dando outro erro:

running generic plugin command:
(you can override it with a local scripts/mdfe/python/prepare file)
/home/implanti/.edoc-gen/cache/scripts/python/prepare | bash -s mdfe v3.00 || echo ''
grep: docs/conf.py: Arquivo ou diretório não encontrado

running generic plugin command:
(you can override it with a local scripts/mdfe/python/after_prepare file)
/home/implanti/.edoc-gen/cache/scripts/python/after_prepare | bash -s mdfe v3.00 || echo ''

processing file mdfeModalAereo_v3.00.xsd ...

running generic plugin command:
(you can override it with a local scripts/mdfe/python/generate_file file)
/home/implanti/.edoc-gen/cache/scripts/python/generate_file | bash -s mdfe v3.00 mdfeModalAereo mdfeModalAereo_v3.00.xsd || echo ''
python /home/implanti/.local/bin//generateDS.py --no-namespace-defs --use-getter-setter=none -f -o /opt/odoo/test/mdfelib/v3.00/mdfeModalAereo.py mdfeModalAereo_v3.00.xsd
Traceback (most recent call last):
File "/home/implanti/.local/bin//generateDS.py", line 8613, in
main()
File "/home/implanti/.local/bin//generateDS.py", line 8595, in main
superModule=superModule)
File "/home/implanti/.local/bin//generateDS.py", line 8015, in parseAndGenerate
prefix, root, options, args, superModule)
File "/home/implanti/.local/bin//generateDS.py", line 7766, in generate
generateSimpleTypes(wrt, prefix, SimpleTypeDict, root)
File "/home/implanti/.local/bin//generateDS.py", line 7695, in generateSimpleTypes
writeEnumClass(simpleType)
File "/home/implanti/.local/bin//generateDS.py", line 7665, in writeEnumClass
output += docstring if docstring else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

processing file mdfeModalAquaviario_v3.00.xsd ...

running generic plugin command:
(you can override it with a local scripts/mdfe/python/generate_file file)
/home/implanti/.edoc-gen/cache/scripts/python/generate_file | bash -s mdfe v3.00 mdfeModalAquaviario mdfeModalAquaviario_v3.00.xsd || echo ''
python /home/implanti/.local/bin//generateDS.py --no-namespace-defs --use-getter-setter=none -f -o /opt/odoo/test/mdfelib/v3.00/mdfeModalAquaviario.py mdfeModalAquaviario_v3.00.xsd
Traceback (most recent call last):
File "/home/implanti/.local/bin//generateDS.py", line 8613, in
main()
File "/home/implanti/.local/bin//generateDS.py", line 8595, in main
superModule=superModule)
File "/home/implanti/.local/bin//generateDS.py", line 8015, in parseAndGenerate
prefix, root, options, args, superModule)
File "/home/implanti/.local/bin//generateDS.py", line 7766, in generate
generateSimpleTypes(wrt, prefix, SimpleTypeDict, root)
File "/home/implanti/.local/bin//generateDS.py", line 7695, in generateSimpleTypes
writeEnumClass(simpleType)
File "/home/implanti/.local/bin//generateDS.py", line 7665, in writeEnumClass
output += docstring if docstring else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

processing file mdfeModalFerroviario_v3.00.xsd ...

running generic plugin command:
(you can override it with a local scripts/mdfe/python/generate_file file)
/home/implanti/.edoc-gen/cache/scripts/python/generate_file | bash -s mdfe v3.00 mdfeModalFerroviario mdfeModalFerroviario_v3.00.xsd || echo ''
python /home/implanti/.local/bin//generateDS.py --no-namespace-defs --use-getter-setter=none -f -o /opt/odoo/test/mdfelib/v3.00/mdfeModalFerroviario.py mdfeModalFerroviario_v3.00.xsd
Traceback (most recent call last):
File "/home/implanti/.local/bin//generateDS.py", line 8613, in
main()
File "/home/implanti/.local/bin//generateDS.py", line 8595, in main
superModule=superModule)
File "/home/implanti/.local/bin//generateDS.py", line 8015, in parseAndGenerate
prefix, root, options, args, superModule)
File "/home/implanti/.local/bin//generateDS.py", line 7766, in generate
generateSimpleTypes(wrt, prefix, SimpleTypeDict, root)
File "/home/implanti/.local/bin//generateDS.py", line 7695, in generateSimpleTypes
writeEnumClass(simpleType)
File "/home/implanti/.local/bin//generateDS.py", line 7665, in writeEnumClass
output += docstring if docstring else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

processing file mdfeModalRodoviario_v3.00.xsd ...

running generic plugin command:
(you can override it with a local scripts/mdfe/python/generate_file file)
/home/implanti/.edoc-gen/cache/scripts/python/generate_file | bash -s mdfe v3.00 mdfeModalRodoviario mdfeModalRodoviario_v3.00.xsd || echo ''
python /home/implanti/.local/bin//generateDS.py --no-namespace-defs --use-getter-setter=none -f -o /opt/odoo/test/mdfelib/v3.00/mdfeModalRodoviario.py mdfeModalRodoviario_v3.00.xsd
Traceback (most recent call last):
File "/home/implanti/.local/bin//generateDS.py", line 8613, in
main()
File "/home/implanti/.local/bin//generateDS.py", line 8595, in main
superModule=superModule)
File "/home/implanti/.local/bin//generateDS.py", line 8015, in parseAndGenerate
prefix, root, options, args, superModule)
File "/home/implanti/.local/bin//generateDS.py", line 7766, in generate
generateSimpleTypes(wrt, prefix, SimpleTypeDict, root)
File "/home/implanti/.local/bin//generateDS.py", line 7695, in generateSimpleTypes
writeEnumClass(simpleType)
File "/home/implanti/.local/bin//generateDS.py", line 7665, in writeEnumClass
output += docstring if docstring else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

processing file mdfe_v3.00.xsd ...

running generic plugin command:
(you can override it with a local scripts/mdfe/python/generate_file file)
/home/implanti/.edoc-gen/cache/scripts/python/generate_file | bash -s mdfe v3.00 mdfe mdfe_v3.00.xsd || echo ''
python /home/implanti/.local/bin//generateDS.py --no-namespace-defs --use-getter-setter=none -f -o /opt/odoo/test/mdfelib/v3.00/mdfe.py mdfe_v3.00.xsd
Traceback (most recent call last):
File "/home/implanti/.local/bin//generateDS.py", line 8613, in
main()
File "/home/implanti/.local/bin//generateDS.py", line 8595, in main
superModule=superModule)
File "/home/implanti/.local/bin//generateDS.py", line 8015, in parseAndGenerate
prefix, root, options, args, superModule)
File "/home/implanti/.local/bin//generateDS.py", line 7766, in generate
generateSimpleTypes(wrt, prefix, SimpleTypeDict, root)
File "/home/implanti/.local/bin//generateDS.py", line 7695, in generateSimpleTypes
writeEnumClass(simpleType)
File "/home/implanti/.local/bin//generateDS.py", line 7665, in writeEnumClass
output += docstring if docstring else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

Apesar de ter gerado o codigo, acredito que esta incompleto por causa do erro acima, bom de cara, eu poderia dizer que poderia dar uma facilitada no modo de uso no readme para iniciantes, pois os links que foi colocado tem muito conteúdo e é meio que confuso.

E por ultima me ajuda nisso kkkk

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.