zypp-io / twinfield Goto Github PK
View Code? Open in Web Editor NEWPython client for the Twinfield API to get and post data
License: Apache License 2.0
Python client for the Twinfield API to get and post data
License: Apache License 2.0
Om efficientie slag te maken, lees alleen 1 regel in:
twinfield/twinfield/messages.py
Lines 301 to 319 in ef380f5
add continous integration
De releases 2.1.0 en 2.1.1 geven onderstaande fout bij de insert/update van dimensies. Dit komt vaak/altijd voor, zodat ik mijn versies heb vastgezet op 2.0.5.
Het lezen van dimensies geeft geen problemen.
2023-03-16 08:21:56.807 [INFO ] [root] - Er zijn 6 dimensies met verschillen
2023-03-16 08:21:56.922 [INFO ] [root] - E-mail verzenden: Verschillen dimensies
2023-03-16 08:21:57.352 [INFO ] [root] - Er zijn 6 regels met wijzigingen en 3 regels met nieuwe dimensies.
2023-03-16 08:21:57.433 [INFO ] [root] - 6 dimensies geselecteerd.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/41_selfbilling_zzp/src/verwerken.py", line 31, in <module>
alleen_verwerken(test=test)
File "/41_selfbilling_zzp/src/verwerken.py", line 22, in alleen_verwerken
verwerken_gegevens(stam=stam, df=df, test=test)
File "/41_selfbilling_zzp/src/main.py", line 29, in verwerken_gegevens
DimensionSync(
File "/usr/local/lib/python3.9/site-packages/dimensions/core.py", line 400, in sync
df_dim_errors, df_dim_correct = self.send_request()
File "/usr/local/lib/python3.9/site-packages/twinfieldtools/dimensions.py", line 295, in send_request
df_errors, df_correct = self.parse_response_dimensions(response)
File "/usr/local/lib/python3.9/site-packages/twinfieldtools/dimensions.py", line 212, in parse_response_dimensions
for dimension in data.findall("dimensions/dimension"):
AttributeError: 'NoneType' object has no attribute 'findall'
De response bevat de volgende tekst. Kan het zijn dat er geen token wordt meegestuurd?
<soap:Fault><faultcode>soap:Client</faultcode><faultstring>Access denied. Token invalid.</faultstring><faultactor>http://api.accounting2.twinfield.com/webservices/processxml.asmx?wsdl</faultactor><detail><message>Access denied. Token invalid.</message><code>354</code><source>ProcessXmlDocument</source></detail></soap:Fault>
When a request fails, we retry. And the idea was to wait couple of seconds between the new requests. We have it in the logging, but we never actually implemented it. This happens here:
Lines 38 to 59 in a4b90f8
Proposal, actually sleep:
time.sleep(wait_sec)
Adding functionality for consolidated figures
The Twinfield SV routine should aggregate on an additional field "administratienummer"
In transforms.py
hebben we drie functies die hetzelfde doen, deze kunnen naar één functie worden samengevoegd. Gaat om: format_100, format_200, format_164
Bij het runnen van projecten krijg je sinds enige tijd de volgende melding:
C:\Users\BenGroenHappyNurse\Workspace\facturen_ons_hnt_31\venv\lib\site-packages\twinfield\functions.py:450: FutureWarning: The default value of regex will change from True to False in a future version.
df.columns = df.columns.str.replace("fin.trs.", "")
C:\Users\BenGroenHappyNurse\Workspace\facturen_ons_hnt_31\venv\lib\site-packages\twinfield\functions.py:450: FutureWarning: The default value of regex will change from True to False in a future version.
df.columns = df.columns.str.replace("fin.trs.", "")
Dat lijkt hier te gebeuren:
def rename_column_labels(df, module):
if module != "040_1":
df.columns = df.columns.str.replace("fin.trs.", "")
else:
login = twinfield_login()
fields = get_metadata(module, login)
df.rename(columns=fields["label"], inplace=True)
return df
Kan dit worden aangepast?
Right now we catch ConnectionError
and SSLError
in our retries when we do a request, but in login we raise a ValueError
. I propose to create a custom error when login fails, so we can specifically catch that error.
Not all functions have (a correct) docstrings. These should be added / cleaned up.
Feature request
An option not to display tqdm loop messages. They pollute logs (example below).
[2023-10-10T12:24:23.920Z] importing module 030_1 - XXX - {'from': '2023/00', 'to': '2023/00'}...: 0%| | 0/1 [00:00<?, ?it/s]
[2023-10-10T12:24:25.319Z] importing module 030_1 - XXX - {'from': '2023/01', 'to': '2023/01'}...: 0%| | 0/1 [00:00<?, ?it/s]
[2023-10-10T12:24:26.177Z] importing module 030_1 - XXX - {'from': '2023/02', 'to': '2023/02'}...: 0%| | 0/1 [00:02<?, ?it/s]
[2023-10-10T12:24:26.983Z] importing module 030_1 - XXX- {'from': '2023/03', 'to': '2023/03'}...: 0%| | 0/1 [00:03<?, ?it/s]
[2023-10-10T12:24:28.157Z] importing module 030_1 - XXX- {'from': '2023/04', 'to': '2023/04'}...: 0%| | 0/1 [00:03<?, ?it/s]
[2023-10-10T12:24:28.998Z] importing module 030_1 - XXX- {'from': '2023/05', 'to': '2023/05'}...: 0%| | 0/1 [00:05<?, ?it/s]
[2023-10-10T12:24:29.700Z] importing module 030_1 - XXX- {'from': '2023/06', 'to': '2023/06'}...: 0%| | 0/1 [00:05<?, ?it/s]
[2023-10-10T12:24:30.627Z] importing module 030_1 - XXX- {'from': '2023/07', 'to': '2023/07'}...: 0%| | 0/1 [00:06<?, ?it/s]
[2023-10-10T12:24:31.423Z] importing module 030_1 - XXX- {'from': '2023/08', 'to': '2023/08'}...: 0%| | 0/1 [00:07<?, ?it/s]
[2023-10-10T12:24:32.338Z] importing module 030_1 - XXX- {'from': '2023/09', 'to': '2023/09'}...: 0%| | 0/1 [00:08<?, ?it/s]
[2023-10-10T12:24:32.923Z] importing module 030_1 - XXX- {'from': '2023/10', 'to': '2023/10'}...: 0%| | 0/1 [00:09<?, ?it/s]
[2023-10-10T12:24:33.430Z] importing module 030_1 - XXX- {'from': '2023/11', 'to': '2023/11'}...: 0%| | 0/1 [00:09<?, ?it/s]
[2023-10-10T12:24:34.139Z] importing module 030_1 - XXX- {'from': '2023/12', 'to': '2023/12'}...: 0%| | 0/1 [00:10<?, ?it/s]
[2023-10-10T12:24:34.872Z] importing module 030_1 - XXX- {'from': '2023/13', 'to': '2023/13'}...: 0%| | 0/1 [00:11<?, ?it/s]
[2023-10-10T12:24:35.554Z] importing module 030_1 - XXX- {'from': '2023/55', 'to': '2023/55'}...: 0%| | 0/1 [00:11<?, ?it/s]
[2023-10-10T12:24:35.554Z] importing module 030_1 - XXX- {'from': '2023/55', 'to': '2023/55'}...: 100%|██████████| 1/1 [00:12<00:00, 12.47s/it]
[2023-10-10T12:24:35.554Z] importing module 030_1 - XXX- {'from': '2023/55', 'to': '2023/55'}...: 100%|██████████| 1/1 [00:12<00:00, 12.48s/it]
Potential solution
Use TQDM feature to detect TTY device and does not print messages if device is incompatible (tqdm(..., disable=None)
).
Some devices might not be detected properly, in this case suggest to implement optional LOG_LEVEL
flag (env variable? config file?) which prints progress bar when flag != 'DEBUG'
References
https://tqdm.github.io/releases/#v490-2016-10-31
tqdm/tqdm#281
Background
I am using the library in a bigger script. I log certain steps and progress bar pollutes log output.
Looking forward to hear your feedback about this idea.
Het is handig als we de errors niet in dezelfde pickle files laten landen als de resultaten. Als we errors opslaan in een errors.pkl, dan is dat netter. ook kunnen we de volgende functionaliteiten inbouwen:
The login method using SessionId seems to be deprecated. We need to develop a new login function.
Functie lijkt niet te worden gebruikt in het project, is deze nog nodig?
twinfield/twinfield/messages.py
Lines 148 to 175 in ef380f5
Adding functionality for consolidated figures
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.