skeenp / agol_backup Goto Github PK
View Code? Open in Web Editor NEWArcGIS Online Backup Utility
ArcGIS Online Backup Utility
Get multithreading working for the scripts (mainly backup_mgr.py and backup_cfg_mgr.py)
Provide functionality to move data to a remove git location as an option.
Need to add in code to reload items on successful connection to AGOL
Check the once logic in app, looks like -1.0 may not be being converted to 0.0 on successful run
Split up large exports into multiple smaller ones by layer id or another mechanism
https://developers.arcgis.com/rest/users-groups-and-items/export-item.htm
Grab OIDs, sort and then order. Query then segments of 5000 records
{
"layers" :[ {"id" : {{itterate layers}} },
{"id" : 1, "where" : "OBJECTID> x"}
]
}
Backupmgr should reference the config file used in processing
Bypass git and schedule
Would be handy to be able to use tags for backups rather than Config files. Suggesting commandline args with multiple options in the format tag:frequency(hours). Allows custom tags to be used based on user preference.
Example {exec} -b {backup path} -t tier1:1 tier2:24 for tier one tagges items to be updated hourly, tier two items to be updated daily.
Remove -c flag, it is not consistent with the other apps
Need to add a try/catch around export functions for items
# Setup export
try:
export = item.export(title=title, export_format=fmt, wait=True, overwrite=True)
# Grab data
file_data = export.download(item_dir)
os.rename(file_data, file_data.replace(postfix, ''))
# Delete export
export.delete()
except Exception as ex:
logger.debug(" > Service Export Failed")
return bla bla bla```
Sort items by name and folder in GUI
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Python39\lib\tkinter_init_.py", line 1885, in call
return self.func(*args)
File "C:\Users\skeenp01\Documents\GitHub\agol_backup\backup_cfg_mgr.py", line 739, in _runnow
backup_mgr.run([cfg_path], logger=self._logger)
File "C:\Users\skeenp01\Documents\GitHub\agol_backup\backup_mgr.py", line 47, in run
os.mkdirs(backup_dir)
AttributeError: module 'os' has no attribute 'mkdirs'
Popup window on console screen in cfg_manager when using 'run now' would be useful.
Move timestamp file creation to end of process
Need to verify config file exists before opening after backup_mgr.py ln 30
backup_mgr.py ln 214, message required in exception
GUI backgrounds for items needs updating to match new white background used in innerframe
Provide method to filter/search for items
2022-01-13 08:26:17,451 INFO: Connected to https://trade.maps.arcgis.com/
2022-01-13 08:26:17,451 INFO: Processing Items
2022-01-13 08:26:17,451 INFO: - 02746f5e401c4734b36923cb20d0018b
2022-01-13 08:27:02,927 ERROR: Unexpected error occured backing up files, please review code and try again.
Traceback (most recent call last):
File "C:\Users\skeenp01\Documents\GitHub\agol_backup\backup_mgr.py", line 163, in run
util.set_ts(os.path.join(itmdir, 'lastupdate.ts'))
File "C:\Users\skeenp01\Documents\GitHub\agol_backup\util.py", line 151, in set_ts
export_obj(file, datetime.isoformat())
TypeError: unbound method datetime.isoformat() needs an argument
export_obj(file, datetime.now().isoformat())
Remove reference to i['name'] when determining if exportable
Add a HTML summary to each out put folder to enable efficient access to backup content. Use item labels rather than ids.
File "C:\Users\skeenp01\Documents\GitHub\agol_backup\backup_cfg_mgr.py", line 501, in _loaditems
if self._ago.gis:
AttributeError: 'NoneType' object has no attribute 'gis'
Need to check for self._ago first
Add a reset flag to bypass last_updated checks
2022-03-28 12:00:27,402 INFO: - 2b7098c4f6e84005bf160eaf7913bffd
2022-03-28 12:00:29,991 ERROR: Unexpected error occured backing up files, please review code and try again.
Traceback (most recent call last):
File "/volume1/WINSW/Backups/_agolbackup/backup_mgr.py", line 147, in run
res = bi.backup(ago.gis, itemid, backup_dir, options, fmt, skip_unmodified=skipunmod, logger=logger)
File "/volume1/WINSW/Backups/_agolbackup/backup_items.py", line 68, in backup
last_modified = datetime.fromtimestamp(last_modified/1000)
ValueError: year 292278994 is out of range
2022-03-28 12:00:30,080 INFO: Script finished at 03/28/2022 12:00:30 after 5 seconds
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.