apigeecs / apigee-migrate-tool Goto Github PK
View Code? Open in Web Editor NEWExport/Import Org Data. Import from CSV files
License: MIT License
Export/Import Org Data. Import from CSV files
License: MIT License
Please provide unit tests, e.g. mocha, so that further commits can be vetted to ensure the continual success of the software.
When running this tool I am getting below error.
Running tasks: exportAll
Running "exportAll" task
Running "exportProxies" task
========================= export Proxies ===========================
getting proxies...https://xxxxxxxxxxxxxxxx
Error: write EPROTO C4370000:error:0A000152:SSL routines:final_renegotiate:unsafe legacy renegotiation disabled:c:\ws\deps\openssl\openssl\ssl\statem\extensions.c:922:
Would appreciate help here?
exportDevs is not exporting all data !
Hi,
When I am running exportAll, it is not downloading all KVMs. In my enironment, I have 19 but it's downloading only 16. What could be the reason?
Then I thought of running exportEnvKVM task individually, but it is just creating env folder under kvm folder of data, but there is nothing in the folder.
Not sure if I am missing something or there is really any issue.
Could someone please help?
Thanks in advance!!
Vivek Chikane
apps are not exported , empty folders created instead
While trying to exportAllSpecs I get an error
“.Cannot read properly ‘length’ of undefined.
The task "deployProxies" fails with these traces :
Running "deployProxies" task
Warning: proxy_url is not defined Use --force to continue.
Aborted due to warnings.
Please indicate which Node.js engine the project requires:
package.json
.node-version
Would you consider adding targets under the environmental configuration as well? You have all of the other state handled within Apigee already. Thanks!
when running "grunt exportProxies" from my prod environment , but its exporting from latest version the proxy not the proxies deployed to prod environment . My requirement is to export all the prod environment api and import to new org. Please suggest how to make this using this tool.
We are migrating from on premises version to public cloud. How to provide "to": version information for edge public cloud. This information need to be set into config.js
https://github.com/apigeecs/apigee-migrate-tool/blob/master/config.js
When running this tool I am getting below error. I am guessing it is due to cert issue. We are using our company signed ssl certificate for management server. Please let us know workaround for this.
Running tasks: exportAll
Running "exportAll" task
Running "exportProxies" task
========================= export Proxies ===========================
getting proxies...https://:/
Error: unable to verify the first certificate
almost all scripts are parsing json responses incorrectly leading to the full tool to fail.
Nothing gets exported.
I fixed one or 2 scripts manually to confirm it was a parsing issue. appending a child element usually does the trick, but all scripts needs to be reviewed
Also export of revision is badly implemented. Last revision from json response is taken, rather than the revision with highest number.
If you export the developers from an organization A and try to import them to organization B the target organization is not set during import but the organization of the source.
Workround: The import works if you manually change the value of the property
"organizationName" : "A",
to
"organizationName" : "B",
in every developer export file.
Using importAll, if the data source is sparsely populated (e.g. missing targetservers) the tool exits prematurely. Happens for all entities.
Workaround is to use the individual commands in that case. See all_import_tasks.sh for the order.
Hi! Thanks for maintaining this tool!
I'm currently trying to use this tool to migrate data between some corporate Apigee installations, I've ran into an issue I've mostly been able to address which is that most of the tasks here will fire off asynchronous javascript requests as fast as possible without waiting for a callback. This was a problem for me because I didn't want to install grunt and NPM on my production servers, and was forced to use this tool over an SSH tunnel; which then gets overwhelmed with requests and dies whenever I try to migrate ~6000 developers or ~7000 something keys.
Now for the most part (and having no javascript experience), I was able to substitute out the request library in each of the tasks for the request-promise library which returns a promise for the callback of the async function, and then use the recent javascript 'async/await' functionality to make those tasks execute synchronously. (Awaiting on any request to callback before proceeding with the loop) However I've found the key.js task quite tricky to get to execute synchronously and is what I'm stuck on now.
Anyway, what I'm getting at is a rate limiting feature would be nice!
So that apigee-migrate-tool can relax strict requirement for ES modules, and thereby work with a wider range of Node.js environments.
This tool doesn't migrate the apimodels resources and subresources. This is not mentionned in the doc.
These resources are required by the devPortals in order to work.
See here a real life use-case when migrating from one instance to another one : https://community.apigee.com/questions/54413/the-model-disappears-when-changing-the-management.html
When you run the exportAll command it downloads the proxies, products and devs only.
Version from https://github.com/stevetraut/apigee-migrate-tool downloads all entities. But it is not up to date (for example flow hooks support is not added there.)
Please fix this problem in current repository.
I am having a issue in the "import app". I have 140 Apps and exported all Apps in the data folder. While running "grunt importApps",
only 70 Apps imported . rest of the Apps not imported .
Also "Credential" section is empty for all import Apps. I want credential section also import.
EX- data/apps/[email protected]/AMEX-DEV-MANAGER-API-700_REGISTERED_NOT_VERIFIED this kind of not import . Any issue here ?
Please help me to fix these TWO issues.
If the value of the target env differs from the value of source env the import of the developer app fails.
Workaround: If you set the target env manually in each DevApp export file
"environments" : ["A"],
the import works.
Hey, I'd like to present this to LLBean, do you think it's possible to make this repo public? LLBean understands these tools are all in beta, but they'd rather tweaking it and perhaps send fixes.
WDYT?
i am trying to export all apps, but get an error. i looked at logs and i think last retrieved developer does have any applications registered..
--
Running "exportApps" task
Retrieved 6 developers
Retrieved 6 developers
Fatal error: Cannot read properties of undefined (reading 'length')
TypeError: Cannot read properties of undefined (reading 'length')
at Request._callback (/Users/santos/src/apigee-migrate-tool/tasks/app.js:55:27)
at Request.self.callback (/Users/santos/src/apigee-migrate-tool/node_modules/request/request.js:185:22)
at Request.emit (node:events:527:28)
at Request. (/Users/santos/src/apigee-migrate-tool/node_modules/request/request.js:1154:10)
at Request.emit (node:events:527:28)
at IncomingMessage. (/Users/santos/src/apigee-migrate-tool/node_modules/request/request.js:1076:12)
at Object.onceWrapper (node:events:641:28)
at IncomingMessage.emit (node:events:539:35)
at endReadableNT (node:internal/streams/readable:1345:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
When I run grunt exportAll -v command, it executes only starting 4 tasks from register task. I tried this by changing the sequence.
grunt.registerTask('exportAll', ['exportDevs','exportProducts','exportApps', 'exportEnvKVM', 'exportProxies', 'exportOrgKVM', 'exportProxyKVM']);
Kindly suggest if I am going somewhere wrong.
I'm just wondering if there is a possibility of adding the ability to migrate Companies using this tool in the near future? If not that's fine I'll just have to find another way of doing it.
Hi,
There is an error in the file "tasks/proxy.js", line 91: the method " grunt.verbose.log(err); " does not exist. This should be replaced with something like " grunt.verbose.writeln(err); "
var req = request.post(url+name, function (err, resp, body) {
if (err) {
grunt.verbose.log(err);
} else {
delete key function is not getting executed;
https://github.com/apigeecs/apigee-migrate-tool/blob/master/tasks/app.js#L220
Export proxy kvm not working not exporting any data
Hi,
I have used "grunt exportAll -v" all data is not getting imported to target environment. So is there any singe command to import all the exported data to local drive.?
Can all the data be imported using single command.?
Or
we need to run all the below commands manually one by one:
grunt importTargetServers
grunt importProxies
grunt importSharedFlows
grunt importDevs
grunt importProducts
grunt importApps
grunt importKeys
grunt importProxyKVM
grunt importEnvKVM
grunt importOrgKVM
There are many areas where request fail for unknown reasons and log messages do not provide an indication as to what failed and why. Often simply logging null
to console and not indicating what resource failed with a helpful message.
exportEnvKVM is showing :
Exported 26 kvms
================== export ENV KVM DONE()
while only 15 are exported!
with i am able to export only latest proxy and imported to new servers.
How i can export and import all the revisions of proxies and import them into now APIGEE system
An org with many (> 10,000) developers fails to export all the developers and generates this error:
Fatal error: error is not defined
exit code $? is 3
Using this tool behind a corporate web-proxy requires uncovering undocumented configs. Needs to be addressed in docs at the least.
Orgs with SAML federation or Orgs with Mandatory 2FA can not use this tool any longer.
Can we support Bearer tokens as an alternative to Basic Auth?
Target Servers place results in ./data/targetservers regardless of env.
Should create sub-directory for each env (e.g. /data/targetservers/test, /data/targetservers/prod)
Flow Hooks place results in ./data/flowhooks regardless of env.
Should create sub-directory for each env (e.g. /data/flowhooks/test, /data/flowhooks/prod)
KVMs follow this pattern.
data
└── kvm
└── env
├── prod
│ └── GetLogValues
└── test
└── GetLogValues
while tryingto export import sharedflow, i noticed sharedflow export is creating file size of 0 size and hence import gives the following error. Tried extracting the zip file of exported shared flow and that errors out as well.
"code" : "messaging.config.beans.InvalidBundle",
"message" : "Bundle is invalid. Empty bundle",
"contexts" : [ ]
[ sharedflows]# ls -rtl
total 0
-rw-r--r-- 1 root root 0 Jun 17 12:54 LoggingPreProxyFlowHookSharedFlow.zip
[ sharedflows]# unzip LoggingPreProxyFlowHookSharedFlow.zip
Archive: LoggingPreProxyFlowHookSharedFlow.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of LoggingPreProxyFlowHookSharedFlow.zip or
LoggingPreProxyFlowHookSharedFlow.zip.zip, and cannot find LoggingPreProxyFlowHookSharedFlow.zip.ZIP, period.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.