marklogic-community / roxy Goto Github PK
View Code? Open in Web Editor NEWDeployment tool for MarkLogic applications. Also provides optional unit test and XQuery MVC structure
License: Other
Deployment tool for MarkLogic applications. Also provides optional unit test and XQuery MVC structure
License: Other
I'd like to be able to set the number of forests for the content database for different environments.
If any of the standard users or roles (nobody, app-user) are used in the build.properties file, then they are removed during 'wipe', If the definitions for these built-in items haven't been copied into ml-config.xml, then they won't be restored by bootstrap. In addition, any other application that relies on them will be broken until they've been restored.
An enhancement would be to skip over the standard users and roles during wipe.
Instead of creating 1 xcc server per every app instance, allow apps to share a single xcc server. Helps to reduce frivolous xcc servers.
The code inside the xdmp:eval() in setup:configure-http-server() in deploy/lib/xquery/setup.xqy needs a namespace declaration for err (declare namespace err="http://marklogic.com/xdmp/error";). The namespace is used in the catch clause.
Following the instructions in the README the user is instructed to modify app-modules-db. The option is not in the build.properties generated after executing "ml init".
In a Roxy application I've create an element range index with the default collation (root collation -> http://marklogic.com/collation/) this resulted to the following:
Switched the collation type of the index to unicode codepoint (http://marklogic.com/collation/codepoint):
I've tested the application using the test DB, this won't happen in the feature, but it would be very uncomfortable to have two DB configs to maintain - one for the test DB and one for the app DB and just keep on changing the index collations.
Thank you!
Silvia
The code modules say that Roxy is licensed under Apache License, Version 2.0. It would be good to make this more prominent by adding this to the readme file and/or as a separate license file.
Workaround: call "ml {env} wipe" twice, the first time gets the app servers and databases, the second time gets the forests.
Fully removing app servers from MarkLogic requires a restart. If you do "ml {env} wipe" followed by "ml {env} bootstrap", you're told that a restart is necessary. You can then to "ml {env} restart". However, this only restarts the target box, not the cluster, so other servers in the cluster may still be in a funky state. Update the restart command to restart the cluster. Update documentation accordingly.
Default is GET, which leaves the password in cleartext on the location bar. The code appears to be in user-lib.xqy
, function uv:build-login
.
Something like: "ml local capture", which would use the application name, servers, and ports, look at the target server, and build deploy/ml-config.xml. It would probably make sense for this to capture particular features (db settings, range indexes) rather than the whole config, so that the substitutions could remain in place.
The list of environments is currently hard coded. Please allow users to dynamically specify the list of environments in build.properties.
Clearing out the pre-installed CA's takes twice as long as it takes to install MarkLogic. Would be nice to have the ability to clear the default CA's, and also load a group of CA's from a text file, similar to the functionality provided using the import feature on the ML CA admin page.
Add a word lexicon to ml-config.xml:
<word-lexicons>
<word-lexicon>http://marklogic.com/collation/</word-lexicon>
</word-lexicons>
Run bootstrap and all's well. Run it a second time and you'll get an error.
I'd like to be able to deploy to a cluster, which means specifying the number of forests per host in the cluster. I'd like to configure what I need in the properties file, maybe like this:
cluster-nodes=ml-box-1,ml-box-2,ml-box-3
forests-per-node=2
Seems that they weren't updated to handle the new load_properties function.
declare function c:register() as item()*
{
ch:use-view("welcome", ("html")),
lib:something-useful()
};
This says that when a client requests /user/register.html, use the view at /src/views/register.html.xqy.
This should use the view at: /src/views/welcome.html.xqy
The controller function code created by the scaffolding needs a () or something to not blow 500 errors.
Some shops will want to automate the source code repository retrieval process.
Can you please add a new command to retrieve the latest code from an SVN and VSS repository?
Example:
ml local getsvn
ml local getvss
Example build.properties for SVN:
svn-url=svn+ssh://[email protected]/project
svn-user=guest
svn-passwd=password
Example build.properties for VSS:
vss-dir=\\avalon\VSS\VSS_LSP
vss-user=guest
vss-passwd=password
forcing an app -reinitialization doesn't seem to work.
I would like to add read permissions to my content when I do a "ml dev deploy content".
It would be very useful to have the bootstrap process be able to create groups and assign hosts to those groups. The configuration would need to allow you to specify which group or groups you want your application installed on.
If you set up a rewrite element like this:
declare variable $c:ROXY-ROUTES :=
<routes xmlns="http://marklogic.com/appservices/rest">
<request uri="^/bug" endpoint="/my/module.xqy"/>
{
$def:ROXY-ROUTES/rest:request
}
</routes>;
and then hit a URL like this:
/bug?uri=foo.xml
then the rewriter drops the uri parameter.
Some of the catch clauses in deploy/lib/xquery/setup.xqy log the exception, but many do not. It would be very helpful to log at least some of these that are not currently logged. The two in particular that would have saved me some time are in setup:apply-database-settings() and setup:configure-http-server(). In general, I would think any failure that would cause a significant portion of the configuration not to succeed (like configuring a database or app server) should be logged.
This is missing :-(
Adding a geospatial-element-index to deploy/ml-config.xml does not let "ml {env} bootstrap" create that index
Currently if I want to create a CRUD style resource in Roxy I have to code the controller by hand. Please add a scaffolding command to do this for me.
ml create resource controller-name [format]
This should create all of the following functions and views for each with the specified format. If none
is supplied as a format then no views will be created.
declare function c:index()
{
};
declare function c:new()
{
};
declare function c:create()
{
};
declare function c:show()
{
};
declare function c:edit()
{
};
declare function c:update()
{
};
declare function c:destroy()
{
};
When bootstrapping, the setup script looks for any privileges referred to by deploy/ml-config.xml. For any it finds, it first deletes them, then recreates them based on the specification in deploy/ml-config.xml. I think the motivation for this is to avoid accidentally having a definition of an execute privilege that doesn't match what's in your config file. The problem is that if I have a role X with privilege A, and then I bootstrap a project that creates role Y with privilege A, then because A gets deleted and recreated, role X no longer has it. A couple possible solutions come to mind:
I'm thinking #1 would be sufficient. Comments?
I was trying to setup a element word lexicon to use with search:suggest and I tried adding the following which seemed to make sense:
http://www.stratml.net/PerformancePlanOrReport Description http://marklogic.com/collation/Though this didn't work. Can we get this added to the roxy deployer?
Thanks,
Andy...
would suggest changing
line 36
ml init
to
./ml init
Document setup and deploying.
In several environments it is against the security policy for a password to be saved in cleartext.
I would suggest a method for Roxy to prompt the user for the password. Also, when invoking recordloader and other child processes, the password should not be passed in where a "ps -ef" will show the password.
I'm working on a project where some of the configuration information is stored in the modules db, rather than the content db. In order to unit test this code, it is necessary to have a separate test-modules db, to prevent overwriting the configuration data in the production/dev db.
dcassel@madison:roxy $ ml init my-app --force
ERROR: Init has already been run. Use --force to rerun it.
ERROR: Missing environment for --force
On "ml local bootstrap", the script responds with "Bootstrap Complete" with no error message, even if there was an error. The error can be seen with "ml local bootstrap -v". Having an error in deploy/ml-config.xml (for instance, a mismatched closing tag) will trigger the problem.
Add support for bootstrapping Range Path indexes in 6.0. Should fail gracefully for < 6.0
Modify the deployer so that bootstrap will set up a scheduled task.
Note: I'm not sure whether it will work to have this as part of bootstrap, since the source code might not be deployed yet. If that doesn't work, then it might need to be a different command (ml {env} deploy tasks), but hopefully it can be part of bootstrap.
I want to set my default controller and function to /customresearch/search
But when I overwrite the default controller in config.xqy with "customresearch" I notice that using http://localhost:8040 doesn't work .. The reason being that the function is defaulting to main() which is fine for app builder but not for my case..
When I overwrite that as follows
config.xqy:
(: the default function that the URL http://server:port/ maps to :)
declare variable $DEFAULT-FUNCTION := "search";
/roxy/rewrite.xqy
<rest:request uri="^/([\w\d_\-]*)/?([\w\d_\-]*)\.?(\w*)/?$" endpoint="/roxy/query-router.xqy">
<rest:uri-param name="controller" default="{$config:DEFAULT-CONTROLLER}">$1</rest:uri-param>
<rest:uri-param name="func" default="{$config:DEFAULT-FUNCTION}">$2</rest:uri-param>
<rest:uri-param name="format">$3</rest:uri-param>
<rest:http method="GET"/>
<rest:http method="HEAD"/>
</rest:request>
Then all works fine. It might be an idea to build this into the default deployment and set DEFAULT-FUNCTION to main() so that appbuilder still works out of the box.
Should be able to specify which group of nodes to deploy content database on, in case cluster has separate E and D groups.
Add samples for range-element-attribute-index and geospatial indexes
Hi all,
I would like to file a feature request:
In the scenario of having several applications that offer different views over the same content, it will be extremely handy to be able to "synchronize" somehow the database properties (like range indexes and such).
Some ideas:
Regards,
Miguel
P.S: I could create a pull request for the first option if you think it's good enough.
When inserting documents into the modules database (and potentially the content database also), ignore the filesystem's path, and just place the files under "src" into the database.
/Users/pmcgowan/Documents/workspace/marklogic-roxy//custom/config/datasource-config.xqy
should be
/custom/config/datasource-config.xqy
It will be quite handy that during setup:apply-database-settings at xquery/setup.xqy, instead of doing nothing whenever a value it's not set, it would default to some other value.
Like this, we would make sure that the database configuration and the configuration are perfectly aligned after a bootstrap.
Example:
let $value := setup:get-setting-from-database-config-as-string($database-config, "directory-creation")
let $admin-config :=
if ($value) then
admin:database-set-directory-creation($admin-config, $database, $value)
else
admin:database-set-directory-creation($admin-config, $database, "manual")
After a bootstrap it won't matter if somebody changed the setting manually or somebody removed the line from the config file. The setting will default to Manual and automatic directory creation will be disabled.
Thanks!
Again: I could send a pull request if you think it's interesting for everybody else.
Much like we do for missing controllers we should show a handy, dandy error message for SEC-PRIV that directs the Roxy user where to add the missing privileges.
Adding a geospatial-element-child-index to deploy/ml-config.xml does not let "ml {env} bootstrap" create that index
A MISSING-VIEW (XDMP-MODNOTFOUND) error is thrown, when trying to use:
ch:use-view("welcome")
from a controller method.
WORKAROUND: This does work when you specify the controller in the use-view method as shown below:
ch:use-view("controller/welcome")
The latest version of MarkLogic, ML6, now support Path Range Indexes and this is not currently supported within Roxy.
It's not clear how you can call a view from a view.
ch:use-layout((), "xml"),
ch:use-layout("...", "html")
a bit more information how to use those helpers in Single page websites
if you add a self-closing collation element on a string element range index as follows
<range-element-index>
<scalar-type>string</scalar-type>
<namespace-uri></namespace-uri>
<localname>KeywordEN</localname>
<collation/>
<range-value-positions>false</range-value-positions>
</range-element-index>
then having run ml local bootstrap , the index is created with a unicode codepoint collation (http://marklogic.com/collation/codepoint). Running normal lexicon queries against it will fail saying the required index with the collation http://marklogic.com/collation was not found.
Does is make sense to change Roxy add the above root collation instead of the unicode collation when the collation element is not specified? Might be a safer default.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.