esgf / esg-orp Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 6.0 24.08 MB

ESGF Openid Relying Party (ORP)

Home Page: http://esgf.org/esg-orp/

License: Other

Shell 10.51% Java 87.17% CSS 0.68% Python 1.64%

esg-orp's People

Contributors

Stargazers

Watchers

Forkers

pchengi alaniwi gavinmbell patrickhildreth-noaa kltsa santandermetgroup

esg-orp's Issues

Print warning message if IP configuration file is not found

The class AuthenticationByIPFilter should print a warning message if one of the supplied configuration files (las_servers.xml, las_servers_static.xml) is not found.

Move source of orp.provider.list

The ESG-ORP application is already using the included file esg-orp.properties for its configuration. The additional value:
orp.provider.list=/esg/config/esgf_known_providers.xml
should be read from the same location.

Adding IP ranges authorization feature

Missing log4j jar prevents esg-orp from starting

Recent changes in ivy build have made log4j jar not to be included in war file any longer

CORDEX licenses

Insert support for CORDEX Research and Commercial licenses.

Closing idle connections

Apparently there's an obscure way to close idle connections when using the Apache HTTP client, which we are not currently using...

Upgrading to Tiles 3 library

Problem with ';' in the redirection URL

I'm having the ';' exception when I'm accessing esg-orp. The app is responding with a redirection URL that "embeds" into the URL the jssession parameter instead of using the Set-Cookie header in the redirection.

INFO: Server startup in 25672 ms
org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL contained a potentially malicious String ";"
    at org.springframework.security.web.firewall.StrictHttpFirewall.rejectedBlacklistedUrls(StrictHttpFirewall.java:265)
    at org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(StrictHttpFirewall.java:245)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:193)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at esg.node.filters.AccessLoggingFilter.doFilter(AccessLoggingFilter.java:201)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at esg.orp.app.FilterAuthorizationEnforcerer.doFilter(FilterAuthorizationEnforcerer.java:54)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at esg.orp.app.RegistrationFilter.doFilter(RegistrationFilter.java:62)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at esg.orp.app.AccessControlFilterTemplate.doFilter(AccessControlFilterTemplate.java:67)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at esg.orp.app.AccessControlFilterTemplate.doFilter(AccessControlFilterTemplate.java:67)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

tomcat has many open filehandles on known providers file

lsof shows that the tomcat java process on data node is not closing filehandles on /esg/config/esgf_known_providers.xml

Example:

count open filehandles while avoiding double-counting with multi-threading

# lsof -P -n -p `pidof java` | cut -b 11-15,22- | sort -u | grep -c esgf_known_providers.xml
23

also set strace going

# strace -f -p `pidof java` -e '!futex' -o strace.out

In browser, load https://<data_node>/esg-orp/home.htm
In terminal, interrupt (ctrl-C) the strace
count the open filehandles again -- one more than previously:

# lsof -P -n -p `pidof java` | cut -b 11-15,22- | sort -u | grep -c esgf_known_providers.xml
24

see what the strace found

# grep known_providers strace.out 
2590  stat("/esg/config/esgf_known_providers.xml", {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  open("/esg/config/esgf_known_providers.xml", O_RDONLY) = 109
2590  stat("/esg/config/esgf_known_providers.xml", {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  stat("/esg/config/esgf_known_providers.xml", {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  stat("/esg/config/esgf_known_providers.xml", {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0

# grep -w 109 strace.out
[... snip ...]
2590  open("/esg/config/esgf_known_providers.xml", O_RDONLY) = 109
2590  fstat(109, {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  read(109, "<?xml version=\"1.0\" encoding=\"UT"..., 8192) = 2396
2590  fstat(109, {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  lseek(109, 0, SEEK_CUR)           = 2396
2590  fstat(109, {st_mode=S_IFREG|0644, st_size=2396, ...}) = 0
2590  lseek(109, 0, SEEK_CUR)           = 2396
2590  read(109, "", 8192)               = 0

with no close

Upgrading JDOM to latest 2.X version

The SAX parser pull-parser conflicts with Tomcat SAX parser

esg-orp/.classpath

Line 99 in 3da3bcf

The pull-parser library overrides the Tomcat SAX parser making the osg-orp application failing to load

Nov 08, 2018 2:37:10 PM org.apache.tomcat.util.descriptor.web.WebXmlParser parseWebXml
SEVERE: Parse error in application web.xml file at [file:/usr/local/apache-tomcat-8.5.34/webapps/esg-orp/WEB-INF/web.xml]
org.xml.sax.SAXNotSupportedException: not supported setting property http://xml.org/sax/properties/lexical-handler
       at org.gjt.xpp.sax2.Driver.setProperty(Driver.java:204)
       at org.apache.tomcat.util.digester.Digester.getXMLReader(Digester.java:871)

I have rename/removed the pull-parser-2.jar in the esg-orp's WEB-INF/lib directory and the esg-orp webapp is up and running.

Insert vertical scrollbar for license dialog.

Some users cannot register for CMIP5 Research/Commercial because their monitor is too small to show the full license document, and the buttons beneath.
The dialog needs to have a vertical scrollbar

leading space " " trips up entry / pulldown not obviou

File this under "usability nits"
Something I've come across and a user picked up on is that with the free openid entry, its easy to have a leading space character before the string and not realize it. This should be stripped.

Secondly, its not clearly obvious (in some browsers) that its a selector at all I suppose.

Upgrade Opensaml libraries

add means to specify authentication-only access control in policies file

For open datasets, currently the policies file allows for an attribute_type="ANY" rule, but we need to distinguish between two cases:
(a) resource is completely open (available for anonymous access in addition to all authenticated users)
(b) resource is available to all users but requires authentication

If I have correctly understood, #34 will allow for the possibility of the filters making SAML authz queries regarding anonymous access, prior to querying about authenticated access only if necessary, which is a necessary prerequisite for implementing the above but does not in itself provide the means for the authorizationService to respond differently in these two cases. Maybe @watucker can confirm this?

If this is correct, then maybe the required steps are:

(1) decide what syntax in the policies file is appropriate to represent cases (a) and (b) above

(2) implement support for it in the authorization service

(3) in the THREDDS web.xml in the ESGF stack, remove the policyFiles entry that makes THREDDS inspect the policies file directly (causing it to bypass the call to the authorizationService in the case of an attribute_type="ANY" rule), so that all authz decisions are handled by the authorizationService.

Secure the openid cookie

The user openid is stored in a cookie which currently is not secured - it can be sent through HTTP or HTTPS connections.

return a 400 in authorizationService if client does not supply valid XML

If the client does not supply a valid XML document, currently the authorizationService returns a 500 with a stack trace.

The issue can be simply reproduced by doing a GET instead of a POST.

The org.opensaml.xml.parse.XMLParserException reported in the traceback should be caught, and a 400 returned.

Allow access control for images and other files served through thredds

Cuurently all .gif and .jpg URLs served by a TDS are not applied any security, because of the "authenticationNOtRequiredPattern" value of the "authenticationFilter" in web.xml. The list of open URLs patterns should be changed to include ONLY those images that are part of the THREDDS distribution, namely:

<init-param>
  <param-name>authenticationNotRequiredPatterns</param-name>      <param-value>"[^?]*(/|(/admin/)(.*)|(/remoteCatalogService\?.*)|(/folder\.gif)|(/threddsIcon\.gif)|(/thredds\.jpg)|(/threddsSmall\.jpg)|(/unidataIcon\.gif)|(/unidataLogo\.gif)|(?&lt;=\.(html|xml|css|js))(\?.*)?)"</param-value>
  <!--  Alternatively, e.g.:
       <param-name>authenticationRequiredPatterns</param-name>
       <param-value>"[^?]*/fileServer/.*", "[^?]*/dodsC/[^?]*(?&lt;!\.html)(\?.*)?"</param-value>
  -->
</init-param>