Describe the bug
Port exposed is non secure, port routed to is secure.

To Reproduce
./apply -u directory from the Kubernetes stuff. Pingdataconsole is not accessible via lb do to a protocol mismatch.

I submitted a easy little pull for this:
#74

Describe the bug

We have been unable to get PingFederate to correctly load the license file that we've mounted inside of the docker container. Instead we're being asked to upload it each time. We're trying to build a docker-compose.yml file to make local development easier.

To Reproduce

Have a valid 10.1 PingFederate license file and the following docker-compose.yml file:

version: "2.4"
services:
  pingdataconsole:
    image: docker.io/pingidentity/pingdataconsole:8.1.0.0
    depends_on:
      - pingdirectory
    ports:
      - "8443:8443"
    networks:
      - pingnet-internal
  pingfederate:
    image: docker.io/pingidentity/pingfederate:10.1.0
    command: wait-for pingdirectory:389 -t 300 -- entrypoint.sh start-server
    depends_on:
      - pingdirectory
    environment:
      - SERVER_PROFILE_URL=https://github.com/pingidentity/pingidentity-server-profiles.git
      - SERVER_PROFILE_PATH=baseline/pingfederate
      - STARTUP_COMMAND=/opt/server/bin/run.sh
    ports:
      - "9031:9031"
      - "9999:9999"
    networks:
      - pingnet-internal
    volumes:
      - pingfederate-out:/opt/out
      - ./PingFederate-10.1-Development.lic:/opt/in/instance/server/default/conf/pingfederate.lic
  pingdirectory:
    image: docker.io/pingidentity/pingdirectory:8.1.0.0
    ulimits:
      nproc:
        soft: 16384
        hard: 16384
      nofile:
        soft: 65535
        hard: 65535
    ports:
      - "1636:636"
      - "1443:443"
      - "1389:389"
    networks:
      - pingnet-internal
    volumes:
      - ./pingdirectory:/opt/in
      - pingdirectory-out:/opt/out
      - ./PingDirectory-8.1-Development.lic:/opt/in/instance/PingDirectory.lic
networks:
    pingnet-internal:
volumes:
  pingaccess-out:
  pingfederate-out:
  pingdirectory-out:

Then run: docker-compose up -d.

Expected behavior

PingFederate should not ask for the license file to be uploaded when logging in to https://localhost:9999/pingfederate/app

Describe the bug
Obfuscating a password using /opt/server/bin/obfuscate.sh doesn't work when using it for the ping fed datastore(ping directory ldap type) password. The password we used has multiple ! symbols maybe and we used the script without legacy flag.

It doesn't seem to work without legacy flag but works with legacy flag.

To Reproduce
Steps to reproduce the behaviour:

1. In ping federate engine server go to /opt/server/bin/obfuscate.sh
2. run /opt/server/bin/obfuscate.sh <password> to obfuscate the password.
3. set obfuscated password for the ldap datastore in ldap.properties
4. Start/Restart ping fed
5. Ping fed will no longer be able to connect to ldap data store using obfuscated password

Expected behavior
Obfuscated(non legacy) password should work in ping federate for ldap password

Screenshots
n/a

Environment:

Github Repo: cloned locally from
Docker Container: pingfederate pingidentity/pingfederate:10.1.0-alpine-edge
Cloud Environment: EKS (AWS)
Docker build

Additional Info
n/a

Hi,

I'm unable to build a container with a slightly older version of a product:

docker build --build-arg VERSION=5.1.1 -t pingidentity/pingaccess:5.1.1 .

This resolves the URL https://s3.amazonaws.com/gte-bits-repo/pingaccess-5.1.1.zip which presents:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>6E15465AC3BFB8A8</RequestId>
<HostId>
ktG8hMr+16SeByBzJocLudePtLBIfUE/ZVxh7Wb7Ttq6+eqIRlLJnhiew1zyobeXDm11xrV3/uA=
</HostId>
</Error>

Are the product distributions available within the S3 bucket?

Describe the bug
Followed steps here to set up a standalone deployment of PingFederate in a Kubernetes cluster. Service, Deployment and ConfigMap (shown here) were successfully created. I then tried to create an Ingress object to expose PingFederate outside the cluster based on instructions from here. Ingress object was successfully created, however it does not work as expected as it results in a ERR_SSL_PROTOCOL_ERROR on a web browser.

I created a custom server profile using the "modify a server profile using local directories" method with the following configs:

• Added a TLS cert for the domain I intend to use in pkcs12 format. I don't typically upload TLS certs into a container and usually define required TLS cert in my ingress definition yaml, but gave this a shot to try and solve my issue to no avail.
• Updated "Virtual Hostnames" configs (System > Virtual Hostnames) to include the two hostnames in pingfederate-standalone-ingress.yaml. These were pingfederate-admin.k8s.ilhaan and pingfederate.k8s.ilhaan for me.
• Updated "Redirect Validation" configs (Security > Redirect Validation) by editing the "Valid Domain Name (leading wildcard '*.' allowed)" field to be *.k8s.ilhaan and unchecking the "Require HTTPS" box. Goal with this config change is to have TLS handled by the Ingress object instead of the the PingFederate container (as is standard for kubernetes deployments).

After making the above mentioned changes, I built a Docker image using instructions from "Built into the image - Server Profile Anatomy" and re-deployed the Service, Deployment and ConfigMap. I used kubectl port-forward <pod-name> 9999:9999 and navigated to https://localhost:9999/pingfederate/app and logged in to verify config changes mentioned above.

I then created Ingress using pingfederate-standalone-ingress.yaml with various config options:

• With TLS cert from Kubernetes secrets. I have the same cert I uploaded to the PingFederate container available in k8s (this is what I use for all other application ingresses).
• Without TLS configs - I removed the tls config block from the ingress file (lines 30-37) to attempt creating the Ingress object without HTTPS in order to debug the issue.

None of the above mentioned Ingress configurations worked.

Expected behavior
Use Ingress endpoint to access PingFederate UI.

Environment:

• Github Repo: https://github.com/pingidentity/pingidentity-devops-getting-started/tree/master/20-kubernetes
• Docker Container: pingfederate:10.1.0
• Cloud Environment: On-prem, Using Rancher managed Kubernetes cluster

Questions

• How do I use Ingress objects to expose PingFederate outside my k8s cluster?
• Have I missed any other PingFederate specific configs in the server profile I created?

Describe the bug
The instructions in the getting started in step 3 should have you clone the forked version that was created in step 2.

To Reproduce
Steps to reproduce the behavior:

1. Review steps as documented here.

Expected behavior
Make the step clearer that the forked version should be used.

Screenshots
Not applicable.

Environment:

• Github Repo: pingidentity-devops-getting-started
• Docker Container: pingfederate
• Cloud Environment: NA

Additional Info
None.

Describe the bug
I'm not able to login and test OAuthPlayGround

To Reproduce
Steps to reproduce the behavior:

1. Go to https://localhost:9031/OAuthPlayground/welcome
2. Click on Implict
3. Provide User Credentials
4. See error

Logs

ERROR [SystemErr] Aug 26, 2019 2:21:05 PM com.unboundid.util.Debug debugLDAPResult
�[32mpingfederate_1 |�[0m INFO: level="INFO" threadID=148 threadName="Connection reader for connection 34 in pool 'pingdirectory' to pingdirectory:389" ldapSDKVersion="4.0.11" revision="3
4e39aab27ea4fb92659a6888933db08099c7e41" connectionID=34 connectionPoolName="pingdirectory" connectedTo="pingdirectory:389" readLDAPResult="SearchResult(resultCode=50 (insufficient access rig
hts), messageID=6, diagnosticMessage='The requested operation requires the config-read privilege')"

Screenshots

Environment:

• Github Repo: pingidentity-devops
• Docker Container: Docker-Compose/ 01-SimpleStack (EDGE)
• Local Environment : Windows 10

Additional Info
I was successfully able test all flow until last week, not sure if its related to new updates

Describe the bug
README for minikube references ./build.sh to build images. However that file does not exist under the minikube dir

To Reproduce
Steps to reproduce the behavior:

1. Go to 'pingidentity-devops-getting-started/tree/master/20-kubernetes/cloud/minikube'
2. View Readme
3. See reference to 'build images locally with ./build.sh'

Expected behavior
reference to file in current directory should exist

Additional Info
I believe the build.sh being references is in a different repo, and should be a link to pingidentity-docker-builds repo

Hi,
I understand that this not yet Production ready but it would be nice if you can version the container in the docker hub,at the moment the only tag available is latest, which makes testing a little harder as you dont know if a new image may be downloaded with breaking changes.

Thx in advance.

Describe the bug
I get a 404 Not found error when trying to access the Administrative console.

To Reproduce
Steps to reproduce the behavior:
This behavior can be seen when running the Ping Directory locally on docker:

1. docker run
   --name pingdirectory
   --publish 1389:389
   --publish 8443:443
   --detach
   --env SERVER_PROFILE_URL=https://github.com/pingidentity/server-profile-pingidentity-getting-started.git
   --env SERVER_PROFILE_PATH=pingdirectory
   pingidentity/pingdirectory
2. Access https://localhost:8443/console from the browser.

Expected behavior
The administrative console should be displayed

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

• Github Repo: pingidentity-devops
• Docker Container: pingdirectory
• Cloud Environment: Localhost and VMSS on Azure cloud

Additional Info
N/A

Currently, the PingDataConsole root page is the standard Apache Tomcat docs. And the console is deployed to a location different from the standard product deployment ("/console").

In order to make the landing page more foolproof, a few things would be nice.

1. Deploy the admin-console.war to ("/console")
2. Create a 302 redirect page "index.html" redirecting the user to "/console"

Currently, the server-profiles can optionally provide env_vars files to be later sourced by docker-build hooks.

As an example, the variable ROOT_USER_PASSWORD_FILE is provided through this mechanism.

However, when it's used (i.e. hook - 183-run-setup.sh), the setup fails if that variable isn't set.

At a minimum, these variables should be checked and an error created. At this point, there doesn't seem to be a need to force an env-vars file, as these variables can come in a number of different ways (i.e. env-vars, docker-run --env facility).

Is your feature request related to a problem? Please describe.
kustomize can be pretty difficult to support when you want a consistent versioned deployment. I ended up using ansible to build the deployments using kustomize but I don't particularly like the solution.

Describe the solution you'd like
helm in my opinion is a better way to manage deployments. Ideally I'd like to be able to do

helm install pingdirectory -f my-overrides.yaml pingidentify/pingdirectory

Is your feature request related to a problem? Please describe.
I haven't seen Ping documentation to expose PingFederate services in EKS using AWS ALB

Describe the solution you'd like
I would like to see Ping recommendation on how to expose PingFederate engine services to internet using AWS ALB as we would like to leverage AWS WAF capabilities.

I see there are multiple options and would like to see Ping recommendation on the challenges and limitations.

1. Use ALB ingress controller – This supports ALB. However it support backend services of type node port. Ping K8 services are of type ClusterIP.
2. Use nginx service type external – This will point to existing ALB.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
There is no documentation around using the PingFederate containers to run upgrades. Furthermore, while the upgrade utility is included in the containers, their default setup runs contrary to expectations made by the upgrade utility.

Describe the solution you'd like
A well developed process for upgrading a server profile between major versions. This solution would ideally be as automated as possible with the output being only the files necessary for a server profile.

Describe alternatives you've considered
Currently we have scripted a process that goes as far as:

1. Taking a server profile as input
2. Constructing a full PF_HOME using the container of the current, old version of PingFederate. We set the STARTUP_COMMAND to /bin/true so instead of starting the server the image just exits once it's completed the work of building out a full PF_HOME in /opt/out/instance.
3. Mounting that PF_HOME into a lightly customized PingFederate container of the new version
4. Running the upgrade utility against the old PF_HOME and having the new PF_HOME in /opt/out mounted to the host for review.

At this point we have to manually go through and only copy over the files that already exist in the server profile and work through the manual changes in the Upgrade Guide.

The lightly customized container involves

1. Adding the bash package
2. Stubbing out the license check hook
3. Setting the SERVER_HOME to /opt/out/pingfederate since the upgrade utility requires both the source and destination directories to be named pingfederate. The defaults use a directory named instance.
4. Setting the STARTUP_COMMAND to a script that navigates to /opt/out/pingfederate/upgrade/bin and runs the upgrade.sh script with the appropriate arguments.

Additional context
I've set up a git repo with minimal example of our work. You will have to provide a server profile in mounts/profile_in to get it working.

https://github.com/dkolb/pingfederate-upgrade

back-port the factored compound Docker build on top of the composite build to tease out common libraries and container flows to a couple of ancestor images
This add to the layers but improves maintainability

Build a script to build a server-profile based on a Ping Identity DevOps container instance or local instance.

I am following the documentation described in this below link.

https://pingidentity-devops.gitbook.io/devops/deploy/deployk8s/deployk8spd-clusters

I have setup replication between the PD pods in one EKS cluster. Works well.

Trying to understand why ports are different for each PD pod.

I would like to keep same ports for each PD pod.

Am I missing anything?

When starting the pingidentity/pingfederate:10.0.0-alpine-edge docker image, I see the following warning in the logs on start up:

----- Starting hook: /opt/staging/hooks/80-post-start.sh
run.sh: JAVA_HOME is not set.  Unexpected results may occur.
run.sh: Set JAVA_HOME to the directory of your local JDK or JRE to avoid this message.

It's just a warning but it's pretty easy to fix so it would be nice if the JAVA_HOME could be set out of the box.

Server profiles:

SERVER_PROFILE_GETTING_STARTED_PATH: getting-started/pingfederate
SERVER_PROFILE_GETTING_STARTED_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PARENT: GETTING_STARTED
SERVER_PROFILE_PATH: pf-aws-s3-clustering/pingfederate
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git

Describe the bug

When running pingdirectory in single cluster replication mode with 3 replicas the PD_STATE is never set to GENESIS on the SEED node and hence the userRoot ldif import is never run.

Replication is initialised and enabled correctly across the single cluster, but the data is never ingested.

To Reproduce
Steps to reproduce the behavior:
using default set up and setting these additional values for the configmap for the statefulset

  K8S_STATEFUL_SET_NAME: pingdirectory
  K8S_STATEFUL_SET_SERVICE_NAME: pingdirectory
  K8S_NUM_REPLICAS: "3"
  K8S_INCREMENT_PORTS: "false"
  ORCHESTRATION_TYPE: KUBERNETES
  REPLICATION_PORT: "8989"
  HTTPS_PORT: "1443"
  STARTUP_COMMAND: /opt/out/instance/bin/start-server
  MAX_HEAP_SIZE: 1024m
  LOCATION: eu-west-1
  VERBOSE: "false"
  PING_DEBUG: "true"

Expected behavior

when 03-build-run-plan.sh hook is run the the PD_STATE to be set to GENESIS and hence run the import ldif section.

Screenshots
replication enabled and cluster correctly initialised

Environment:

EKS using Docker build.
FROM pingidentity/pingdirectory:8.1.0.0-alpine-java11-edge

Additional Info
Add any other information about the problem here.

Is your feature request related to a problem? Please describe.
As a security best practice pods are not allowed to run as root users. Since Ping's containers need access to /opt directory it is preventing them from running on K8s cluster.

Describe the solution you'd like
Ability to run Ping containers as non-root users.

Describe alternatives you've considered
N/A

Additional context
https://kubernetes.io/docs/concepts/policy/pod-security-policy/#users-and-groups (this didn't work for the Ping containers)

The images, more specifically, hook 14-get-remote-server-profile.sh should provide the ability to specify a git tag when getting a remote server profile.

Example, a YAML should allow for either a SERVER_PROFILE_BRANCH or a SERVER_PROFILE_TAG to be specified.

      - SERVER_PROFILE_URL=https://github.com/pingidentity/server-profile-......git
      - SERVER_PROFILE_TAG=v1.1
      - SERVER_PROFILE_PATH=pingdirectory

In this case, the hook should:

    git checkout "${SERVER_PROFILE_TAG}"

There shouldn't be a reason for for both a TAG and BRANCH to be specified. If they are, we should either:

1. provide precedence to 1 of them (probably TAG) <--- provably preference
2. error out

Describe the bug
Windows doesn't allow filenames to contain certain characters which are allowed in Linux. There are several files which are invalid on NTFS, and as a result, users can't checkout branches with the files.

To Reproduce
Steps to reproduce the behavior:

1. Go to a Windows Machine

2. Clone pingidentity-devops-getting-started:

    $ git clone [email protected]:pingidentity/pingidentity-devops-getting-started.git	Cloning into 'ap126177-pingidentity-devops-getting-started'...
    remote: Counting objects: 9539, done.
    remote: Compressing objects: 100% (4413/4413), done.
    remote: Total 9539 (delta 5908), reused 7957 (delta 4751)
    Receiving objects: 100% (9539/9539), 18.10 MiB | 1.59 MiB/s, done.
    Resolving deltas: 100% (5908/5908), done.
    error: unable to create file docs/docker-images/ldap-sdk-tools/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingbase/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingdatagovernance/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingdatasync/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingdelegator/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingdirectoryproxy/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingdownloader/hooks/*.md: Invalid argument
    error: unable to create file docs/docker-images/pingfederate/hooks/*.md: Invalid argument
   

3. Note the errors complaining about files named *.md The * character is literal, and files named *.md are created on a Linux filesystem.

Expected behavior
I'd expect to be able to git clone <repo> and checkout the master branch

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

• Github Repo: pingidentity-devops-getting-started
• Docker Container: N/A: This is an error due to the way the files are named in git.
• Cloud Environment: N/A: This is an error due to the way the files are named in git.

Additional Info

• See (This link](https://brendanforster.com/notes/fixing-invalid-git-paths-on-windows/)
• Deleting the files that are literally named *.md resolves the issue.

Hello,

I am interested in mounting the PingDirectory DB Files as volume from the host into the Docker container, so that they persist after the container has stopped and can be mounted again in a new container. The problem is a do not find documentation on where these files are stored, neither could I find something inspecting a running container. Could you help adding a few lines in the readme about this ?

Describe the bug
I am setting the LICENSE_DIR but the pingfederate does not seem to use it.

To Reproduce
Steps to reproduce the behavior:

1. set LICENSE_DIR to a nondefault value
2. Run docker container
3. View logs

Expected behavior
I would expect that the license file directory would be respected or at least copied to /opt/out/instance/server/default/conf/

Logs (redacted some lines for brevity)

----- Starting hook: /opt/staging/hooks/03-build-run-plan.sh
Missing SERVER_ROOT: /opt/out/instance
################################################################################
#    Docker Image Information
################################################################################
                     IMAGE_VERSION : pingfederate-alpine-az11-10.2.0-210112-4797
                     IMAGE_GIT_REV : 4797657ce77b559f4576a2df671b3e818b5a9057
################################################################################
#    Directory Variables
################################################################################
                       LICENSE_DIR : /blah
################################################################################
#    License Key Info
################################################################################
                 LICENSE_FILE_NAME : pingfederate.lic
                LICENSE_SHORT_NAME : PF
                   LICENSE_VERSION : 10.2
         MUTE_LICENSE_VERIFICATION : --- empty ---

----- Starting hook: /opt/staging/hooks/17-check-license.sh
Verifying license with a network query to https://license.pingidentity.com.
You may opt out of this setting environment variable 'MUTE_LICENSE_VERIFICATION=yes'.
   License File: /blah/pingfederate.lic
        License: redacted
----- Starting hook: /opt/staging/hooks/80-post-start.sh
INFO: waiting for healthy admin before post-start..
2021-01-29 22:43:44,646  WARN  [org.sourceid.util.license.PingLicense]
**********************************************************
No license found for this server (/opt/out/instance/server/default/conf/pingfederate.lic)
You do not have a valid license configured. To obtain a valid license, please contact your vendor's Customer Support.
**********************************************************
----- Starting hook: /opt/staging/hooks/83-configure-admin.sh
PingFederate running...
INFO: new server found, must create admin
**********************************************************
No license file was found.
You do not have a valid license configured. To obtain a valid license, please contact your vendor's Customer Support.
**********************************************************
{
  "resultId": "invalid_license",
  "message": "The license for this PingFederate instance is missing or invalid."
}
error attempting to create admin
CONTAINER FAILURE: Error running 83-configure-admin.sh

Environment:
AWS, EKS, pingfederate:2012-alpine-10.2.0

**Additional Details: **

> printenv | grep LICENSE
LICENSE_FILE_NAME=pingfederate.lic
LICENSE_DIR=/blah
LICENSE_SHORT_NAME=PF
LICENSE_VERSION=10.2

> ls /blah/
pingfederate.lic

> cat /blah/pingfederate.lic
ID=redacted
WSTrustSTS=true
OAuth=true
SaasProvisioning=true
Product=PingFederate
Version=10.2
EnforcementType=redacted
Tier=redacted
IssueDate=redacted
ExpirationDate=redacted
DeploymentMethod=redacted
Organization=redacted
SignCode=redacted
Signature=redacted

Unable to deploy ping Directory pod in kubernetes cluster getting readiness probe error in pod Event logs.

Any help or any pointers would be much appreciated.

Hi,

With the latest images for PingFederate 10.0 I need to set the enviroment variables PING_PRODUCT and STARTUP_COMMAND otherwise the application doesn't start. I get the following error:

##################################################################################
#    Ping Product Info
##################################################################################
                      PING_PRODUCT : --- empty --- 
         Required: PING_PRODUCT
     Valid Values: i.e. PingFederate,PingDirectory
        More Info: Must be a valid Ping prouduct type
                          LOCATION : Docker 
                         LDAP_PORT : 389 
                        LDAPS_PORT : 636 
                        HTTPS_PORT : 443 
                          JMX_PORT : 689 
                      USER_BASE_DN : dc=example,dc=com 
         PD_ENGINE_PUBLIC_HOSTNAME : localhost 
          PF_ADMIN_PUBLIC_HOSTNAME : localhost 
         PF_ENGINE_PUBLIC_HOSTNAME : localhost 
          PA_ADMIN_PUBLIC_HOSTNAME : localhost 
         PA_ENGINE_PUBLIC_HOSTNAME : localhost 
                      ROOT_USER_DN : cn=administrator 
##################################################################################
#    JVM Details
##################################################################################
                     MAX_HEAP_SIZE : 384m 
                        JVM_TUNING : AGGRESSIVE 
Please resolve the validation issues!
CONTAINER FAILURE: Error running 04-check-variables.sh
CONTAINER FAILURE: Error running 01-start-server.sh

The documentation says these have default values and the defaults are what I need so I would rather not have to specify them.

Here are the server profiles that I am using:

SERVER_PROFILE_GETTING_STARTED_PATH: getting-started/pingfederate
SERVER_PROFILE_GETTING_STARTED_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PARENT: GETTING_STARTED
SERVER_PROFILE_PATH: pf-aws-s3-clustering/pingfederate
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git

Describe the bug
_pf-export-config.sh prints out localhost PF host independently on set PF variables:

Downloading config from https://localhost:9999...

It is confusing.

To Reproduce
Steps to reproduce the behavior:

1. Set envvar to something other than localhost
2. run _pf-export-config.sh

Expected behavior
Should print out actual host curl is connecting to.

Screenshots
NA

Environment:
NA

Additional Info
PA export script does the same thing.

First cut of the compound should be done on PingFederate first for simplicity. This should allow to quickly take PingAccess as a proof-point next. Then PingDirectory. Finally PingDataSync.

Hi everyone,

as you mentioned in this page: https://pingidentity-devops.gitbook.io/devops/dockerimagesref/pingdirectory#running-a-pingdirectory-container I can just only run this command in order to test PingDirectory...

As soon I'm doing this, a new image including a container will be created, but quickly after a few seconds (4-5sec) the container is stopped (Exited) and can't not be used... is there a issue or do I have to configure something before I run this command?

Ran the command (./docker-run.sh pingfederate) as Root in the server but unable to start.

Following is the logs entry we get from the docker image. Please let us know if any pre-requisite missed.

cp: can't create directory '/opt/out/instance': Permission denied

• apply_server_profile
• test -z https://github.com/pingidentity/server-profile-group-pingidentity-getting-started.git
• git clone https://github.com/pingidentity/server-profile-group-pingidentity-getting-started.git /opt/server-profile
  fatal: destination path '/opt/server-profile' already exists and is not an empty directory.
  CONTAINER FAILURE: Git clone failure
• die_on_error 78 'Git clone failure'
• errorCode=128
• exitCode=78
• shift
• test 128 -ne 0
• echo 'CONTAINER FAILURE: Git clone failure'
• rm -rf /opt/out/instance
• exit 78

Hi,

When I run the latest images for pingidentity/pingfederate:10.0.0-alpine-edge I get PingFederate to start but the admin console is not responding (blank page).

If I run the following command:

curl -u administrator:2FederateM0re -k 'https://localhost:9999/pf-admin-api/v1/cluster/status' --header 'x-xsrf-header: PingFederate'

I get:

{"resultId":"license_agreement_not_accepted","message":"License Agreement has to be accepted to use PingFederate."}

So it looks like it could be a EULA issue but I have the environment variable PING_IDENTITY_ACCEPT_EULA set to YES.

Here are the server profiles that I am using:

SERVER_PROFILE_GETTING_STARTED_PATH: getting-started/pingfederate
SERVER_PROFILE_GETTING_STARTED_URL: https://github.com/pingidentity/pingidentity-server-profiles.git
SERVER_PROFILE_PARENT: GETTING_STARTED
SERVER_PROFILE_PATH: pf-aws-s3-clustering/pingfederate
SERVER_PROFILE_URL: https://github.com/pingidentity/pingidentity-server-profiles.git

Here are my logs:

----- Starting hook: ./entrypoint.sh
Command: start-server
##################################################################################
#    Ping Identity DevOps Docker Image
#     IMAGE_VERSION: pingfederate-alpine-10.0.0-200122-2d7c
#     IMAGE_GIT_REV: 2d7cc00630d318fdbb8c4d6635f17e75f21c34df
#           STARTED: Thu Jan 23 00:45:34 UTC 2020
#          HOSTNAME: ip-10-13-2-72.ec2.internal
#        DOMAINNAME: ec2.internal
##################################################################################
copying local IN_DIR files (/opt/in) to STAGING_DIR (/opt/staging)

----- Starting hook: /opt/staging/hooks/01-start-server.sh

----- Starting hook: /opt/staging/hooks/02-get-remote-server-profile.sh
Getting SERVER_PROFILE_GETTING_STARTED
  git url: *** REDACTED ***
     path: getting-started/pingfederate
Cloning into '/tmp/server-profile'...
Getting SERVER_PROFILE
  git url: *** REDACTED ***
     path: pf-aws-ec2-clustering/pingfederate
Cloning into '/tmp/server-profile'...


----- Starting hook: /opt/staging/hooks/03-build-run-plan.sh
Missing SERVER_ROOT: /opt/out/instance
##################################################################################
#    
###################################################################################
#                      RUN_PLAN: START
###################################################################################

##################################################################################


----- Starting hook: /opt/staging/hooks/04-check-variables.sh
##################################################################################
#    Docker Image Information
##################################################################################
                     IMAGE_VERSION : pingfederate-alpine-10.0.0-200122-2d7c 
                     IMAGE_GIT_REV : 2d7cc00630d318fdbb8c4d6635f17e75f21c34df 
                          HOSTNAME : ip-10-13-2-72.ec2.internal 
                        DOMAINNAME : ec2.internal 
##################################################################################
#    Directory Variables
##################################################################################
                              BASE : /opt 
                            IN_DIR : /opt/in 
                           OUT_DIR : /opt/out 
                   SERVER_ROOT_DIR : /opt/out/instance 
                       STAGING_DIR : /opt/staging 
                         HOOKS_DIR : /opt/staging/hooks 
                SERVER_PROFILE_DIR : /tmp/server-profile 
                           BAK_DIR : /opt/backup 
                          LOGS_DIR : /opt/logs 
                       SECRETS_DIR : /usr/local/secrets 
                       LICENSE_DIR : /opt/out/instance 
##################################################################################
#    File Variables
##################################################################################
                     TOPOLOGY_FILE : /opt/staging/topology.json 
                    TAIL_LOG_FILES : --- empty --- 
                     COLORIZE_LOGS : true 
##################################################################################
#    Server Profile
##################################################################################
                SERVER_PROFILE_URL : *** REDACTED *** 
             SERVER_PROFILE_BRANCH : --- empty --- 
               SERVER_PROFILE_PATH : pf-aws-ec2-clustering/pingfederate 
             SERVER_PROFILE_UPDATE : false 
##################################################################################
#    DevOps User/Key
##################################################################################
         PING_IDENTITY_DEVOPS_USER : *** REDACTED ***  
          PING_IDENTITY_DEVOPS_KEY : *** REDACTED *** 
##################################################################################
#    License Key Info
##################################################################################
                 LICENSE_FILE_NAME : pingfederate.lic 
                LICENSE_SHORT_NAME : PF 
                   LICENSE_VERSION : 10.0 
##################################################################################
#    Product Startup
##################################################################################
                   STARTUP_COMMAND : /opt/server/bin/run.sh 
           STARTUP_FOREGROUND_OPTS : --- empty --- 
           STARTUP_BACKGROUND_OPTS : --- empty --- 
                           VERBOSE : false 
                        PING_DEBUG : false 
##################################################################################
#    Orchestration Info
##################################################################################
                ORCHESTRATION_TYPE : --- empty --- 
##################################################################################
#    Ping Product Info
##################################################################################
                      PING_PRODUCT : PingFederate 
                          LOCATION : Docker 
                         LDAP_PORT : 389 
                        LDAPS_PORT : 636 
                        HTTPS_PORT : 443 
                          JMX_PORT : 689 
                      USER_BASE_DN : dc=example,dc=com 
         PD_ENGINE_PUBLIC_HOSTNAME : localhost 
          PF_ADMIN_PUBLIC_HOSTNAME : localhost 
         PF_ENGINE_PUBLIC_HOSTNAME : localhost 
          PA_ADMIN_PUBLIC_HOSTNAME : localhost 
         PA_ENGINE_PUBLIC_HOSTNAME : localhost 
                      ROOT_USER_DN : cn=administrator 
##################################################################################
#    JVM Details
##################################################################################
                     MAX_HEAP_SIZE : 384m 
                        JVM_TUNING : AGGRESSIVE 



----- Starting hook: /opt/staging/hooks/05-expand-templates.sh
expanding files...
  - ./instance/server/default/conf/log4j2.xml.subst
  - ./instance/server/default/conf/tcp.xml.subst


----- Starting hook: /opt/staging/hooks/06-copy-product-bits.sh
Copying product bits to SERVER_ROOT


----- Starting hook: /opt/staging/hooks/07-apply-server-profile.sh
merging /opt/staging/instance to /opt/out/instance


----- Starting hook: /opt/staging/hooks/09-build-motd.sh
Successfully downloaded MOTD from https://raw.githubusercontent.com/pingidentity/pingidentity-devops-getting-started/master/motd/motd.json
Current /etc/motd
    
    ##################################################################################
                    Ping Identity DevOps Docker Image
    
           Version: pingfederate-alpine-10.0.0-200122-2d7c
       DevOps User: *** REDACTED *** 
          Hostname: ip-10-13-2-72.ec2.internal
           Started: Thu Jan 23 00:45:37 UTC 2020
    ##################################################################################
    
                         Security Warning
    
    The server profile being used to configure this Ping Identity Docker Image
    is for demo and documentation purposes only. The configuration contains default
    keys/credentials etc and is not suitable for production as it carries a substantial
    security risk.
    
    For additional information, please see
    https://github.com/pingidentity/pingidentity-devops-getting-started/blob/master/SECURITY.md
    
    
    ---- SUBJECT: PING_CONTAINER_PRIVILEGED will default to false
    Set PING_CONTAINER_PRIVILEGED to false and verify that everything works in your environment, the Ping Identity Docker images will default to non-root run time PID starting 2020-Feb-01
    
    Continued use of the Ping Identity Docker container constitute acceptance of the Terms of Use of Ping Identity software.
    For details, see:
    
        https://www.pingidentity.com/en/legal/subscription-agreement.html
    
    ##################################################################################



----- Starting hook: /opt/staging/hooks/10-start-sequence.sh
Initializing server for the first time

----- Starting hook: /opt/staging/hooks/17-check-license.sh
Pulling evaluation license from Ping Identity for:
              Prod License: PF - v10.0 
               DevOps User: *** REDACTED *** 
Successfully pulled evaluation license from Ping Identity



----- Starting hook: /opt/staging/hooks/18-setup-sequence.sh



----- Starting hook: /opt/staging/hooks/50-before-post-start.sh

Starting server in foreground: (/opt/server/bin/run.sh )
----- Starting hook: /opt/staging/hooks/80-post-start.sh
Listening for transport dt_socket at address: 9030
PingFederate running...

Describe the bug
As per the Gitbook, the SERVER_PROFILE_URL is a Git URL that is cloned by the container at the time of start-up. Since this configuration contains sensitive information, it is ideal that we store this in a private repo on organization's SVN (VSTS).
When using the private repo, the container fails to start because it is unable to authenticate to the private Git repo.

To Reproduce
Steps to reproduce the behavior:

1. create a Kube deployment for Ping Directory with environment variable (SERVER_PROFILE_URL) set to a private GitRepo.
2. Deploy the container.
   Expected behavior
   Should clone the Git Repo and start successfully.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

• Github Repo: [ping-directory, server-profile, docker-builds]
• Docker Container: pingdirectory
• Cloud Environment: Azure

Additional Info
Add any other information about the problem here.

Is your feature request related to a problem? Please describe.
Hey After running the ping-devops config it seems all the autocomplete scripts are set up to only install in the .bash_profile.

Describe the solution you'd like

I would like the option to target the file that the aliases are added to, or have some check that looks at what shell has executed ping-devops config then offer to add the aliases to that shells config file.

I know there are other popular shells out there like fish, perhaps support it as well would be helpful.

Describe alternatives you've considered
I could just let ping-devops config install the aliases to the .bash_profile, then copy the required changes over to my .zshrc.

Additional context
not really a big issue more a nice to have 😸

Additional info
Apologies in advance for being so verbose in my explanation. I appreciate your time to review this issue knowing that you must be very busy working on more critical issues.

Describe the bug
Unfortunately, I was unable to get the fullstack swarm to start as expected. (pingidentity-devops-getting-started/12-docker-swarm/fullstack.yaml).

Below is what I observed, and the steps I took to get the fullstack swarm to start as expected. I'm still not sure exactly why my solution worked, but it did, and I am stoked. :)

I discovered the following error in the service logs for the pingaccess, pingfederate-admin, and pingfederate swarm services (docker service logs -f ${service-name})

Example error from fullstack_pingfederate service logs:

fullstack_pingfederate.1.zfaznu425nhx@node1    | ----- Starting hook: ./bootstrap.sh
fullstack_pingfederate.1.zfaznu425nhx@node1    | ### Bootstrap
fullstack_pingfederate.1.zfaznu425nhx@node1    | ----- Starting hook: /opt/entrypoint.sh
fullstack_pingfederate.1.zfaznu425nhx@node1    | Command: sh -c wait-for pingfederate-admin:9999 -t 600 -- entrypoint.sh start-server
fullstack_pingfederate.1.zfaznu425nhx@node1    | ##################################################################################
fullstack_pingfederate.1.zfaznu425nhx@node1    | #    Ping Identity DevOps Docker Image
fullstack_pingfederate.1.zfaznu425nhx@node1    | #     IMAGE_VERSION: pingfederate-alpine-10.0.0-200126-9f12
fullstack_pingfederate.1.zfaznu425nhx@node1    | #     IMAGE_GIT_REV: 9f120fd51f74475c6bf5d6f741a92981be78fc55
fullstack_pingfederate.1.zfaznu425nhx@node1    | #           STARTED: Fri Jan 31 10:30:52 UTC 2020
fullstack_pingfederate.1.zfaznu425nhx@node1    | #          HOSTNAME: e114761ebc8f
fullstack_pingfederate.1.zfaznu425nhx@node1    | #        DOMAINNAME: 
fullstack_pingfederate.1.zfaznu425nhx@node1    | ##################################################################################
fullstack_pingfederate.1.zfaznu425nhx@node1    | Running command: sh -c wait-for pingfederate-admin:9999 -t 600 -- entrypoint.sh start-server
fullstack_pingfederate.1.zfaznu425nhx@node1    | Error: you need to provide a host and port to test.
fullstack_pingfederate.1.zfaznu425nhx@node1    | Usage:
fullstack_pingfederate.1.zfaznu425nhx@node1    |    host:port [-t timeout] [-- command args]
fullstack_pingfederate.1.zfaznu425nhx@node1    |   -q | --quiet                        Do not output any status messages
fullstack_pingfederate.1.zfaznu425nhx@node1    |   -t TIMEOUT | --timeout=timeout      Timeout in seconds, zero for no timeout
fullstack_pingfederate.1.zfaznu425nhx@node1    |   -- COMMAND ARGS                     Execute command with args after the test finishes

Troubleshooting
I ended-up commenting line 29 of 'fullstack.yaml' so that I could get the pingfederate-admin continer to simply start. I also added a published port of 22 so that I could "SSH" into the container and test the /opt/wait-for script, which worked as expected from the container interactive shell.

Next, I double-checked the Docker Compose documentation and tested using a different syntax for the service command override, which didn't fix the issue.

https://docs.docker.com/compose/compose-file/#command

command

Override the default command.

command: bundle exec thin -p 3000

The command can also be a list, in a manner similar to dockerfile:

command: ["bundle", "exec", "thin", "-p", "3000"]

Fix
Keeping the alternate command syntax, I next tried modifying the command to use the absolute path to the script, instead of invoking the script with 'sh -c'. With the below modifications to fullstack.yaml, I was able to successfully start the swarm. :)

line 7 original (generates similar error as above):

command: ["sh", "-c", "wait-for pingfederate:9031 -t 700 -- entrypoint.sh start-server"]

line 7 changed (works as expected):

command: /opt/wait-for pingfederate:9031 -t 700 -- entrypoint.sh start-server

line 29 original (generates similar error as above):

command: ["sh", "-c", "wait-for pingdirectory:389 -t 300 -- entrypoint.sh start-server"]

line 29 changed (works as expected):

command: /opt/wait-for pingdirectory:389 -t 300 -- entrypoint.sh start-server

line 48 original (generates similar error as above):

command: ["sh", "-c", "wait-for pingfederate-admin:9999 -t 600 -- entrypoint.sh start-server"]

line 48 changed (works as expected):

command: /opt/wait-for pingfederate-admin:9999 -t 600 -- entrypoint.sh start-server

Example success from fullstack_pingaccess service logs:

~/projects/pingidentity-devops-getting-started/12-docker-swarm$ docker service logs -f fullstack_pingaccess 
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | ----- Starting hook: ./bootstrap.sh
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | ### Bootstrap
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | ----- Starting hook: /opt/entrypoint.sh
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | Command: /opt/wait-for pingfederate:9031 -t 700 -- entrypoint.sh start-server
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | ##################################################################################
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #    Ping Identity DevOps Docker Image
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #     IMAGE_VERSION: pingaccess-alpine-6.0.0-200126-9f12
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #     IMAGE_GIT_REV: 9f120fd51f74475c6bf5d6f741a92981be78fc55
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #           STARTED: Fri Jan 31 12:06:22 UTC 2020
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #          HOSTNAME: 66a8c9659a89
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | #        DOMAINNAME: 
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | ##################################################################################
fullstack_pingaccess.1.l8vcjwx57xb9@node1    | Running command: /opt/wait-for pingfederate:9031 -t 700 -- entrypoint.sh start-server
...
...
fullstack_pingaccess.1.pl2n1k4jphfw@node2    | 2020-01-31T12:15:05,953  INFO [] com.pingidentity.pa.cli.RouterCLI - PingAccess started in 30s:214ms

/projects/pingidentity-devops-getting-started/12-docker-swarm$ docker service ls
ID                  NAME                           MODE                REPLICAS            IMAGE                               PORTS
h4rb1wl1phrh        fullstack_pingaccess           replicated          1/1                 pingidentity/pingaccess:edge        *:443->443/tcp, *:9000->9000/tcp
7f7s7g4yt64v        fullstack_pingdataconsole      replicated          1/1                 pingidentity/pingdataconsole:edge   *:8443->8443/tcp
pskuz8cndc4z        fullstack_pingdirectory        replicated          1/1                 pingidentity/pingdirectory:edge     *:1443->443/tcp, *:1636->636/tcp
tdl528pb94i7        fullstack_pingfederate         replicated          1/1                 pingidentity/pingfederate:edge      *:9031->9031/tcp
p41ew8jtwl2m        fullstack_pingfederate-admin   replicated          1/1                 pingidentity/pingfederate:edge      *:9999->9999/tcp

Environment:

• Github Repo: pingidentity-devops-getting-started/12-docker-swarm/fullstack.yaml
• Docker Container: n/a
• Cloud Environment: 3x CentOs 7.7.1908 Docker Swarm worker node VMs that I manage from my Linux Mint dev VM, which is the Docker Swarm manager.

Additional info
After discovering Ping Identity DevOps, I realized that I wanted/needed to learn Docker. I started learning Docker in December, 2019 by watching a really good Udemy course by Bret Fisher, a Docker Captain (https://www.bretfisher.com/). I am about 50% through the course, which last covered Docker Swarm Mode, and next up is Kubernetes (excited :) ).

I say all that so you will hopefully understand that I may not be using the most correct terms when I describe the issue I experienced, and also to say thank you for creating such awesome documentation! I simply cannot wait until we can hopefully start managing our PingFederate and PingAccess services using DevOps practices, instead of the very manual, error-prone method we use today to manage our considerably-sized clusters.

I've finally learned enough about Docker to start testing with Ping Identity products on my home lab where I have 3x CentOs 7.7.1908 Docker Swarm worker node VMs that I manage from my Linux Mint dev VM, which is the Docker Swarm manager.

Right now I am primarily focused on learning Swarm because, as it was interpreted by me from Bret Fisher's course, a small team that is new to DevOps using Docker will most likely be successful moving to Swarm first, since Swarm will cover most use-cases (80-20 rule), and then, things like Kubernetes, Docker Enterprise UCP, etc. can be added later, as the DevOps practices of the team matures.

My customer doesn't allow root access to the /opt directory. Trying to change this directory to something else either causes the build to fail or the operation of access.

We tried to work around this, but the rabbit hole went deep. I would like to see a fix for this.

https://github.com/pingidentity/docker-builds/blob/190d58ab44bf551d267685e12ce1bcdc7183ed6b/pingaccess/Dockerfile#L2

It would be nice add an --operationPurpose on the ldapseach liveness.sh checks, so this shows up in the logs.

Example:

  ldapsearch ... --operationPurpose docker-liveness-check ...

would create a log with:

[25/Mar/2019:19:06:45.869 +0000] SEARCH RESULT ...  purpose='docker-liveness-check'" ...

Is your feature request related to a problem? Please describe.
My k8s cluster does not have outside access and therefore retrieving a profile from a git url is not a valid option. Also it would make it hard to maintain multiple environment configs

Describe the solution you'd like
https://github.com/pingidentity/pingidentity-devops-getting-started/blob/master/20-kubernetes/04-clustered-pingaccess/env_vars.pingaccess-engine#L1

Expose all files via kubernetes configmaps

Describe alternatives you've considered

• Manually create configmap and expose it with file:/// syntax
• GCP or AWS buckets

Is your feature request related to a problem? Please describe.
When orchestrating Pingfederate containers, the console is pre-defined with a default admin password. To my knowledge there is no known public api endpoint that lets us change the password through automation. This creates manual effort to change the default password every time a container is orchestrated.

Describe the solution you'd like
Create a docker environment variable that allows us to pass a predefined password. Or expose a tool that allows us to generate the hash of the password and send it in the container. The process would feed the pingfederate-admin-user.xml which would have the password ready by the time the container starts

Describe alternatives you've considered
Alternative solution #1: Expose and API to allow us to change password once Pingfed console is up and API can start handling requests. To my knowledge no known API exists. Checked pings documentation as well as explored the /pf-admin-api/api-docs.
Alternative solution #2: Allow administrators to use a tool to hash a users password that can be imported into the pingfederate-admin-users.xml file. This could be similar to the obfuscate.sh script.

Describe the bug

SigningKeys added via the GUI and then exported configuration when re-deployed does not list them. neither updated connections are able to find it.

Not a BUG (neither a feature request).

To Reproduce
Steps to reproduce the behavior:

1. Add Signing Key
2. Update a connection with that signing key
3. export configuration archive
4. re-reploy updated instance/server/default/data

Expected behavior

• See the additional signingKey in the list after redeploy with updated /opt/in data.

• Connections to be able to use that signingKey

Environment:

• Github Repo: cloned locally from
• Docker Container: pingfederate pingidentity/pingfederate:10.1.0-alpine-edge
• Cloud Environment: EKS (AWS)
  • Docker build

Additional Info

I can see an entry added in this file

instance/server/default/data/config-store/org.sourceid.config.CoreConfig.xml

<!--Dsig keystore information: name and password-->
    <item name="DsigKeystoreName">ping-dsig.jks</item>
    <item name="DsigKeystorePassword">eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2Iiwia2lkIjoiRW1JY1UxOVdueSIsInZlcnNpb24iOiI5LjIuMC4xMCJ9.._Vc01Xs2XGCczUYA2vgnhA.56GD2bR9FsFcabI8CGadZ3cjVmWsPhkoIOMeczB2YG8.cpOanhUAV723mgT7h8EbUw</item>
    <!--Do not modify-->
    <!--Map of Dsig keystore aliases to passwords; the system uses this map
     to open private keys for generating signatures, X509
     authentication, etc. For each item, the name of the item maps
     to the alias of a private key, and the value of the item is the
     password to that particular key.-->
    <map name="DsigPrivateKeyPasswords">
        <item name="v8m5h9qrw85dgmndgv47k76xz">eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2Iiwia2lkIjoiRW1JY1UxOVdueSIsInZlcnNpb24iOiI5LjIuMC4xMCJ9..lnQoasV2J29-6KtVWlIWLQ.5of9PpT1aJtyrjWAX_LTz3SfghNxIFBLmU8_8tne0rU.FL732jjTyz1sTcAu9Msi2A</item>
        <item name="someGenVal">eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2Iiwia2lkIjoiRW1JY1UxOVdueSIsInZlcnNpb24iOiI5LjIuMC4xMCJ9..lnQoasV2J29-6KtVWlIWLQ.5of9PpT1aJtyrjWAX_LTz3SfghNxIFBLmU8_8tne0rU.FL732jjTyz1sTcAu9Msi2A</item>
    </map>

i.e this line

<item name="someGenVal">eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2Iiwia2lkIjoiRW1JY1UxOVdueSIsInZlcnNpb24iOiI5LjIuMC4xMCJ9..lnQoasV2J29-6KtVWlIWLQ.5of9PpT1aJtyrjWAX_LTz3SfghNxIFBLmU8_8tne0rU.FL732jjTyz1sTcAu9Msi2A</item>

I am not sure what other places need changing.

I have included the private key in JKS format in the root along with the other "keys and certs".

Is the required workload to create an additional record here

    <item name="DsigKeystoreName">custom-dsig.jks</item>

and then inside the DsigPrivateKeyPasswords to use

<item name="custom-dsig.jks-alias">$ping_fed_obfuscator_utility_keystore_pwd for custom-dsig.jks?</item>

Thanks for your help

Using the current server-profile data.zip for PingFederate, the ${username} seems to be blanked out in the resulting PasswordCredentialValidator, probably due to a substitution during the setup of the container.

Describe the bug
While running the export utility with the default PF IN config, passwords for the listed native users in the Password credential Validators resource type are being substituted out with a duplicate variable name. This is due to the users/passwords having the same json path, we need some method create unique variable names when the json path is the same for items in an array.

To Reproduce
Steps to reproduce the behavior:

1. in pf, have a simpleUsernamePasswordCredentialValidator with multiple users/passwords setup
2. Run the export utility using the default provided PF config file
3. in the rsulting subst export file, Go to the "resourceType": "/passwordCredentialValidators" section, find your simpleUsernamePasswordCredentialValidator , and see that the substitution variable names for all of the users passwords have the same variable name.

Expected behavior
A unique variable name should be generated for all unique password fields.

Screenshots
This is a screenshot of the resulting "resourceType": "/passwordCredentialValidators" after an export, showing the duplicate password substitution variable name for different users, because the json path is the same

Environment:

• Github Repo: pingidentity/pingidentity-devops-getting-started
• Product: ping federate

Additional Info

Describe the bug
A clear and concise description of what the bug is.
Login to https://localhost:9999/pingfederate/app fails without error message in the browser and redirects to https://localhost:9999/pingfederate/app?service=page/login

Note that https://localhost:9999/pf-admin-api/api-docs/ works: can use the APIs

To Reproduce
Steps to reproduce the behavior:

1. Deploy https://github.com/pingidentity/pingidentity-devops-getting-started/tree/master/11-docker-compose/00-standalone/pingfederate commit 8068051 with pingfederate docker image: 10.1.0
2. https://localhost:9999/pingfederate/app?service=page/login
3. Enter username and password, press enter

Expected behavior
It should successfully log in the admin console

Environment:

• Github Repo: https://github.com/pingidentity/pingidentity-devops-getting-started/tree/master/11-docker-compose/00-standalone/pingfederate commit 8068051
• Docker Container: pingidentity/pingfederate:10.1.0
• Cloud Environment: Laptop CentOs 7

Additional Info
Logs pingfederate:
pingfederate_1 | 2020-07-31 05:22:02,012 INFO [com.pingidentity.fsm.SessionTimeout] Application '/pingfederate' session timeout is 30 minutes.

The link from documentation to
https://pingidentity-devops.gitbook.io/devops/docker-builds returns 404.

thanks!

When running docker compose for the 11-docker-compose/02-replicated-pair example in getting started, I noticed that the script does not detect that I have these PING_IDENTITY_DEVOPS_USER and PING_IDENTITY_DEVOPS_KEY set. I have run the proper configuration commands and they are in my .bash_profile and are also shown in my env listing.

To Reproduce
Run docker compose in the directory 11-docker-compose/02-replicated-pair (locally).

Expected behavior
Should have detected PING_IDENTITY_DEVOPS_USER and PING_IDENTITY_DEVOPS_KEY values. Not sure why this is not seeing the values from my environment.

Screenshots
pingdirectory_1 | ##################################################################################
pingdirectory_1 | # DevOps User/Key
pingdirectory_1 | ##################################################################################
pingdirectory_1 | PING_IDENTITY_DEVOPS_USER : --- empty ---
pingdirectory_1 | PING_IDENTITY_DEVOPS_KEY : --- empty ---

Additional Info
When I looked at the code I see that you are doing a && test of both of these, meaning that both need to be missing before that section of the code will kick in, but you may want to have that as a || check so if either of these values are not set, it will use the temporary ones.
Line 23 of pingcommon/hooks/17-check-license.sh

Describe the bug
Cannot start my PingDirectory StatefulSet.

I am running into this condition but when I apply the suggested changes the PingDirectory PD_STATE will never be GENESIS due to this logic actually I think it is because of this instead.

I might be misunderstanding this...

To Reproduce
Steps to reproduce the behavior:

1. Start a new single PingDirectory cluster on Kubernetes
2. kubectl logs pods/pingdirectory-0 --follow
3. Scroll down to 'CONTAINER FAILURE: Resolve issues with pingdirectory Kubernetes cluster service, and restart'
4. sh into a container

hostname
# pingdirectory-0
hostname -i
# 10.1.1.99
getent ahosts pingdirectory-0.pingdirectory-cluster
# returns nothing, but this is what _podHostname is being set to; not sure why we are appending the service name...
getent ahosts pingdirectory-cluster
# returns our pod since the service has `publishNotReadyAddresses: true`
# 10.1.1.99    STREAM pingdirectory-cluster.pingidentity-dev.svc.cluster.local
# 10.1.1.99    DGRAM  pingdirectory-cluster.pingidentity-dev.svc.cluster.local

Expected behavior
I expect the container to not be so opinionated about its perceptions of my environment, lol. jk. I just want to boot this puppy up.

Environment:
K8s, single cluster, EKS

  ORCHESTRATION_TYPE: KUBERNETES
  K8S_STATEFUL_SET_NAME: pingdirectory
  K8S_STATEFUL_SET_SERVICE_NAME: pingdirectory-cluster

Architecture:

1. pingdirectory service (NodePort, port 443): This is used to expose the PingDirectory REST Service
   AWS ALB --> EKS Worker Node --> PingDirectory Pod (rest service)

2. pingdirectory-cluster service (ClusterIP: None, port 636): This is my headless service
   (K8S Cluster internal; headless service)

Describe the bug
Following the documentation to run the pingfederate standalone container with devops credentials does not result in a devops license being provisioned for the server

To Reproduce
Steps to reproduce the behavior:\

gcallaghan at grant-nf-mbp in ~/projects/devops/pingidentity-devops-getting-started/10-docker-standalone on master
$ ./docker-run.sh pingfederate

gcallaghan at grant-nf-mbp in ~/projects/devops/pingidentity-devops-getting-started/10-docker-standalone on master
$ docker container exec -it pingfederate /bin/sh

##################################################################################
                Ping Identity DevOps Docker Image

       Version: pingfederate-alpine-10.0.1-200305-d0dc
   DevOps User: <REDACTED>
      Hostname: a386f257fad2
       Started: Thu Mar  5 22:59:08 UTC 2020
##################################################################################

                     Security Warning

The server profile being used to configure this Ping Identity Docker Image
is for demo and documentation purposes only. The configuration contains default
keys/credentials etc and is not suitable for production as it carries a substantial
security risk.

For additional information, please see
https://github.com/pingidentity/pingidentity-devops-getting-started/blob/master/SECURITY.md

##################################################################################
PingFederate:a386f257fad2:/opt
> env
PF_ADMIN_DEBUG=false
SERVER_ROOT_DIR=/opt/out/instance
STARTUP_COMMAND=/opt/out/instance/bin/run.sh
LICENSE_FILE_NAME=pingfederate.lic
PF_DEBUG_PORT=9030
HOSTNAME=a386f257fad2
PA_ENGINE_PUBLIC_HOSTNAME=localhost
PF_ENGINE_PORT=9031
SERVER_PROFILE_UPDATE=false
PA_ADMIN_PUBLIC_HOSTNAME=localhost
PF_ADMIN_PORT=9999
SHLVL=1
MOTD_URL=https://raw.githubusercontent.com/pingidentity/pingidentity-devops-getting-started/master/motd/motd.json
HOME=/root
LOCATION_VALIDATION=true|Any string denoting a logical/physical location|Must be a string
PD_ENGINE_PUBLIC_HOSTNAME=localhost
PING_CONTAINER_GID=9999
PING_IDENTITY_DEVOPS_KEY_REDACT=true
PING_DEBUG=false
PF_ENGINE_PUBLIC_HOSTNAME=localhost
PS1=${PING_PRODUCT}:\h:\w\n>
PING_IDENTITY_DEVOPS_HOME=${HOME}/projects/devops
LOGS_DIR=/opt/logs
JMX_PORT=689
PF_ADMIN_PUBLIC_HOSTNAME=localhost
PING_IDENTITY_DEVOPS_TAG=edge
SERVER_PROFILE_PATH=getting-started/pingfederate
SERVER_PROFILE_DIR=/tmp/server-profile
STARTUP_FOREGROUND_OPTS=
TAIL_LOG_FILES=/opt/out/instance/log/server.log
ENV=/opt/.profile
BASE=/opt
DOLLAR=$
PING_CONTAINER_UNAME=ping
PING_IDENTITY_ACCEPT_EULA=YES
LDAP_PORT=389
SERVER_PROFILE_URL_REDACT=true
TERM=xterm
LDAPS_PORT=636
BAK_DIR=/opt/backup
PATH=/opt/java/bin:/opt:/opt/out/instance/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LICENSE_DIR=/opt/out/instance/server/default/conf
PING_CONTAINER_UID=9031
SERVER_BITS_DIR=/opt/server
PING_PRODUCT_VALIDATION=true|i.e. PingFederate,PingDirectory|Must be a valid Ping prouduct type
IMAGE_VERSION=pingfederate-alpine-10.0.1-200305-d0dc
SERVER_PROFILE_BRANCH=
STARTUP_BACKGROUND_OPTS=
LOCATION=Docker
PING_IDENTITY_DEVOPS_KEY=<REDACTED>
TOPOLOGY_FILE=/opt/staging/topology.json
COLORIZE_LOGS=true
HOOKS_DIR=/opt/staging/hooks
HTTPS_PORT=443
OPERATIONAL_MODE=STANDALONE
STAGING_DIR=/opt/staging
ORCHESTRATION_TYPE=
ACCEPT_EULA=NO
ROOT_USER_DN=cn=administrator
MAX_HEAP_SIZE=384m
SERVER_PROFILE_URL=https://github.com/pingidentity/pingidentity-server-profiles
PING_IDENTITY_DEVOPS_USER=<REDACTED>
IMAGE_GIT_REV=d0dcf74ec926f9295ec22a75b92e03cca3cfee35
JAVA_HOME=/opt/java
PWD=/opt
JVM_TUNING=AGGRESSIVE
PING_CONTAINER_PRIVILEGED=true
OUT_DIR=/opt/out
USER_BASE_DN=dc=example,dc=com
LICENSE_SHORT_NAME=PF
LICENSE_VERSION=10.0
CLUSTER_BIND_ADDRESS=NON_LOOPBACK
IN_DIR=/opt/in
SECRETS_DIR=/usr/local/secrets
VERBOSE=false
PING_PRODUCT=PingFederate
PING_CONTAINER_GNAME=identity
PF_ENGINE_DEBUG=false
PingFederate:a386f257fad2:/opt

$ docker container exec -it pingfederate /bin/sh

##################################################################################
                Ping Identity DevOps Docker Image

       Version: pingfederate-alpine-10.0.1-200305-d0dc
   DevOps User: <REDACTED>
      Hostname: a386f257fad2
       Started: Thu Mar  5 22:59:08 UTC 2020
##################################################################################

                     Security Warning

The server profile being used to configure this Ping Identity Docker Image
is for demo and documentation purposes only. The configuration contains default
keys/credentials etc and is not suitable for production as it carries a substantial
security risk.

For additional information, please see
https://github.com/pingidentity/pingidentity-devops-getting-started/blob/master/SECURITY.md

##################################################################################
PingFederate:a386f257fad2:/opt
> curl -k https://license.pingidentity.com/devops/v2/license
{ "error":"missing devops-user header" }PingFederate:a386f257fad2:/opt```


**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Environment:**
 - Github Repo: pingidentity-devops
 - Docker Container: pingfederate
 - Cloud Environment: docker

**Additional Info**
Add any other information about the problem here.

Describe the bug
Pingfederate starting fine on openshift and pod restart is crashing with below error.

----- Starting hook: /opt/staging/hooks/20-restart-sequence.sh
Restarting container

----- Starting hook: /opt/staging/hooks/21-update-server-profile.sh
cp: can't stat '/opt/out/instance/config/java.properties': No such file or directory
sed: /opt/out/instance/config/java.properties: No such file or directory
/opt/staging/hooks/21-update-server-profile.sh: line 47: dsjavaproperties: not found
CONTAINER FAILURE: Error running 21-update-server-profile.sh
CONTAINER FAILURE: Error running 20-restart-sequence.sh

MAX_HEAP_SIZE is affecting this which is supposed to run only for directory server.

To Reproduce
Steps to reproduce the behavior:

1. Setup pingfederate on openshift
2. Start the admin pod
3. kill the pod for restart
4. Pod crashes with above errors.

Expected behavior
MAX_HEAP_SIZE parameter conflicting with openshift. It should be some other variable to update dsjavaproperties.

Screenshots
Can be provided if required

Environment:

• Github Repo: docker-builds
• Docker Container: pingfederate
• Cloud Environment: OpenShift

Additional Info
Let me know if any other details required.

Is your feature request related to a problem? Please describe.
We deploy several JAR files in our server profiles. It'd be nice to be able to store these via git-lfs so that we aren't always cleaning binary blobs from the profile git repo as checkout times grown.

Describe the solution you'd like
As best I can tell, simply doing apk add git-lfs (or similar for CentOS and Ubuntu shims) and git lfs install in the pingbase image should be sufficient for supporting cloning repositories with files in LFS.

Describe alternatives you've considered
We are currently not using git LFS and just cleaning up previous JAR files from the repo using BFG. Our JAR files are also duplicated across the repo for each environment.

Additional context
No additional context.