Reported using May 2018 F2F meeting.

Currently there is no way to update the product URL for a printer in the IPP Everywhere database.

According to RFC 6763 section 4.1.1 (http://tools.ietf.org/html/rfc6763#section-4.1.1) a DNS-SD instance name may contain up to 63 octets of printable (non-control character) Net-Unicode characters. There is no restriction on the allowable characters for the first character. That includes the underscore ("_") and dash ("-") characters.

The 'ippfind' tool supports the following option and argument syntaxes:
ippfind [ options ] regtype[,subtype][.domain.] ... [ expression... ]
ippfind [ options ] name[.regtype[.domain.]] ... [ expression... ]

If 'ippfind' is used with a printer whose service instance name is "Smitty Normal Printer._ipp._tcp.local.", the 'ippfind' tool will assume the form being used is the first form, but the user will be wanting it to use the second form.

There is the "-n" or "--name" argument, which can be used as a workaround, but it would be better to collapse them into a single form and depend on the "-n" argument for all instances where the name is provided.

"image/jpeg" is optional for mono printer in ipp everywhere version 1.1

ipp-tests.sh
EXPECT document-format-supported OF-TYPE mimeMediaType IN-GROUP printer-attributes-tag WITH-VALUE "image/jpeg"

./runtests.sh bonjour-tests.sh "Example Test Printer"

See thread here: http://www.pwg.org/pipermail/ipp/2016/018995.html

Original message was this:

"While running the IPP Everywhere Self Certification suite against a printer that has no persistent clock, we saw several failures having to do with the state of some time related attributes. When I started digging into those attributes, I discovered that some attributes, such as "date-time-at-processing" or "date-time-at-completed", are defined as (dateTime | no-value) while others such as "date-time-at-creation" or "printer-current-time" are simply defined to be of type "dateTime". And yet the description of "printer-current-time" in even the latest draft of RFC 2911bis allowed the attribute to have the out-of-band value 'no-value'.

What is the right thing to do in these cases? And should it be acceptable if a Printer that lacks a clock reports 'no-value' for "printer-config-change-date-time"? If it is OK to return 'no-value' then I think tests I-9 and I-14 may need to be fixed."

IPP WG minutes (http://ftp.pwg.org/pub/pwg/ipp/minutes/ippv2-concall-minutes-20161005.pdf) gave this guidance for updating the IPP Everywhere Self Certification test suite:

⁃ Allow both no-value and unknown for date-time-at-xxx and
printer-current-time and printer-config-change-date-time and
printer-state-change-date-time

For IPP Everywhere 1.1, we want to add a check box to specify whether a firmware update may be needed for the IPP Everywhere functionality to be available.

Because Bonjour is a trademarked name, rename all of the scripts and tests to "dnssd" instead of "bonjour".

The current v1.0 self-certification tools for Windows depend on developer DLLs. They need to be rebuilt for the release DLLs.

Some printers don't support progressive JPEGs. They will declare their JPEG support via the IPP "jpeg-features-supported" attribute. But the "color.jpeg" sample file is a progressive JPEG, there is no non-progressive JPEG available, and the script doesn't check "jpeg-features-supported" before running its tests.

"image/jpeg" is optional for mono printer in ipp everywhere v1.1

D-2. JPEG Tests (mandatory) [FAIL]
RECEIVED: 7876 bytes in response
status-code = successful-ok (successful-ok)
EXPECTED: document-format-supported WITH-VALUE "image/jpeg"
GOT: document-format-supported="application/PCLm"
GOT: document-format-supported="application/octet-stream"
GOT: document-format-supported="image/pwg-raster"
GOT: document-format-supported="image/urf"
D-2.1 Print color.jpg with defaults [FAIL]

The "ippfind" tool takes a regular expression. If the service name provided to the script is actually a sub-string of another service name, then it will be run against all the matches. To fix this, the scripts should call "ippfind" with a regular expression wrapped in '^' and '$' to make sure there is an exact match, like so:

ippfind --name "^${NAME}$" "_ipp._tcp,_print" ...

B-4. IPP TXT values test: ippfind: Unable to execute "./bonjour-tests.sh": No such file or directory

As per PWG SC guidance, move the license for this project to Apache v2. Waiting on the move to CUPS 2.3 (ippeveselfcert 1.1 timeframe).

document-test default to 600dpi while printer advertises "pwg-raster-document-resolution-supported (1setOf resolution) = 300dpi,600dpi"

shouldn't to test both 300 and 600dpi?

Because HAVE_IDENTIFY_PRINTER isn't defined until I-10, add a new I-9.1 to check for Identify-Printer support, and then make the existing I-9 be I-9.2. That way, Identify-Printer will be tested.

The invocation of 'ippfind' should use the '-n' or '--name' option to make sure it can handle instance names that start with characters such as '_' that are used in the real world but are not supported by the currently used version of 'ippfind'

So, change this:

${IPPFIND} "$1._ipp._tcp.local." -x "${IPPTOOL}" -P "$1 IPP Results.plist" -I '{}' ipp-tests.test ;

to this:

${IPPFIND} --name "${1}" "_ipp._tcp.local." -x "${IPPTOOL}" -P "$1 IPP Results $(date +'%Y%m%d%H%M').plist" -I '{}' ipp-tests.test ;

(This also includes an additional date code element in the plist file name created to better support multiple runs.)

JPEG is not a MUST requirement in ipp everywhere 1.1

Unfortunately, it looks like the ippevesubmit program's -m option is broken. The read_string function incorrectly is stripping everything on a line after the first space.

When execut "B-5.1 HTTP Upgrade test" of dnssd-test.bat, after the certification tool sends OPTIONS request, the test Printer sends 101 Switching Protocols response.
However, the certification tool closes the current connection(does not start the ssl handshark), and re-send a OPTIONS request on a new connection.
The certification tool repeat above operations, and finally, this test failed.

Currently the Get-Printer-Attributes tests (I-10.x) verify strict conformance to STD92 regarding attribute group names in the "requested-attributes" operation attribute. After discussion in the IPP workgroup, we have decided to relax the conformance tests to simply require attributes in the group be returned but not throw an error for attributes not in the group. IOW, allow the group names to be treated as 'all' for a naive implementation since it does not affect interoperability, just performance.

The current IPP Everywhere v1.0 (and 1.1) tests unconditionally require pwg-raster-document-sheet-back, but that is only needed for duplex printing.

Test I-19. Cancel-Job Operation (completed job), which tries to cancel a completed job results in status-code = client-error-not-authenticated when the first job of the queue was sent by another user.

The solution would be to modify the preceding test-case (I-18. Get-Jobs Operation) to query only jobs from the requesting user by adding ATTR boolean my-jobs true.

As I said, the problem is only visible when multiple clients are interacting with the same printer and the first job in the queue doesn't belong to the requesting user.

Need to bring in the libcups, ippfind, and ipptool changes from CUPS 2.2.x into the ippeveselfcert project:

• libcups
• ippfind
• ipptool

• When I run "document-tests.sh", it complains about missing PWG Raster sample files and asks to download them from ftp.pwg.org/pub/pwg/ipp/examples/ . That's ok, but then...
• ...looking at that location, I see different versions: there are ones from 2011, and newer ones from 2015. Which one(s) to use?
• The "document-tests.sh" script itself has hard-coded a test for the "2011"-versions and bails out if only "2015"-versions are available.
• Also, the associated "documents-tests.test" file also contains the "2011" version names hard-coded....
• Finally, it is not clear if one should download the PWG raster samples for ALL resolutions (I guess the resolutions reported by the IPP printer to be supported should suffice?). To clarify that in the message printed to the user when the test fails would be helpful.

In lieu of a suitable IPP printer in my current environment I resorted to test against a locally installed CUPS-1.7.5 queue (knowing it is not supposed to pass most of the tests, but wanting to see in general how this testing would work). This is when I noticed the problem with the hard-coded versions, plus the following:

"document-tests.test":
    D-1. PWG Raster Format Tests (mandatory)                             [PASS]
ipptool: Filename "pwg-raster-samples-${RASTER_RESOLUTION}-20111130/black-1/color.jpg-4x6-black-1-600dpi.pwg" on line 68 cannot be read.
ipptool: Filename mapped to "pwg-raster-samples-1200dpi-20111130/black-1/color.jpg-4x6-black-1-600dpi.pwg".

That "line 68" is from "document-tests.test". It reads:

   FILE pwg-raster-samples-${RASTER_RESOLUTION}-20111130/black-1/color.jpg-4x6-black-1-600dpi.pwg

I'm not sure why the var ${RASTER_RESOLUTION} did not resolve in the output line? And why it seems to be "mapped to 1200dpi"? And why is the "600dpi.pwg" file suffix hard-coded throughout the "document-tests.test" file?

(Note, I've not run this on one of the required/recommended OS platforms, but on Debian Jessie.)

Currently there is no automated way of testing the self-certification tools. Goals:

• Use ippeveprinter to simulate a variety of printers, both conforming and non-conforming.
• Develop self-test scripts for Windows and macOS/Linux (that can be bundled with the tools) that runs them against ippeveprinter.
• Make this self-test script part of the release process, such that a clean install should be able to pass the self-test on any of our supported platforms.

If you install the IPP Everywhere Self Certification 1.0 Update 3 for Windows on a Windows 10 system (https://ftp.pwg.org/pub/pwg/tools/sw-ippeveselfcert10-20181023-windows.msi), and run "c:\Program Files\IPP Everywhere v1.0 Printer Self-Certification Tools\ipptool.exe", a dialog appears saying "The code execution cannot proceed because regex.dll was not found. Reinstalling the program may fix this problem."

From the April 2019 F2F:

• ippevesubmit help isn't aligned properly
• ippevesubmit should ask if this is for a print server
• ippevesubmit should write to "printer name.json"

The latest ippfind program supports a new --literal-name option that does an exact match instead of a regex match. This avoids problems with matching subsets of names and when the name contains regex characters like "[" and "]".

You must first download and extract the PWG Raster Format sample files from:

http://ftp.pwg.org/pub/pwg/ipp/examples/

Before you can run this script.

README:
The corresponding PWG Raster sample files (see link in the introduction) MUST be placed in a subdirectory of the "tests" directory. For example:

cd tests
curl http://ftp.pwg.org/pub/pwg/ipp/examples/pwg-raster-samples-300dpi-20150616.zip >temp.zip
unzip temp.zip
rm temp.zip
cd ..

The current README.md shipped with the tools is more of a manifest and less of a how-to document. Look at migrating as much of the portal documentation as possible to this file for future tools releases.

Per IPP WG meeting on May 9, 2016, the IPP tests should not require the document-numbers attribute be supported in "overrides-supported".

http://ftp.pwg.org/pub/pwg/ipp/minutes/ippv2-concall-minutes-20160509.pdf

The "ipp-tests.test" / "ipp-tests.test.in" files include a number of tests that test printing using JPEG files. This is now not acceptable because with IPP Everywhere 1.1 Printers are now only required to support JPEG if they are color printers. This should be refactored to use a PWG Raster file of the appropriate resolution.

Run below command to access "media-col-database" attribute in windows, it will show junk output.
ipptool.exe -vtL ipp:///ipp/print d:\Data\ipptool\get-printer-attribute.test
With "-X" option it works.

get-printer-attribute.test
{
VERSION 2.0
OPERATION Get-Printer-Attributes
GROUP operation-attributes-tag
ATTR charset attributes-charset utf-8
ATTR naturalLanguage attributes-natural-language en
ATTR uri printer-uri $uri

ATTR keyword requested-attributes media-col-database

}

This issue was not seen in "ipptool-20140328-windows.msi" tool.
Tested using HP Laserjet printers.
ipptool-issue-media-col-database.zip

On line 1112, at the start of Test I-27, there is the directive:
"DELAY 10"

There are some printers that have a longer than 10 second delay before reporting 'media-needed'. Suggest leaving the "DELAY 10" but adding a retry to the EXPECT statement on line 1125 so that it becomes:

EXPECT printer-state-reasons OF-TYPE keyword IN-GROUP printer-attributes-tag WITH-VALUE "/^media-needed/" REPEAT-NO-MATCH REPEAT-LIMIT 12

This puts an upper bounds on the wait but will list the retries. Polling is not desirable but is inevitable here. (IPP Eventing? Let's not go there here...) We could do DELAY 5 and double the REPEAT-LIMIT so that the polling frequency is 5 seconds.

This line:

Should say instead:

EXPECT finishings-supported OF-TYPE enum IN-GROUP printer-attributes-tag WITH-VALUE >3 DEFINE-MATCH HAVE_FINISHINGS

This way it will only define HAVE_FINISHINGS if "finishings-supported" contains a value other than 3 (none).

Right now the JSON output is not a complete JSON value, just the fragment for the array.

Add a dummy top-level definition to make it a complete JSON value that can be validated.

The commit c25124e included a file rename (LICENSE.txt -> LICENSE.md) and mods to the scripts/make-ippeveselfcert.sh, but the modifications didn't properly deal with the file renaming.

So:

1. fix the renamed file entry in the script
   -cp LICENSE.txt $pkgdir
   +cp LICENSE.md $pkgdir

2. Get rid of the script's copy for "tests/LICENSE.md" which doesn't exist in that subdirectory

3. Rename tests/README.txt to tests/README.md to make it be Markdown

Issue #50 relaxed the Get-Printer-Attributes requirements for not returning attributes that aren't in the group. Chris Rizzo from Xerox asked for this to still be available in a "strict" or "pedantic" mode.

If one runs the "bonjour-tests.sh" script against an IPP Everywhere Printer object hosted by a CUPS print server, it will fail because the resource path is not the best-practice "ipp/print" resource path. IPP Everywhere doesn't require that the printer's resource path is "ipp/print" so this should be removed from the test, to allow servers to pass. If there is a required resource path mandated in updates to IPP Everywhere this can be re-added.

Over in CUPS land we have discovered that some printers have issues with supporting the REQUIRED "requested-attributes" operation attribute with certain values. We need to add additional tests.

I developed a short test suite for CUPS, which can be adapted to the IPP tests for IPP Everywhere v1.1. It is attached.

get-printer-attributes-suite.test.txt

ipp-tests.bat "XXX-XXXX-0198D4"
I-10.6. Get-Printer-Attributes Operation (job-template) [FAIL]
RECEIVED: 4355 bytes in response
status-code = successful-ok (successful-ok)
EXPECTED: media-bottom-margin-supported
EXPECTED: media-col-supported
EXPECTED: media-left-margin-supported
EXPECTED: media-right-margin-supported
EXPECTED: media-size-supported
EXPECTED: media-source-supported
EXPECTED: media-top-margin-supported
EXPECTED: media-type-supported

In Table5 of IPP Everywhere v1.1, above expected attributes are defined as printer description attributes, not job template attributes. But this test item requies these attributes in job-template. Is it right?

This project doesn't have the latest stable revisions of ipptool and ippfind, which will resolve some problems with current sources that affect pending test updates (e.g. issue reported in istopwg/ippsample#2)

Ubuntu Results:
B-1. IPP Browse test: FAIL
B-2. IPP TXT keys test: FAIL
B-3. IPP Resolve test: FAIL
B-4. IPP TXT values test: FAIL
pdl is missing image/jpeg: 'application/PCLm,application/octet-stream,image/pwg-raster,image/urf'
"bonjour-value-tests.test":
Validate TXT record values using Get-Printer-Attributes [PASS]
B-5. TLS tests: SKIP
B-5.1 HTTP Upgrade test: SKIP
B-5.2 IPPS Browse test: SKIP
B-5.3 IPPS TXT keys test: SKIP
B-5.4 IPPS Resolve test: SKIP
B-5.5 IPPS TXT values test: SKIP
Summary: 10 tests, 0 passed, 4 failed, 6 skipped
Score: 60%

macOS Results:
B-1. IPP Browse test: FAIL
B-2. IPP TXT keys test: PASS
B-3. IPP Resolve test: PASS
B-4. IPP TXT values test: FAIL
pdl is missing image/jpeg: 'application/PCLm,application/octet-stream,image/pwg-raster,image/urf'
"bonjour-value-tests.test":
Validate TXT record values using Get-Printer-Attributes [PASS]
B-5. TLS tests: PASS
B-5.1 HTTP Upgrade test: PASS
B-5.2 IPPS Browse test: FAIL
B-5.3 IPPS TXT keys test: PASS
B-5.4 IPPS Resolve test: PASS
B-5.5 IPPS TXT values test: FAIL
pdl is missing image/jpeg: 'application/PCLm,application/octet-stream,image/pwg-raster,image/urf'
"bonjour-value-tests.test":
Validate TXT record values using Get-Printer-Attributes [PASS]
Summary: 10 tests, 6 passed, 4 failed, 0 skipped
Score: 60%

Based on our discussions at the last F2F, we will proceed with the development of a submission tool that creates JSON data appropriate for the new IPP Everywhere portal.

Need to update test suite and/or submission page to better support IPP Everywhere print servers, particularly for virtual print servers that have no associated hardware marking devices (e.g. process and send PDF file over some transport) and thus do not have supplies, media, or printer identification.

Reported using May 2018 F2F.

Currently the only way to add a new product to an existing submission is to resubmit using the original self-certification files. It would be nice to have a way to "clone" an existing submission to add new products.

On line 318 of "ipp-tests.test" and "ipp-tests.test.in", the line:

EXPECT overrides-supported OF-TYPE keyword IN-GROUP printer-attributes-tag WITH-VALUE "document-number" IF-DEFINED HAVE_PDF

ought to instead read:

EXPECT overrides-supported OF-TYPE keyword IN-GROUP printer-attributes-tag WITH-VALUE "document-numbers" IF-DEFINED HAVE_PDF

since the member of "overrides" / "overrides-supported" is "document-numbers" according to the IANA registry and PWG 5100.6.

Reported by Eren Rodriguez from HP Inc.

ipptool is currently unable to use a regular expression to match an octetString value.

Also, the regular expressions for these attributes seem to be broken...

Fix for 1.0 update 4 and 1.1.

1. git clone the latest test suite.
2. cd tests
3. curl http://ftp.pwg.org/pub/pwg/ipp/examples/pwg-raster-samples-300dpi-20150616.zip >temp.zip
4. rm temp.zip
5. cd ..
6. ./runtests.sh document-tests.sh "Example Test Printer"

Actual Results:
You must first download and extract the PWG Raster Format sample files from:

http://ftp.pwg.org/pub/pwg/ipp/examples/

Before you can run this script.

(Look for the @PWGRASTER_VERSION@ files...)

Test I-16 which attempts to Get-Job-Attributes until the Print-Job from I-12 is completed, uses a REPEAT-LIMIT of 30 on line 666:

EXPECT job-state OF-TYPE unknown|enum IN-GROUP job-attributes-tag COUNT 1 WITH-VALUE >6 REPEAT-NO-MATCH REPEAT-LIMIT 30

After the limit is reached, I-16 will not be repeated. This causes I-16 to pass even when the job from I-12 is in the processing state. In the case where I-16 (incorrectly?) passes with a processing job, if the job from I-12 is still not completed and no completed job exists, I-17 and I-18 can fail because they send a Get-Jobs request with which-jobs=completed and expect job attributes in the response. Similarly, in the case where I-16 passes with a processing job, if the job from I-12 is still not completed, the Cancel-Job operation from I-19 can also fail because it assumes the job from I-12 is completed and expects the returned status code of client-error-not-possible.

As Smith indicated for a different issue, polling is not desirable but is inevitable here. We found printers that could take about 60 iterations to pass but I advise a REPEAT-LIMIT of 100 so that we don't revisit this issue again for slower printer mechanisms.