Add Debug category of tests which uses facilities like Minio admin API's to give preliminary information about the system and the Minio deployment before we start to debug.

All the functional tests using minio-go SDK should be removed from Mint and be maintained in minio-go SDK repo.

Once minio/minio-java#585 is implemented, following tests should move to mint full mode.

• Large file uploads.
• Unknown sized input stream of multipart test.
• ListObject test with 1050 file uploads.

New modes can be added here: https://github.com/minio/mint/blob/master/tests.sh#L159

Add correctness test cases based on aws-cli to Mint.

Tests should

• Access all the public Minio server APIs via aws-cli.
• Include one positive case and couple of boundary cases for each API.
• Have minimum file size uploads required to test multipart uploads.
• Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.

It is very slow to build a Mint docker image to check if everything is okay. We do a lot of iterations of writing code and testing, so this slowness is bad for us.

Why building the image is slow? The reason is that our docker script does a lot of tasks to download and install all dependencies. Since it is generally considered better to include the least possible number of RUN commands in a docker file, I suggest to make an exception for Dockerfile.dev.

In Dockerfile.dev, we can call RUN only twice:
*) The first RUN will init, download and install all dependencies.
*) The second RUN will copy our tests code.

Dockerfile and Dockerfile.dev should be similar, the only difference is that RUN in Dockerfile will be splitted into two RUNs in Dockerfile.dev

Right now its non object oriented. Let's make it ruby style and make it a class.

Add correctness test cases based on aws-ruby to Mint. Tests should be

• Named and commented to indicate the use case being tested.
• Access all the public Minio server APIs via aws-ruby.
• Include one positive case and couple of boundary cases for each API.
• Use streaming files as preferred way to generate files for testing. (This can be a ref: minio/minio-java#586)
• Use static files only in case where streaming can't be used.

• Need to provide options to select category of tests of run.

• Web app checks for pre-defined directories
• Pull logs from the directory and display via web interface.

$ git grep FileOfSize
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize65MB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize65MB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt1MB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt32KB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize128mb")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize65MB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize65MB")
run/core/minio-go/minio-go-tests.go:    fName := getFilePath("FileOfSize65MB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt32KB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt32KB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt32KB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSizeGt32KB")
run/core/minio-go/minio-go-tests.go:    fileName := getFilePath("FileOfSize100KB")
run/core/minio-java/FunctionalTest.java:  private static String FileOfSize1b=dataDir + "/FileOfSize1B";
run/core/minio-java/FunctionalTest.java:  private static String FileOfSize6mb=dataDir + "/FileOfSize6MB";
run/core/minio-java/FunctionalTest.java:  private static String smallFile=dataDir + "/FileOfSize1MB";
run/core/minio-java/FunctionalTest.java:  private static String largeFile=dataDir + "/FileOfSize11MB";
run/core/minio-java/FunctionalTest.java:      client.putObject(bucketName, objectNames[i],FileOfSize1b);
run/core/minio-java/FunctionalTest.java:    InputStream is = Files.newInputStream(Paths.get(FileOfSize6mb));
run/core/minio-java/FunctionalTest.java:    InputStream is = Files.newInputStream(Paths.get(FileOfSize6mb));
run/core/minio-java/FunctionalTest.java:    InputStream is = Files.newInputStream(Paths.get(FileOfSize6mb));
run/core/minio-java/FunctionalTest.java:    InputStream is = Files.newInputStream(Paths.get(FileOfSize6mb));
run/core/minio-js/test/functional-tests.js:    var _100kb = fs.readFileSync(dataDir + "/FileOfSize100KB")
run/core/minio-js/test/functional-tests.js:    var _11mb = fs.readFileSync(dataDir + "/FileOfSize11MB")
run/core/minio-js/test/functional-tests.js:    var _5mb = fs.readFileSync(dataDir + "/FileOfSize5MB")
run/core/minio-py/functional_test.py:    largefile = data_dir + "/" + "FileOfSize6MB"
run/core/minio-py/functional_test.py:    testfile = data_dir + "/" + "FileOfSize100KB"
run/core/minio-py/functional_test.py:    largefile = data_dir + "/" + "FileOfSize6MB"

Without these changes minio-go, minio-java, minio-js and minio-py won't run.

When no options are the test should run on play.minio.io

Steps to reproduce:

docker run -e S3_ADDRESS=172.18.0.1:9000 -e ACCESS_KEY=minio -e SECRET_KEY=minio123 minio/mint:alpha

Current behaviour:

The following files' permissions are changed.

	modified:   build.sh
	modified:   run.sh
	modified:   ../sdk-tests/build.sh
	modified:   ../sdk-tests/run.sh

Expected behaviour:

Permissions of file in repo shouldn't change.

Add client library example codes from minio-java, minio-py, minio-go, minio-js to functional tests category in Mint.

As described here minio/minio#4400 , check for last modified stamp in putobject part test.

May be even out of Mint's scope, but as described in the issue minio/minio#4236 , create internal documentation on measuring internode traffic, this will help as catch and reduce any unnecessary data transfer across nodes.

Inspired by minio/minio#4261 , we need tests for validating anonymous requests.

https://blog.docker.com/2016/09/build-your-first-docker-windows-server-container/

This will help us run minio-dotnet tests also via mint.

After minio/minio-py#547 is merged,

• Remove local copy of minio-py functional tests.
• Change the mint build process to pull tests from minio-py SDK (https://github.com/minio/minio-py) and execute those tests.

eg. verbose, quick, performance, stress, etc. Needs discussion.

Add Minio-Java functional tests to SDK tests.

It would be good if developers can have an option to choose to test with master or released versions of all packages. That way we can be confident that our next release won't cause any problems.

• Pull the Minio server master branch code from GitHub
• Build Minio server, deploy on local server in FS and Erasure mode.
• Run Mint against both the Minio server instances.
• Output logs to a time stamped directory, so web app can pick from the location.

mc already has tests in Mint : https://github.com/minio/mint/tree/master/run/core/mc but

• We need to add more test cases to check for bucket policy and enhance the test scenarios done with mint.

• Logging needs to be fixed for false alerts in error.log file

All the functional tests using minio-java SDK should be removed from Mint and be maintained in minio-java SDK.

Command to build mint docker image: docker build -t minio/mint:alpha . is stuck

Sending build context to Docker daemon 494.1 kB
Step 1/5 : FROM golang:1.7-alpine
 ---> ad5b584cc9b4
Step 2/5 : COPY . /home
 ---> Using cache
 ---> 90deb1576e2c
Step 3/5 : WORKDIR /home
 ---> Using cache
 ---> d016e95e2a92
Step 4/5 : RUN apk add --no-cache bash git openssh mailcap &&        go get -u github.com/minio/minio-go &&        chmod +x run.sh
 ---> Running in abb3d982e183
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
(1/14) Installing ncurses-terminfo-base (6.0-r7)
(2/14) Installing ncurses-terminfo (6.0-r7)
(3/14) Installing ncurses-libs (6.0-r7)
(4/14) Installing readline (6.3.008-r4)
(5/14) Installing bash (4.3.42-r5)
Executing bash-4.3.42-r5.post-install
(6/14) Installing libssh2 (1.7.0-r0)
(7/14) Installing libcurl (7.52.1-r2)
(8/14) Installing expat (2.2.0-r0)
(9/14) Installing pcre (8.38-r1)
(10/14) Installing git (2.8.5-r0)
(11/14) Installing mailcap (2.1.44-r0)
(12/14) Installing openssh-client (7.2_p2-r4)
(13/14) Installing openssh-sftp-server (7.2_p2-r4)
(14/14) Installing openssh (7.2_p2-r4)
Executing busybox-1.24.2-r13.trigger
OK: 37 MiB in 26 packages

There is no confirmation if the build is completed or not.

All the functional tests using minio-js SDK should be removed from Mint and be maintained in minio-js SDK repo.

Add concurrent tests to be able to catch potential race conditions in the code. The Minio binary has to be built with -race before running this test on the server.

All the functional tests using minio-dotnet SDK should be removed from Mint and be maintained in minio-dotnet SDK repo.

Add correctness test cases based on aws-java to Mint. Same test from minio-java, but use aws-java-sdk instead.

Add correctness test cases based on mc to Mint.

Tests should

• Access all the public Minio server APIs via mc.
• Include one positive case and couple of boundary cases for each API.
• Have minimum file size uploads required to test multipart uploads.
• Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.

Derived from minio/minio#4395 , some of the backends may not stand test of parallel uploads, we need add that case for generic tests.

Need to check whether the remote Minio server under test is reachable during the initialization.

Mint pulls functional test cases from minio-py sdk.

Minio-Py SDK tests should be updated to address the following

• Named and commented to indicate the use case being tested.
• Access all the public Minio server APIs via minio-py.
• Include one positive case and couple of boundary cases for each API.

Add correctness test cases based on aws-php to Mint.

Tests should

• Access all the public Minio server APIs via aws-php.
• Include one positive case and couple of boundary cases for each API.
• Have minimum file size uploads required to test multipart uploads.
  Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.

Currently the name of all the tests run are being printed on console, it should be done only on enabling the verbose option.

Need to see if the test for ListObject Parts exist and add the case.

Current minio-js has functional tests but we need to take a fresh look at the tests -

Tests should

• Be named to indicate the use case they are testing.
• Access all the public Minio server APIs via minio-js.
• Include one positive case and couple of boundary cases for each API.
• Have minimum file size uploads required to test multipart uploads.
• Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.

Currently all the tests are run by default. Option should be provided to run a particular category of tests.

minio/minio#4294: the gateway was returning ETags with double double quotes. It would be great if mint can detect this problem.

• mc + minio sdks + aws java + aws cli + aws php sdk run these in mint.
• Focus on correctness
• run this on minio server, minio gateway modes.

Current minio-java has functional tests but we need to take a fresh look at the tests -

Tests should

• Complete fast. No stress tests.
• Be named to indicate the use case they are testing.
• Access all the public Minio server APIs via minio-java.
• Include positive case and edge cases for each API.
• Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.

If the environment variable SERVER_ENDPOINT is not specified, it would be great if Mint can download the Minio server and runs it using a random port, then runs all sdk tests.

Add correctness test cases based on aws-ruby to Mint.

Tests should

• Access all the public Minio server APIs via aws-ruby.
• Include one positive case and couple of boundary cases for each API.
• Have minimum file size uploads required to test multipart uploads.
• Refer test files from a static data directory as in (https://github.com/minio/minio-go/pull/727/files#diff-81a03de42b955e14be71afa81768a78cR52). This helps keep the Mint container light weight and avoids the file creation in runtime, making the tests run faster.

Tests should not include

• Big file uploads.
• Multiple tests related to the same API.