Requirement of Bucket Usage and Performance metrics

Is your feature request related to a problem?/Why is this needed
Kubernetes PVC support storage usage promethues metrics via Kubelet exporter metrics. Similar information should be available for Object Buckets as well. It can include the following:

• Storage(Per Bucket):

  • usage
  • capacity ( Not sure if applicable )

• Performance(Per bucket):

  • Latency
  • Throughput
  • IOPS

Use cases:

• Showing consumption and aggregated information on monitoring tools( Grafana etc. )
• Finding out top n storage consumers
• Finding out the buckets with performance below or above certain thresholds
• Creating Prometheus alerts when the bucket consumption cross certain thresholds to notify users.

Describe the solution you'd like in detail

Required metrics to be implemented as additional metrics in already existing Prometheus exporters in Kubernetes ( Most Probably Kubelet )

Describe alternatives you've considered
None

Additional context
None

• SPEC parity: container-object-storage-interface-spec and KEP definitions need to be in sync

As the gRPC API is starting to become stable, the specification needs to be documented here so that plugin writers have a up-to-date reference

Enhancement

Is your feature request related to a problem?/Why is this needed
The created Secret is in a format my app can't process.

An Operator manages the application and I have no latitude to extract the JSON file and convert it into a format consumable by the application. In this case, this is the Kafka Operator by Confluent.

Describe the solution you'd like in detail
The enhancement request is to produce the bucket details (endpoint, credentials, region, etc.) as environment variables.

Environment variables are a common way to pass configuration to containers and will be supported in that particular case .

Describe alternatives you've considered
Another option would be to have the option to give the COSI driver a template file to craft the expected configuration file instead of using a hard-coded JSON.

Additional context
This ticket relates to the conversation here.

Enhancement

The objective of this Issue is to track and discuss the changes that are yet to be made or has already been made to the Protocol Field in Bucket Class. This is also to discuss and come up with ideas on how we should brand COSI. Few discussions have started within the COSI community to change the name "COSI" and the two candidates we came up with are OSI, KOSI.

I think both the names are really good and we should try to have both. It might sound strange at first but I will try to clarify my self.

OSI, can we have this name! My personal opinion will be that we are not quite mature enough for this name. As it will be "The Object Storage Interface" but we should definitely plan for it to be the future of COSI.

KOSI, I think this name suits the current position of COSI. We should plan for creating a standard for object storage in Kubernetes. Since anyway we are already doing it. Once we reach this goal then, we can plan for OSI.

For this to happen we will have to re-discuss the Protocol field in Bucket Class.

Parameters: Final Conclusion

BucketClass

provisioner: s3.amazonaws.com
protocol: 
  name: s3
  version: 2012-11-05
anonymousAccessMode: Public
releasePolicy: Retain
allowedNamespaces:
  - profiles
parameters: 
    region: us-east-1

COSI ---> KOSI ----> OSI

Define a protobuf structure for CreateBucket API.

Enhancement

Define Protocol Structure to marshal protocol specific information through gRPC between Sidecar and Driver even if the driver to back-end is marshaled as map[string]string.

Define the send and return gRPC messages transmitted by DeleteBucket() call and policies around Greenfield and Brownfield cases.

There should be a fake provisioner client autogenerated with the language bindings.

The current readme.md is out of date, and does not give guidelines for new persons interested in the project to easily get started and contribute.

Copy pasting request from rook/rook#13396

When creating buckets with a bucket claim like this:

apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClaim
metadata:
  name: my-bucket
spec:
  bucketClassName: ceph-bucket-class
  protocols:
    - s3

The bucket created is ceph is not my-bucket but instead a random auto generated name. In our use case we need to be able to create buckets with specified names. Is that possible somehow?