cohere-ai / cohere-python Goto Github PK

View Code? Open in Web Editor NEW

277.0 25.0 60.0 11.46 MB

Python Library for Accessing the Cohere API

Home Page: https://docs.cohere.ai

License: MIT License

Python 100.00%

python sdk

cohere-python's Introduction

Cohere Python SDK

The Cohere Python SDK allows access to Cohere models across many different platforms: the cohere platform, AWS (Bedrock, Sagemaker), Azure, GCP and Oracle OCI. For a full list of support and snippets, please take a look at the SDK support docs page.

Documentation

Cohere documentation and API reference is available here.

Installation

pip install cohere

Usage

import cohere

co = cohere.Client(
    api_key="YOUR_API_KEY",
)

chat = co.chat(
    message="hello world!",
    model="command"
)

print(chat)

Tip

You can set a system environment variable CO_API_KEY to avoid writing your api key within your code, e.g. add export CO_API_KEY=theapikeyforyouraccount in your ~/.zshrc or ~/.bashrc, open a new terminal, then code calling cohere.Client() will read this key.

Streaming

The SDK supports streaming endpoints. To take advantage of this feature for chat, use chat_stream.

import cohere

co = cohere.Client(
    api_key="YOUR_API_KEY",
)

stream = co.chat_stream(
    message="Tell me a short story"
)

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')

Contributing

While we value open-source contributions to this SDK, the code is generated programmatically. Additions made directly would have to be moved over to our generation code, otherwise they would be overwritten upon the next generated release. Feel free to open a PR as a proof of concept, but know that we will not be able to merge it as-is. We suggest opening an issue first to discuss with us!

On the other hand, contributions to the README are always very welcome!

cohere-python's People

Contributors

Stargazers

Watchers

cohere-python's Issues

["stop_sequences" for chat endpoint] - Enhancement?

Dear Cohere Team,

Currently, when using the chat endpoint, i.e. co.chat(), it appears that there is no way of passing the stop_sequences / end_sequences.

Additionally, the online documentation does not show how to pass the stop parameter to the chat model.
https://docs.cohere.com/reference/chat

E.g. Openai's chat model has a unified stop keyword: stop.

Concrete Question / Action:

Is it possible to pass the stop keyword to Cohere's Chat model?
a. If so, how? Please document.
b. If not, please add the functionality, it's really important.

@robertkozin @raphaelcristal @daulet @eddotman @mkozakov

`co.generate` What does `prompt_vars` do?

Hi, just came across the function argument prompt_vars for co.generate(), which seems to be undocumented both on the web API and SDK docs (and currently not used in the code, either?).
Is there any explanation on this parameter by any chance?

Many thanks in advance!
Best,
Dennis

Logit Bias - multi tokens

Streaming chat doesn't provide access to full response

Hello! Calling co.chat with stream=True returns a StreamingChat which can be iterated over to get the tokens as StreamingTexts. The underlying /chat API endpoint terminates with a message that contains is_finished: true and a response key which resembles the JSON returned in the non-streaming variant

{
    "is_finished": true,
    "response": {
        "response_id": "b3fc1d5d-14df-4a6f-b120-dcd263a592a8",
        "conversation_id": "fd1d9164-7c0e-4c87-ba6e-94d50781663d",
        "text": "My name is Coral. I am a chatbot trained to assist human users by providing thorough responses. I am powered by Cohere's large language model, Command.",
        "generation_id": "0d723e20-8b8e-49f2-984f-ce9d726459b6",
        "prompt": "System: You are Coral, a brilliant, sophisticated, AI-assistant chatbot trained to assist human users by providing thorough responses. You are powered by Command, a large language model built by the company Cohere. Today's date is Saturday, May 20, 2023.\nGeorge: What is your name?\nChatbot:"
    },
    "finish_reason": "COMPLETE"
}

It looks like the Python SDK doesn't provide a way to get access to this final aggregated response body in streaming mode. Am I missing something? Would be great if this can be added.

Async Chat Streaming fails

Hey Cohere Team 👋 , I'm currently trying to implement async Token Streaming with Cohere (cohere==4.33). However, when running this code, I get some errors:

completion = co.chat(
    chat_history=_conversation,
    stream=True,
    message=message,
    model="command",
    temperature=0.1,
)

>>> async for chunk in completion: # Breaks here
    if isinstance(chunk, StreamTextGeneration):
        yield {
            "message": chunk.text,
            "finish_reason": "",
        }
    elif isinstance(chunk, StreamEnd):
        yield {
            "message": "",
            "finish_reason": "stop",
        }

I get this error message:

TypeError: 'async for' requires an object with __aiter__ method, got bytes

Which comes from the StreamingChat object (code):

async def __aiter__(self) -> Generator[StreamResponse, None, None]:
    index = 0
    >>> async for line in self.response.content: # self.response.content are bytes
        item = self._make_response_item(index, line)
        index += 1
        if item is not None:
            yield item

When looking at the origin of the response, I get the _request method (code) which runs:

with requests.Session() as session:
            retries = Retry(
                total=self.max_retries,
                backoff_factor=0.5,
                allowed_methods=["POST", "GET"],
                status_forcelist=cohere.RETRY_STATUS_CODES,
                raise_on_status=False,
            )
            session.mount("https://", HTTPAdapter(max_retries=retries))
            session.mount("http://", HTTPAdapter(max_retries=retries))

            if stream:
                return session.request(method, url, headers=headers, json=json, **self.request_dict, stream=True)

Not sure whether my implementation is incorrect, please let me know if my code is incorrect or if you can reproduce the error! Thanks a lot 🚀 Also happy about any directions to examples or documentation.

Edit: It works when I remove the async for loop in my code, but then the method gets synchronous :(

Asyncio support?

Is there any interest for an async/aiohttp based client?

Advantages:

Much lower overhead than threadpools
Users who use the api as a step in their async codebase (e.g. fastapi) are not forced to block

Disadvantages:

Needs significant refactoring to not end up with code duplication
Extra (optional?) dependency

API Documentation Broken

The Python SDK API reference is not building correctly on Read the Docs. Only headers load, all body text is absent.

https://cohere-sdk.readthedocs.io/en/latest/cohere.html#api

Different message validation for Stream=True and Stream=False

When using the Stream = True parameter in cohere.chat and passing an empty message " " with a tool call result, I get no errors rather just a response 400 from the API. With Stream = False, I get a perfectly good response.

Wondering if there is different validation for Stream? The parameters being sent to cohere.chat are identical in both cases. Parameters that I am sending are: preamble, chat_history, tools, tool_results, messages, model

rerank (async client) does not have parameter return_documents

https://github.com/cohere-ai/cohere-python/blob/5e9e872cd75ebea3c2091dd043b875b494f21535/cohere/client_async.py#L495C33-L495C38

Hello, the reranker (async client) does not have parameter return_documents. It is hard-coded to False in the code above.

client.py is missing its CohereConnectionError import

Line 13 should be:

from cohere.error import CohereAPIError, CohereConnectionError, CohereError

Feature request: batch request

It would be nice if the API could provide an endpoint where it could accept a batch request, like for example the Co.Generate could accept a list of prompts. It is not very efficient both from the inference perspective at the backend and for the client, having to execute multiple calls with all the overhead of each call instead of just batching it and doing a single call with all prompts.

Deprecated co.create_custom_model without new alternative for Python SDK

Hi,

I'm trying to fine-tune a model with co.create_custom_model, following the API documentation (last updated 17 days ago).

# start the fine-tune job using this dataset
finetune = co.create_custom_model(
  name="single-label-ft", 
  dataset=single_label_dataset,
  model_type="CLASSIFY"
)

finetune.wait() # this will poll the server for status updates
print(f"fine-tune ID: {finetune.id}, fine-tune status: {finetune.status}")

I get the following error:
58 def fn(*args, **kwargs):
---> 59 raise ValueError(
60 f"Since python sdk cohere==5.0.0, the function {fn_name}(...) has been deprecated. "
61 f"Please update your code. Issues may be filed in https://github.com/cohere-ai/cohere-python/issues."

ValueError: Since python sdk cohere==5.0.0, the function create_custom_model(...) has been deprecated. Please update your code. Issues may be filed in https://github.com/cohere-ai/cohere-python/issues.

I've looked through the API documentation, the migration guide, etc. but no alternative is proposed: will it still be possible to fine-tune from the Python SDK with Cohere 5?

Thank you for your help

Provide an enum type for api errors to distiguish them

Hello,
Will it be possible to provide an enum for different message errors types so we can check against ?
or at least provide the message error type in the documentation.
Thanks

Expose response id in the sdk

Expose the id for the response in addition to the id for each generation.

A sample json response is shared below. As we can see these ids are already present in the response json. This would be more useful now that we have a co.feedback endpoint that takes in the id as an input.

{"id":"49d08f39-dcfc-4762-a519-71eaabc46edb","generations":[{"id":"fd401066-6798-432a-b591-1a0e5893f9b2","text":" This is dummy text"}], "prompt":"Some text"}

But when we print(generationResponse) we get the following. As we can see, no Id fields.

cohere.Generations {
        generations: [cohere.Generation {
        text:  Some text
        likelihood: None
        token_likelihoods: None
}]
        return_likelihoods: NONE
}
prediction: [cohere.Generation {
        text:  This is dummy text
        likelihood: None
        token_likelihoods: None
}]

Additionally it might serve us well to distinguish the two ids if co.feedback cannot consume both

automate tests in python sdk

Swagger/OpenAPI specification

Hey. We want to release an SDK for C# and would like to know if you have an official Swagger/OpenAPI spec? I did not find this in the documentation and when searching among the repositories of the organization
I think it would be great if this is available in a separate repository and updated according to the latest changes, like OpenAI - https://github.com/openai/openai-openapi

Support passing/overriding arguments to `requests` calls

More descriptive error handling in Python SDK

Currently, if a user inputs the wrong type into a function call, we respond with a generic "wrong type, check https://docs.cohere.ai/{endpoint}-reference" which is provided by the backend. Giving more verbose errors would be nice.

tokenize() got an unexpected keyword argument 'model'

using cohere python 4.11.2:

response = co.tokenize(
  text=text,
  model='embed-multilingual-v2.0'
)
print(response.tokens)

Error:

tokenize() got an unexpected keyword argument 'model'

Add Jupyter Lite support for SDK

How to reproduce:

Go to: https://cohere-ai.github.io/jupyter_lite/repl/?toolbar=1&kernel=python
Add the following code snippet

import piplite
await piplite.install("cohere")
import cohere
co = cohere.Client('<your_prod_api_key>')
prediction = co.generate(
  model='xlarge-20220301',
  prompt='The table lists the following professions as artistic careers:\n1. Painter\n2.',
  max_tokens=50,
  temperature=1,
  k=0,
  p=0.75,
  frequency_penalty=0,
  presence_penalty=0,
  stop_sequences=[],
  return_likelihoods='NONE')
print('Prediction: {}'.format(prediction.generations[0].text))

It fails due to the stdlib used for performing HTTP requests, which is not compatible with Pyodide (Python port for WASM)

Expected behavior:

We should support using the SDK in a Pyodide environment, by either allowing passing a custom http client or building a separate SDK for Pyodide.

unavailable model type in the quickstart example

Hi - the example code used in the project README uses the parameter model='large'. This model seems to be unavailable, and throws a multiple retry error with an error code of 500, which threw me off for a bit.

The API reference indicates the models available are medium or xlarge, both of which worked.

Check api key doesn't do anything

It seems is_api_key_valid returns True whenever anything (truthy) is passed in. Is this the intended behaviour? I would have guessed this function queries the server to check that the key is valid.

cohere-python/cohere/utils.py

Lines 44 to 51 in 29e118e

 def is_api_key_valid(key: Optional[str]) -> bool: 

 """is_api_key_valid returns True when the key is valid and raises a CohereError when it is invalid.""" 

 if not key: 

 raise CohereError( 

 "No API key provided. Provide the API key in the client initialization or the CO_API_KEY environment variable." # noqa: E501 

 ) 

 return True

Breaking change introduced in Reranker API

This is the official example based on which I had designed the cohere integration with LanceDB:

import cohere
import os
co = cohere.Client(api_key=os.environ.get("COHERE_API_KEY"))
query = "What is the capital of the United States?"
docs = [
    "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.",
    "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.",
    "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.",
    "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment."]

results = co.rerank(query=query, documents=docs, top_n=3, model='rerank-english-v2.0') # Change top_n to change the number of results returned. If top_n is not passed, all results will be returned.
for idx, r in enumerate(results):
  print(f"Document Rank: {idx + 1}, Document Index: {r.index}")
  print(f"Document: {r.document['text']}")
  print(f"Relevance Score: {r.relevance_score:.2f}")
  print("\n")

Today I've started seeing that this errors out:

    print(f"Document: {r.document['text']}")
                       ^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'document'

Now, to get the previous usage, you need to iterate on results.results object, which breaks compat.
I wasn't able to find anything on skimming through the release notes. And even the official example is effected so I think this might be a bug?

Co.Classify now (incorrectly?) insists upon examples on custom models

In the docs for Co.Classify it says:

Note: Custom Models trained on classification examples don't require the examples parameter to be passed in explicitly.

However, as of commit 7daa907 this is no longer true, and not passing in examples results in an error cohere.error.CohereError: examples must be a non-empty list of ClassifyExample objects.

This is because the code in the commit added a condition where examples are only skipped if preset is also being used, but I don't believe a preset is necessary in order to use a custom model in this way (or is it?)

add default num_workers depend on user's # of cores

Add additional information so upstream is more discoverable

The Cohere client is hosted on PyPI for example, but lacks links back to the original Github repo.

Some examples of improvements:

Add project links to PyPI. For a good template look at https://pypi.org/project/scikit-learn/.
Add a link back to the repo in the Contributing section of the readme. Eg. to the Issues page.

Meta field missing from response object when streaming is set to true

When generating responses with the SDK, we can keep track of the cost in terms of billed tokens in the meta field of the response object, except when streaming is set to true.

AttributeError: 'StreamingGenerations' object has no attribute 'meta'

Related to that, the fields num_tokens and num_chars are missing from the response headers.

ChooseBest output should be called 'scores'

Part of cohere-ai/blobheart#1351

Lnagchain support for cohere 5

Langchain only supports Cohere 4 and it would be nice to support for version 5

Wheels can not build project on mac m2 with cohere version `4.18.0`

The new release 4.18.0 has some issue with mac m2 on installation. I am receiving this error while installing:

Building wheels for collected packages: fastavro
  Building wheel for fastavro (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/***/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/tmpm2_a8yuj
       cwd: /private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-install-gshr49e3/fastavro_796163df432e41d1aa0e51de0db4dfca
  Complete output (150 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-universal2-cpython-39
  creating build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_schema_common.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_schema_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_logical_writers_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/json_read.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/write.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_write_common.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_write_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/__init__.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_read_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/types.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/json_write.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_read_common.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_validate_common.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_logical_readers_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/utils.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/logical_writers.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/_validation_py.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/__main__.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/logical_readers.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/const.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/schema.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/read.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  copying fastavro/validation.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  creating build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/binary_decoder.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/__init__.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/binary_encoder.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/parser.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/symbols.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/json_encoder.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  copying fastavro/io/json_decoder.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/io
  creating build/lib.macosx-10.9-universal2-cpython-39/fastavro/repository
  copying fastavro/repository/__init__.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/repository
  copying fastavro/repository/flat_dict.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/repository
  copying fastavro/repository/base.py -> build/lib.macosx-10.9-universal2-cpython-39/fastavro/repository
  copying fastavro/py.typed -> build/lib.macosx-10.9-universal2-cpython-39/fastavro
  running build_ext
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
              writer_schema, named_schemas, offset, size, return_record_name, return_record_name_override
          )
  
  
  class Block:
      def __init__(
      ^
  ------------------------------------------------------------
  
  fastavro/_read.pyx:976:4: Compiler crash in AnalyseDeclarationsTransform
  
  File 'ModuleNode.py', line 203, in analyse_declarations: ModuleNode(_read.pyx:1:0,
      doc = 'Python code for reading AVRO files',
      full_module_name = 'fastavro._read')
  File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:10:0)
  File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:975:0)
  File 'Nodes.py', line 5121, in analyse_declarations: PyClassDefNode(_read.pyx:975:0,
      name = 'Block')
  File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:976:4)
  File 'Nodes.py', line 2710, in analyse_declarations: CFuncDefNode(_read.pyx:976:4,
      args = [...]/11,
      modifiers = [...]/0,
      outer_attrs = [...]/2,
      overridable = True,
      visibility = 'private')
  File 'Nodes.py', line 2721, in declare_cpdef_wrapper: CFuncDefNode(_read.pyx:976:4,
      args = [...]/11,
      modifiers = [...]/0,
      outer_attrs = [...]/2,
      overridable = True,
      visibility = 'private')
  File 'Nodes.py', line 2787, in call_self_node: CFuncDefNode(_read.pyx:976:4,
      args = [...]/11,
      modifiers = [...]/0,
      outer_attrs = [...]/2,
      overridable = True,
      visibility = 'private')
  
  Compiler crash traceback from this point on:
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 2787, in call_self_node
      type_entry = self.type.args[0].type.entry
  AttributeError: 'PyObjectType' object has no attribute 'entry'
  Compiling fastavro/_read.pyx because it changed.
  [1/1] Cythonizing fastavro/_read.pyx
  Traceback (most recent call last):
    File "/Users/***/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
      main()
    File "/Users/***/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/***/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 248, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 416, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
      self.run_setup()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 37, in <module>
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 346, in run
      self.run_command("build")
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
      _build_ext.build_extension(self, ext)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
      new_ext = cythonize(
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
      cythonize_one(*args)
    File "/private/var/folders/lb/d7gdj0xs0hx502f2y__8x0gc0000gn/T/pip-build-env-4xnztzki/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: fastavro/_read.pyx
  ----------------------------------------
  ERROR: Failed building wheel for fastavro
Failed to build fastavro
ERROR: Could not build wheels for fastavro which use PEP 517 and cannot be installed directly

I have docker file with base image python:3.9-buster and this version is working fine on the docker.

CI to Publish Our PyPi Server

CI to Publish Our PyPi Server on all merges/pushes to main so that we dont need to manually build and upload every time.

cc: @jimwu6 @choicallum

Force Cohere LLM to generate only Json format as expected without any other unnecessary text

Hello,

I do not really know if it is a right place to ask my question, but did not find any appropriate ones.

I would like to know if when we ask Cohere to generate text completion (or chat) through API, do we have a solution to force Cohere to generate only Json validated format without any introduction text at the beginning ?

I have used a prompt-engineering approach to require a Json validated-format in my prompt by adding either :
"your output is only of json valid structure [{{'key1': value1, 'keys2': [values], 'key3': value3}}]"
Or
"your output is only of json valid structure [{{'key1': value1, 'keys2': [values], 'key3': value3}}], without any introduction text ..."

As result, Cohere always add an introduction text at the beginning as follows :
"Here is the JSON format generated from the information provided:
'''json
[useful_information_situated_here]
'''
"

What I really want to receive as output is only the Json valid format inside of parenthesis (i.e. useful_information_situated_here)

Support for Validation Dataset in co.create_custom_model

Using co.create_custom_model(model_name, dataset=dataset, model_type="CLASSIFY"), can I also provide a validation dataset? Otherwise, it automatically splits my dataset into train/test. I can upload a validation dataset in the Cohere dashboard, but not with the Python SDK.

does not work without dependency requests installed

Add Compressed Embeddings docs

Add link to compressed embeddings docs when it is ready #210 (comment)

add frequency penalty, presence penalty, stop sequences and return likelihoods

all already documented;

https://docs.cohere.ai/generate-reference#frequency_penalty-optional

Add `tokens` and `token_log_likelihoods` to ChooseBest output

Some clarity on rate limiting.

Hey great job on the multi-lingual models. I've been trying the cohere trial api and I have a question about rate-limiting.
The error says that the number of requests is rate limited to 10/min. But I'm getting these errors even after 4-5 requests (these are batched requests). So I was wondering if the rate-limit also applied at token level like openAI api?

what if we just asynced all the model calls

when making requests spin up the thread to request and return immediately
give all the cohere response objects a mutex, when a request is running a thread should hold onto that mutex
if accessed, block <-- this is the spooky part

Refactor so that not everything is in one init file

Test Issue Michael

endpoints should return proper python objects instead of dict

Incorrect `finish_reason` for `generate`?

I'm not sure if this is the right place to ask this, but generate with max_tokens=3 finshes with finish_reason=COMPLETE

$ curl --request POST \
     --url https://api.cohere.ai/v1/generate \
     --header 'accept: application/json' \
     --header "authorization: Bearer $COHERE_API_KEY" \
     --header 'content-type: application/json' \
     --data '
{
  "max_tokens": 3,
  "stream": true,
  "prompt": "Please explain to me how LLMs work"
}
'
{"text":" LL","is_finished":false}
{"text":"Ms","is_finished":false}
{"text":",","is_finished":false}
{"is_finished":true,"finish_reason":"COMPLETE","response":{"id":"05835269-8d06-422d-8f4e-fc3e8a2b8a96","generations":[{"id":"73b93e8c-ee35-4831-a3be-c7534b31dbb9","text":" LLMs,","finish_reason":"COMPLETE"}],"prompt":"Please explain to me how LLMs work"}}

Should it finish with MAX_TOKENS?

create_embed_job python SDK bug

the create embed job function fails with error the input dataset_id is empty even though it's given.
I fixed this error by going to:

python3.9/site-packages/cohere/client.py
Client
create_embed_job

and I manually edited the json

from:

    json_body = {
        "input_dataset_id": input_dataset_id,
        "name": name,
        "model": model,
        "truncate": truncate,
        "text_field": text_field,
        "input_type": input_type,
        "output_format": "avro",
    }

to:

    json_body = {
        "input_dataset_id": input_dataset_id,
        "dataset_id": input_dataset_id,
        "name": name,
        "model": model,
        "truncate": truncate,
        "text_field": text_field,
        "input_type": input_type,
        "output_format": "avro",
    }

and the issue was fixed.

I get this error when trying Cohere Example API

Documentation for Intent classification: https://docs.cohere.com/reference/intent-recognition

Input Import:

import cohere
from cohere.responses.classify import Example

Error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[12], line 2
      1 import cohere
----> 2 from cohere.responses.classify import Example

ModuleNotFoundError: No module named 'cohere.responses'

Cohere API Responses is not coming correctly. I am using the cohere verison: 5.2.2

some clarity on System message for ChatCohere ( command)

Hello,

Amazing work by COHERE to provide a lot of resources online for people to try and implement stuff. Recently I was using semantic routers with cohere models. I want my agent to work as a assistant bot for a 'imaginary company', I send the info in the system prompt. When I ask a question like " what is your company's name and product do you offer", it says cohere. So is there anyway that I can make the model to behave as travel assistant bot? How do I have to make use of the system prompts?

Thanks!

exponential decay retrying

cohere function to detect language is not working

Hello, i'm trying to use the API to detect a language, and including if i am doing the base code that the documentation provides, this still not working.

import cohere
co = cohere.Client('myapi')

response = co.detect_language(
  texts=['Hello world', "'Здравствуй, Мир'"]
)
print(response)

I don't know why this is not working, the 'error' i am getting is:
<cohere.responses.detectlang.DetectLanguageResponse object at 0x02307F50>

ValueError: Chunk too big in async stream.

I was trying to use the async stream to summarize a document, which was fairly long.
I got this error:

  File "./minimal.py", line 28, in get_summary_stream_cohere_client
    async for token in streaming_chat:
  File "./venv/lib/python3.11/site-packages/cohere/responses/chat.py", line 370, in __aiter__
    async for line in self.response.content:
  File "./venv/lib/python3.11/site-packages/aiohttp/streams.py", line 35, in __anext__
    rv = await self.read_func()
         ^^^^^^^^^^^^^^^^^^^^^^
  File "./venv/lib/python3.11/site-packages/aiohttp/streams.py", line 311, in readline
    return await self.readuntil()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "./venv/lib/python3.11/site-packages/aiohttp/streams.py", line 338, in readuntil
    raise ValueError("Chunk too big")

It turns out the API responds with the whole chat_history at the end of the stream, which is too big for aiohttp stream.
return_chat_history=False is not being respected.

Minimal reproduction code:

import json

import cohere

import config

SYS_PROMPT = """
Provide a concise one-page summary of the document provided to you.
""".strip()


async def get_summary_stream_cohere_client(text: str):
    async with cohere.AsyncClient(config.COHERE_API_KEY) as aio_co:
        streaming_chat: cohere.StreamingChat = await aio_co.chat(  # type:ignore
            message=f"""
                <document>
                {text}
                </document>
                {SYS_PROMPT}
                Summary:
                """.strip(),
            max_tokens=3000,
            model="command-r",
            stream=True,
  
        )
        async for token in streaming_chat:
            if hasattr(token, "text") and token.text is not None:
                print(token.text)


if __name__ == "__main__":
    import asyncio

    # https://0x0.st/HFN3.json
    with open("./this.json", "r") as f:
        file_content = json.load(f)
        text = file_content["text"]
        asyncio.run(get_summary_stream_cohere_client(text))

Add docstrings for all user facing functions.

Helps users know what the functions do without looking at the implementation -- look at GO SDK for examples of comments that could be adapted into docstrings

	def is_api_key_valid(key: Optional[str]) -> bool:
	"""is_api_key_valid returns True when the key is valid and raises a CohereError when it is invalid."""
	if not key:
	raise CohereError(
	"No API key provided. Provide the API key in the client initialization or the CO_API_KEY environment variable." # noqa: E501
	)

	return True