Running apify create my-first-actor --template python-start

Error: ENOENT: no such file or directory, open \my-first-actor\package.json

You are able manually add package.json and start script to run actor.

Issue Type: LOW

I'm trying to find out how to run an actor and specify the input for it. E.g., the LinkedIn Company URL needs something like {"queries": "Tesla\nmicrosoft.com"}

The API docs are clear that whatever data is passed to the POST request will be interpreted as input to the actor.

However, I can't find a proper alternative in the Python docs, e.g. in the Usage concepts. There's dataset's push_items(), but I don't think that's it. Is there an equivalent?

Check the documentation for completeness, typos and grammar errors

Write a design document for the Apify Python Client in Notion

Do not use Any type as suggested in the ANN401.

Instead of

from typing import Any

def get_first(container: list[Any]) -> Any:
    return container[0]

use the following

from typing import TypeVar

T = TypeVar('T')

def get_first(container: list[T]) -> T:
    return container[0]

Any can probably still make sense for the *args / **kwargs.

Use some of ACTOR_XXX env vars instead of APIFY_ACTOR_XXX

See the list in actor spec

Some of our configuration files currently use 2 space indent and others use 4 space indent.

Let's unify this and use the same indent (2 spaces) for all configuration files (yaml, toml, ini/cfg, ...).

Ruff is a new extremely fast Python linter written in Rust, which supports many rules from the flake8 & pylint world (700+).

They recently released a formatter, where single quotes are an option :).

We should have the linting, type-checking, unit testing and documentation checking in a pre-commit hook, automatically installed when you run make install-dev.

Write a nice blogpost announcing the Python Client availability (maybe this should be done only after app supports Docker images with the client preinstalled).

Set up deployment of the client package to PyPI (possibly automated through GitHub actions)

See https://github.com/apify/apify-client-js/pull/178/files for reference implementation in the JS client.

Set up building of the documentation from docstrings

• ideally to markdown
• upload the built docs to S3 so they can be shown on the web
• possibly through GitHub actions

After the API server implements this feature https://github.com/apify/apify-core/issues/2082.
It needs to change how we are passing API token into the HTTP header instead of the query parameter. It will improve the security of API token.

• In apify/apify-shared-python#1 we introduced apify-shared library for sharing general constants and utils across our Python projects. Let's start using it.
  • https://github.com/apify/apify-shared-python
  • https://pypi.org/project/apify-shared/

Currently, we use the underscore prefix in objects that are imported from other modules (e.g. all objects in https://github.com/apify/apify-client-python/blob/master/src/apify_client/_utils.py) - are not private.

This was intended to let users of the library know, that these objects are for internal usage only.

However, this does not correspond to the usage of underscore prefixes in the Python world. We should remove these prefixes from the non-private objects.

We can still use the underscore prefix in the module names, to let users know, this module is only for internal usage and should not be imported by library users.

From email conversation:

We are using Coegil platform to run apify-client library and getting this issue.

Pip install passed fine, installing all dependencies.

from apify_client import ApifyClient

ModuleNotFoundError: No module named 'apify_client'

Utilizing star imports, such as from apify_client import *, is generally considered as a bad practice in Python. This is because it can lead to namespace conflicts. While there may be specific scenarios where star imports could be useful, I don't see the case in the context of our packages.

I suggest removing them from our codebase so that we don't incentivize users to adopt this practice. Also, we won't have to maintain these lists anymore.

Write integration tests against production / staging API

We have an endpoint /acts/ACTOR-ID/validate-input which is not implemented in the client, we should implement it.

It takes the input to validate as POST payload, and optionally a build query parameter to specify the build tag against which to validate.

It returns a response with:

• HTTP status 200 and body { "valid": true }
• HTTP status 400 and body with the validation error

We should first add it to the documentation, so that we can refer to it in the docstrings. apify/apify-docs#722

Add a .reboot() method to the RunClient class. It will invoke the endpoint located at /v2/actor-runs/{actorRunId}/reboot via a POST request. The endpoint has no parameters.

I want to speed up consecutive calls that use the same actor. There are places on the website that describe a build cache for the docker containers but say they are only available on the API.

Create a shell of the project, including:

• setting up a dependency installer (Pipfile or requirements.txt)
• setting up linting
• setting up a test framework

There were a few changes to the JS client in the past that were not propagated to the Python client, we need to catch up.

We need to add these features:

• actor run env vars client (apify/apify-client-js#202)
• x-apify-workflow-key header (apify/apify-client-js#212)
• schema parameter in get_or_create methods for key-value store & dataset (apify/apify-client-js#233)
• origin param for last actor/task run endpoints (apify/apify-client-js#248)
• view parameter in dataset items methods (apify/apify-client-js#226)
• flatten parameter in dataset items methods (apify/apify-client-js#264)
• optional title field in task, schedule, key-value store, dataset, and request queue client methods (apify/apify-client-js#271)

Manually test all the endpoints to verify that they work (or return proper errors), and add automated tests if you find something that should have been tested but is not.

See apify/apify-client-js#181 for a reference implementation in the JS client.

We have introduced apify-shared-python to consolidate general constants for both the Client and SDK.

To facilitate a seamless transition to the new package, we have implemented re-imports with DeprecationWarning in the existing codebase - consts.py.

This is only a temporary state and after a few new releases, we should get rid of it.

Hello,
Judging from source code, I believe, it is not possible to specify maxItems parameter like this:

https://api.apify.com/v2/acts/actor-name/runs?token=YOUR_TOKEN&maxItems=10

Something like this:
actor.start(run_input={some input}, memory_mbytes=32768, build='latest', timeout_secs=100, max_items=100)

Please correct me if I'm wrong, and if so what's the way of accomplishing this?

We have some error subclasses like ApifyApiError defined in https://github.com/apify/apify-client-python/blob/master/src/apify_client/_errors.py, with the underscore suggesting it's a private submodule.

We have them documented in the docs, though, suggesting people should use them in their isinstance checks etc, which they should be able to, since the thrown errors should be a part of the public API of a module.

We should move them out of the private _errors submodule to a public errors submodule, to make it clear that these are OK to use by end users.

ListPage should be generic, i.e. allow to specify a type for the data inside items so it's a List[T] instead of just a List

Rename identifiers flagged with # noqa: A003 for built-in attribute shadowing.

In the case of methods named list, it even breaks mypy in some places. If you use the list type hint in the same module, e.g. here - https://github.com/apify/apify-client-python/blob/master/src/apify_client/clients/resource_clients/actor_collection.py#L50.

It seems that the following code:

response: httpx.Response = await self.http_client.call(
    url=self._url(f'records/{key}'),
    method='GET',
    params=self._params(),
)

returns a response of type httpx.Response. Later the field _maybe_parsed_body is accessed:

return {
    'value': response._maybe_parsed_body,
    '...': '...',
}

There are 2 occurrences of this:

• https://github.com/apify/apify-client-python/blob/v1.6.0/src/apify_client/clients/resource_clients/key_value_store.py#L120
• https://github.com/apify/apify-client-python/blob/v1.6.0/src/apify_client/clients/resource_clients/key_value_store.py#L323

Check if http_client.call really returns a httpx.Response object and in such case fix the accessing of non-existing field _maybe_parsed_body.

Since this project has matured quite a bit, and we're launching the Apify SDK for Python soon, and since the 0.7.0 beta has so many changes, many of which are breaking, it would be worth it to change the 0.7.0 version to 1.0.0.

We can't retrieve all posts for a user, it only returns posts from the first page:
Here is our script:

from apify_client import ApifyClient


from apify_client import ApifyClient
apify_client = ApifyClient('token')


actor_call = apify_client.actor('apify/instagram-post-scraper').call(run_input={'username' : ['username'], "limit": 100}, )
### get dataset and posts 
dataset_items = apify_client.dataset(actor_call['defaultDatasetId']).list_items().items

Is there any way work around to fix this?

I believe we can use just poetry instead of pip, virtualenv, setuptools and twine (although, it uses some of them under the hood).

https://python-poetry.org/

Implement the base of the client, including

• class structure
• http client
• base client resource classes
• utilities (response parsing etc)