Trying to use the name_resolve method:

>>> api.name_resolve('QmXfrS3pHerg44zzK6QKQj6JDk8H6cMtQS7pdXbohwNQfK')

Results in the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: request() takes exactly 2 arguments (4 given)

The API seems to mangle UTF-8 encoded text in a way as if it had received ISO-8859-1 encoded bytes from IPFS and then transformed each byte to a multibyte UTF-8 sequence.

First we create a file with UTF-8 contents:

$ echo Frække frølår | ipfs add
added QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT
$ ipfs cat QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT
Frække frølår
$

Next, let's try and fetch the same content from Python:

>>> import ipfsApi
>>> api = ipfsApi.Client('127.0.0.1', 5001)
>>> res = api.cat('QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT').rstrip('\n')
>>> print(res)
Frække frølår
>>> print(res.encode('iso-8859-1').decode('utf-8'))
Frække frølår
>>>

In ipfsApi/commands.py, the code does this:

def prepare(self, client, **kwargs):
    kwargs.update(self.defaults)
    return functools.partial(self.request, client, **kwargs)

This means that any keyword arguments passed in kwargs will be replaced by values set in self.defaults. This is unexpected behavior for something called a "default".

I just tried calling object_new, which in turns tries to call _object_new. But _object_new is not defined. Patch attached.
object_new.patch.txt

In the IPFS CLI, you can use ipfs get -o <hash> to get a file and save it with a given name, but the ipfsApi.Client().get() function does not have this capability right now.

@Alexander255 Just discovered a package/util for making uploading to PyPI easier. Might simplify our process. It's called Flit, and it is configured through an ini file instead of a setup.py file. I believe that it is cross-platform too, which is nice. Thoughts?

If no ipfs daemon is present/running, the tox test runs/hangs for a lengthy period of time when entering the functional/tests portion. It might be a more streamlined experience to perform a pre-check on the ipfs daemon state and fail fast rather than running/hanging.

I am trying to get the hash of an existing file without adding to the IPFS.
ipfs add myfile --only_hash seems to work
but ipfs_api.add(myfile, only_hash=True):
TypeError: request() got an unexpected keyword argument 'only_hash'

Am I doing something wrong?
Thanks.
Pat.

Is there a simple inverse operation for hash = ipfs.add(directory_path, recrusive=True)[-1]["Hash"]?

as discussed, would be great to have this in the ipfs/ org so people find it and all collab on one.

let's:

• move it over (can you transfer it to ipfs/ user/org?)
• add you as admin so you can add whoever else you want.

The README suggests looking to https://ipfs.io/ipns/QmZ86ow1byeyhNRJEatWxGPJKcnQKG7s51MtbHdxxUddTH/Software/Python/ipfsapi/ for documentation but upon accessing the link, the following message appears:

ipfs resolve -r /ipns/QmZ86ow1byeyhNRJEatWxGPJKcnQKG7s51MtbHdxxUddTH/Software/Python/ipfsapi/: Could not resolve name.

...with no other documentation.

Hi,

This works fine:
In [17]: ipfsApi.Client().dht_get('abc')
Out[17]: u'def'

But longer strings, give an error, like this (as well as with "real" hashes):

In [19]: ipfsApi.Client().dht_get('Qmxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
TypeError Traceback (most recent call last)
in ()
----> 1 ipfsApi.Client().dht_get('Qmxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

/usr/local/lib/python2.7/site-packages/ipfsApi/utils.pyc in wrapper(api, _args, *_kwargs)
155 def wrapper(api, _args, *_kwargs):
156 res = cmd(api, _args, *_kwargs)
--> 157 return res[self.field]
158 return wrapper

TypeError: list indices must be integers, not str

Both work fine with the command line tool, is the python client doing something special?

Thanks

Looking at the current README, I just noticed this passage:

This module also contains some helper functions for adding strings, json, and even python objects to IPFS:

>>> lst = [1, 77, 'lol']
>>> api.add_pyobj(lst)
'QmRFqz1ABQtbMBDfjpMubTaginvpVnf58Y87gheRzGfe4i'
>>> api.get_pyobj(_)
[1, 77, 'lol']

What this doesn't mention unfortunately is that doing that .get_pyobj() on a hash from an untrusted source = remote code execution. Our documentation does mention this, but even that passage doesn't properly show the magnitude of the problem.

What definitely has to happen:

• Replace the example with an example showing JSON serialization instead

What we may also want to do:

• Deprecate the .get_pyobj() and .add_pyobj() function and remove them for good in 0.5.x

Thoughts?

Using go-ipfs (0.4.10):

~/t/s/processing % ipfs add -r 8c63a38e8df4f014a8fb128b99f11c1b6e21989f
added QmQrzduYxP27AjwPZZbLyTaHkXtHxQorMe8QgZbTg6iDyx 8c63a38e8df4f014a8fb128b99f11c1b6e21989f/large.jpg
added QmP54LoYywz15HaoatMiRHDB8mk1YMZjDseENgeLmBjKhK 8c63a38e8df4f014a8fb128b99f11c1b6e21989f/nano.jpg
added QmNzpeNwrTgujTAMgS4GFPQ4hQbk2CTWGbwamLZPRjSTAe 8c63a38e8df4f014a8fb128b99f11c1b6e21989f/original.jpg
added QmQAezoEfsuJSG147Jc42hgeGPc28a9NTFbfpYZK3JAcon 8c63a38e8df4f014a8fb128b99f11c1b6e21989f/small.jpg
added QmRFtHgrXg3Uex2n5658o4iHzFzBrQHkL9fsX87aw9cBHU 8c63a38e8df4f014a8fb128b99f11c1b6e21989f/tiny.jpg
added QmVRG1S2ypL8TekkQrFznbFQxqHMoAXq5m9t8juDNCoMNV 8c63a38e8df4f014a8fb128b99f11c1b6e21989f

Using py-ipfs-api (ipfsapi-0.4.2):

self.api = ipfsapi.Client(host=os.getenv("IPFS_API_HOST", "localhost"))
res = self.api.add(path, recursive=True)

Only outputs 1 or 2 items:

(Pdb) res
[{'Name': '8c63a38e8df4f014a8fb128b99f11c1b6e21989f/small.jpg', 'Hash': 'QmQAezoEfsuJSG147Jc42hgeGPc28a9NTFbfpYZK3JAcon'}, {'Name': '8c63a38e8df4f014a8fb128b99f11c1b6e21989f/tiny.jpg', 'Hash': 'QmRFtHgrXg3Uex2n5658o4iHzFzBrQHkL9fsX87aw9cBHU'}]

Another example:

In [37]: pwd
Out[37]: '/home/user/tmp/steemq/processing/0f264e013df17aae836068b70942d7a256cb0a79'

In [38]: ls
raw_video.mkv  tilesheet_96_5_144_108.png

In [39]: import ipfsapi

In [40]: c = ipfsapi.Client()

In [41]: path = !pwd

In [42]: path
Out[42]: ['/home/user/tmp/steemq/processing/0f264e013df17aae836068b70942d7a256cb0a79']

In [43]: r = c.add(path[0], recursive=True)

In [44]: r
Out[44]: 
{'Hash': 'QmXcZ7fGbtCAikLWbscznZrKMBmaDF5QruhpWZtMsVHAYx',
 'Name': '0f264e013df17aae836068b70942d7a256cb0a79/raw_video.mkv'}

In [45]: !du -sh *
13M	raw_video.mkv
268K	tilesheet_96_5_144_108.png

In [46]: 

IPFS Daemon output on add invoke:

05:45:27.223 ERROR commands/h: err: http: invalid Read on closed Body handler.go:285

What is the completeness of this api? in relation say to https://github.com/ipfs/js-ipfs-api ?

Many people are using ipfs from python and they've asked me this question lots. I think it would be good to have a spec completeness markdown checklist.

Hello! Recently I have not been as involved with this project as I would like to be, and it is likely that I will not have much time in the future (although I would still like to contribute from time to time). Would anyone be interested in being a maintainer? I will keep reviewing and merging PRs in the meantime but I think this project could use some love!

In the future I would like to have built some more abstraction on top of the core api (@jgraef @Mec-iS python3-ipfs-api has some really awesome ideas), especially seeing as the IPFS team will be eventually rolling out some kick-ass features such as pub-sub. So if anyone is interested in maintaining py-ipfs-api and giving it a face-lift, post here!

If I call something like:

value = api.cat('invalidhash')

I should not get back the string {u'Message': u'invalid ipfs ref path', u'Code': 0}. I should get an exception. Since we're using the requests library, a simple first step may be calling:

response.raise_for_status()

...assuming that the HTTP API is returning useful status code.

The IPFS command line utility provides a way to publish an IPFS path for a particular IPNS key by using its name, e.g. ipfs name publish --key=mykey ...; ipfsapi doesn't seem to have a way to do this, however.

I see that the simple writeup is only for connecting to daemon and adding a file.

Please add for
-How to list all the files added with different methods
-How to get a specific file with multiple ways
-Explain the different parts of the id() result

Without this, the usability is insufficient.

Hi,

The 0.4.0 release of go-ipfs introduced a change to multipart file upload that breaks compatibility when adding directories (ipfs/kubo#2046)

Currently when adding a directory via py-ipfs-api with recursive=True, the server adds a plain file with multipart encoding noise, e.g.:

--7587cf2cd7c54e3393bc49f454e626ce
Content-Disposition: file; filename="a%2Fb"
Content-Type: multipart/mixed; boundary="debd9482563f4d0eb04c6dcfa5cef500"

--debd9482563f4d0eb04c6dcfa5cef500
Content-Disposition: file; filename="a%2Fb%2Fc"
Content-Type: application/octet-stream

abcd123

--debd9482563f4d0eb04c6dcfa5cef500--
--7587cf2cd7c54e3393bc49f454e626ce--

Instead of nested multipart items that are chunked by size, the API now expects one multipart boundary, with one part per file or directory. Also, directories and symlinks are now expected to have the Content-Type of application/x-directory or application/symlink.

Here's an example of the expected input format: https://github.com/ipfs/go-ipfs/blob/8c4900b3b8341fdc4667e57d9e45fe73bad4d123/commands/files/file_test.go#L82

Here are some things I'd like to work on over the next couple weeks:

• documentation
  • write doc strings for all api methods
  • make a readthedocs.org page
• file streaming
  • chunk large files and multiple files instead of adding them all to the same buffer
  • use file streaming for all add operations, not just recursive add
  • support ipfs get, i.e. streaming files/directories directly to disk (re: #28)
• async
  • edit: make client more friendly for async integration

From #26

I'm able to schedule time to work on this project now, and I have a goal list that is mostly accomplished by @amstocker. (context: BrendanBenshoof/ipfs_scratch#1)

The big things that may be handled well, but I need to figure out and insure are:

• Error Handling (do we provide useful Exception (codes or types) both when requests fails and when ipfs returns an error? How does the generalized "command type" model interact with ipfs command specific
• Documentation? We essentially want to re-document every ipfs command with context on what is returned by our api. Inline pydoc doc-strings are a little harder because of the model @amstocker used (which I actually really like). I might just go ahead and add them to the "COMMANDS" section of client.py following the associated definition and in the worst case I'll have to compile a documentation page myself.
• Python 3 compatibility
  • Looks like there is a lot of work to do on this front. I am still getting a more solid understanding of what.

I have a directory with a single image in it which I want to put into
IPFS:

steem@rootu:/var/ipfs/tmp$ ~/.local/bin/ipfs add -r tmpif66w1c4/
added QmU8fTvjUP8a2PoaPqf1pDQ4Fm2yrtAruS8pmHHAaZRrxJ tmpif66w1c4/045f1518-2828-11e6-9574-9db9ce0b34be.png
added QmQVwkefgTHCpauE3nAxixpWsiWRD23AUENnhJ94DGMQtJ tmpif66w1c4

So that is good, but I want to use python:

steem@rootu:/var/ipfs/tmp$ python3
Python 3.4.2 (default, Oct  8 2014, 10:45:20) 
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipfsApi
>>> api = ipfsApi.Client('127.0.0.1', 5001)
>>> res = api.add("tmpif66w1c4", recursive=True)
r>>> res
{'Name': 'tmpif66w1c4', 'Hash': 'QmY91butp7Pcnm9DtUUqWUupYXkVdfMwKTPbGuK3Ha85nZ'}

Didn't work. How about I add a /?

>>> res = api.add("tmpif66w1c4/", recursive=True)
>>> res
[{'Name': 'tmpif66w1c4/', 'Hash': 'QmUPU3iYDqD8xSp4NoBByWR5wMbLh1hiFSq5Ys6FRNaagj'}, {'Name': 'tmpif66w1c4', 'Hash': 'QmVmgQfRgP4fy8NH2rCotuRt4kwCUiiBPiPA3vo3aAyZ6T'}]

WTF?

The documentation link in README seems to be down.
Could this be ipns related? Gateway says "Could not resolve name".

For some requests it makes sense to use HEAD over GET if it is only
relevant whether the request succeeded or not.

Background:

client.repo_gc() can potentially return many values (scale: easily 10-100k) depending on how much garbage was collected.
It takes quite some time to parse and decode (json) all that even if the result does not matter
inside the calling function.
Most of the time you do not need to know the hashes of the garbage.

I haven't been able to get file streaming for a recursive add working yet (right now I do multiple requests and manually set up the links). I'm pretty sure requests itself can't render multi-part/mixed bodies, which is how the IPFS http api expects it; https://github.com/ipfs/go-ipfs/blob/master/docs/implement-api-bindings.md#note-on-multipart--inspecting-requests, so we might have to use urllib3 or just do it manually. If anyone knows how to do this, some help would be greatly appreciated.

The recursive add functionality does not currently use the filter that it accepts to check the names of files. This should be implemented.

While running tests against newest versions of go-ipfs i noticed that "test_add_multiple_from_list" was seemingly randomly failing.

It seems like while adding multiple files using "add(['file1', 'file2'])" sometimes this call fails.

>>> a.add(['/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/fsdfgh', '/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/popoiopiu'])
[{'Hash': 'QmQcCtMgLVwvMQGu6mvsRYLjwqrZJcYtH4mboM9urWW9vX', 'Name': 'fsdfgh'}, {'Hash': 'QmYAhvKYu46rh5NcHzeu6Bhc7NG9SqkF9wySj2jvB74Rkv', 'Name': 'popoiopiu'}]
>>> a.add(['/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/fsdfgh', '/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/popoiopiu'])
{'Hash': 'QmQcCtMgLVwvMQGu6mvsRYLjwqrZJcYtH4mboM9urWW9vX', 'Name': 'fsdfgh'}
>>> a.add(['/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/fsdfgh', '/home/krzysiek/Dokumenty/PyCharmProjects/py-ipfs-api/test/functional/fake_dir/popoiopiu'])
[{'Hash': 'QmQcCtMgLVwvMQGu6mvsRYLjwqrZJcYtH4mboM9urWW9vX', 'Name': 'fsdfgh'}, {'Hash': 'QmYAhvKYu46rh5NcHzeu6Bhc7NG9SqkF9wySj2jvB74Rkv', 'Name': 'popoiopiu'}]

When this happens the following error is printed in go-ipfs daemon console:

13:46:35.014 ERROR commands/h: err: http: invalid Read on closed Body handler.go:288

The methods on an API client object have no documentation:

>>> import ipfsApi
>>> api = ipfsApi.Client()
>>> help(api.ls)

This yields documentation on the partial class, which isn't terribly helpful. It is also impossible to ask for documentation on these methods using pydoc:

$ pydoc ipfsApi.Client.ls
no Python documentation found for 'ipfsApi.Client.ls'

This all seems very developer-unfriendly.

My team and I have been working on documenting and testing this library, but much of our testing is made harder by the Python 2.6 support that this library currently has. We've added support for Python 3.5 to our version, and we'd like to remove support for 2.6. Does anyone have a problem with that?

This is obviously a no-go.

I'd like to lay out some short-term objectives for this repo to get everyone's opinions about them. Some of them might not be great ideas, and I'd like to have that pointed out before I spend any energy on them.

Concrete Objectives

• Get 100% test coverage and enforce it with CI
• Implement more of the HTTP API Spec
• Implement and enforce commit message standards (like signing off with name/license)
• Rename PIP package and python module to be called ipfsapi

Fuzzy Objectives

• Integrate project with Codacy for better code insight
• Develop "good" documentation (Docstring and pydoc, I think)
• Create a better developer on-ramp (add a CONTRIBUTING.md with good instructions for setting up the py-ipfs-api development environment)

Social Objectives

• Have at least two people review the code for PRs before merging them (this requires critical mass in contributors)

Thoughts? Additions? I'd love some feedback.

@BrendanBenshoof I just realized that with #23 the default kwargs from Client are no longer passed into request calls. I have an idea of how to fix this cleanly that I'll work on when i get home later tonight.

Link to the client API reference is broken in the README page

https://ipfs.io/ipns/QmZ86ow1byeyhNRJEatWxGPJKcnQKG7s51MtbHdxxUddTH/Software/Python/ipfsapi/

Quite some key names in the data structures returned by the API are either camelcase or (if only one word) starting with an uppercase letter.

PEP8 mandates this_kind_of_names for function/method parameter names and there is some correspondance between them and dicts: if you use stuff like **kwargs, kwargs is a dict that has the parameter names as keys. If you explode a dict, key names from the dict get parameter names of functions and methods.

So, would it be a good idea to rename the key names to this_style? If desired, this change should be done early, before the api gets in wide usage.

When host, port, or base is None, ipfsAPi.client.Client should use the default value rather than trying to work with None. This makes the module easier to use; consider, for example, if I have a Python client that includes:

import argparse

def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument('--host')
    p.add_argument('--port')
    p.add_argument('--base')
    return p.parse_args()

I would like to be able to do this...

api = ipfsApi.Client(host=args.host, port=args.port, base=args.base)

...rather than requiring a complex set of conditionals to deal with various combinations of user-specified vs. user-wants-the-default values.

I believe this is because it's testing Python 2.6, which #46 dropped support for. Additionally, the functional tests require a running IPFS daemon on the same machine. I believe we could make TravisCI spin one up with some finagling. I'll investigate that further.

In the README, you show both

api.pin_ls(opts={'type':'all'})

and also

api.add('fake_dir', recursive=True)

...which is correct usage?

In particular, I'm trying to do the equivalent of ipfs pin add -r, but it's unclear to me whether it should be:

api.pin_add(addr, recursive=True)

or

api.pin_add(addr, opts={"recursive":"True"})

or maybe something else?

The encoding.py code seems a lot like multicodec implementation, should we separate it out ? so that it can be used in py-ifps too ?

See #26 for discussion.

I'll be honest; this is over my head right now. I've never done any Python async stuff. I'd love for anyone with more experience to chime in with thoughts or ideas about how/why to do this.

Why does client.py do this:

class Client(object):

    __client__ = http.HTTPClient

And then:

self._client = self.__client__(host, port, base, default_enc)

That's a lot of underscores, and _client and __client__ are
perhaps too similar. Why not simply:

self._client = http.HTTPClient(host, port, base, default_enc)

? I'm not clear what the extra level of indirection is getting us.

This isn't really an issue. I just wanted to mention that I've used the ipfsApi module to implement a simple gitremote-helper that permits pushing git repositories to/clonging git repositories from ipfs:

• https://github.com/larsks/git-remote-ipfs

It's mostly a proof-of-concept at this point, but I thought folks might find it interesting to see the ipfs api in use in actual code.

In [28]: !ipfs add  -r tmp/
added QmQDze3FMH2GxVfomdghYeLGAkQ4QJrvcMGKykW7zBwJUH tmp/Devcon2_-_Orbit_-_Distributed__real-time_Web3_application_with_IPFS_and_Ethereum.pdf
added QmSTbjW14eBr8ZgJzxjDtEFF9CBhVccvjkbcGeiMthm6xC tmp/Maker_Devcon2.pdf
added QmXKtze4Z97osJGjvRZPLb8cfhf2sh2nShy9zqsxTTnrFf tmp/Zcash-slides.pdf
added QmTkEnbpLjuotXxqK8Wyn4tozMJXaAwqBtQguKQ7yyo4kf tmp/test/site4.png
added QmeMBXfX3xqQnoxPa4We5rLoMxqcbUmqDSnsDKAXFtW8yY tmp/test
added QmQsYbAp34mGm4RHxZxjpRruWzoWEAcVyq5m43tegA6psq tmp

In [29]: c.add('tmp/', recursive=True)
Out[29]: 
[{'Hash': 'QmT5DQ74JNwwhH7y5LmAjEND9Yu1iukaxBhdjQnwxMCHj8', 'Name': 'tmp/'},
 {'Hash': 'QmPtdDjMCtf3BgMytQqwfMfvru6bmxNDAHyvMjaivYKCuY', 'Name': 'tmp'}]

In [30]: c.add('tmp', recursive=True)
Out[30]: {'Hash': 'QmRkjAFXWVfY9DQvfeQJ2UfGdvEygJQVoS3YkfobEDDVo8', 'Name': 'tmp'}

In [31]: 

Furthermore, QmRkjAFXWVfY9DQvfeQJ2UfGdvEygJQVoS3YkfobEDDVo8, QmPtdDjMCtf3BgMytQqwfMfvru6bmxNDAHyvMjaivYKCuY and QmT5DQ74JNwwhH7y5LmAjEND9Yu1iukaxBhdjQnwxMCHj8 appears to be garbage.

go-ipfs v0.4.10 introduced the ipfs shutdown directive to more gracefully kill the IPFS daemon instead of CTRL-C or killing the PID (or in ipwb, killall ipfs). It would be much preferred to use the ipfs-api module to accomplish this but no ipfsapi.shutdown() is implemented as far as I can tell, as the API is currently aligned with go-ipfs v0.4.9 per the README.

Is there a way to pass ipfsapi an arbitrary command to relay to the ipfs daemon if py-ipfs-api does not yet support it? If not, that would be a nice feature to have to account for the lag in the API to the daemon implementation.

Are there plans to update the py-ipfs-api to the new features in go-ipfs 0.4.10? The above more graceful shutdown would be handy.

Since JS-IPFS has (mostly) feature-parity with Go-IPFS we should probably look into testing the client library with that daemon implementation as well.

Steps to be done:

• Fire up an JS-IPFS instance and run the py-ipfs-api test suite, then report what happend
• Fix any issues encountered (including skipping tests for features not present)
• Add CI-testing for Python 2.7 and the latest Python 3.X

To install py-ipfs-api with pip, you run pip install ipfs-api, but to use it in a python file you write import ipfsApi.

I think this is because python doesn't allow hyphens in module names, which is understandable. However, this package effectively goes by three names.

1. The git repo is py-ipfs-api,
2. The pip package is ipfs-api,
3. And the module is ipfsApi

Can/Should anything be done to make this more consistent? I think it's probably confusing to new users (I know it confused me at first), but I don't know how hard to change it would be, or how bad of an idea that might be. Thoughts?

Looking at our contribution page, the largest contribution a protocol labs employee has made to this project was @RichardLitt by updating our README with the mentioned badge. Even IBM et al has made more contributions to this project than Protocol Labs 😉.

Unless Protocol Labs hires me or @whereswaldon, I think it would only be honest to drop that badge in our next release. 🙂

Any objections?