Hi,
I just finish to move all methods to Cython, all unit tests in test_rados.py pass locally
Let's see want the CI think, now.

Just to be sure, I can continue to work on this, Can you confirm that this feature is something your are interested in ?

Cheers,

Yes, I'm excited by this! @marcan is likely interested as well.

I haven't looked closely yet, but I think we'll want make ceph.in (turns into /usr/bin/ceph) do the run-in-thread dance itself, so it can be interrupted with ^C. I agree getting rid of it in the python bindings is the way to go.

Thanks for adding tests and keeping things backwards-compatible too!
We don't have it automated yet, but if you could verify that test_rados.py still works with python3 that'd be great. After this change, we'll definitely want to make separate python3 build steps and packages, since they need recompiling rather than just copying .py files around (http://tracker.ceph.com/issues/13959).

I work on Gnocchi project (http://gnocchi.xyz/), we uses rados API to store object. The current rados python binding is slow as hell, that why I started this.

About run_in_thread, it should not leave into the lib. Into our application we already use threads to do concurrency IO, this additional thread (run_in_thread) on each C call is an useless big overhead for us (We currently monkeypatch the lib to disable this thread).

I run some preliminary benchmark of our application today, the result is just impressive. We now process 10000 measures in 30 secs instead of 8 minutes with the Ceph backend.

So I will continue on this:

• By providing a patch to ceph.in to keep CTRL+C fast as before (even it's already better with cython than ctypes)
• And ensuring that tests pass with python3 too and post the result.

Yes, this is something I was looking to get around to doing sooner or later. Thanks for taking the lead!

Looks like you have some spurious egg-info files in here. I assume you're still working on this and pending some cleanup/rewriting of the commits.

I have:

• fixes a couple of str/bytes issues to pass tests in python 3.
• fixes the rpm/deb package of python-rados (for python 2)
• fixes ceph.in to load the new rados modules and adds run_in_thread

todo:

• fixes python-rbd to use cluster handle from the new rados binding

I think I can propose something for python3-rados and python3-rbd packaging but in other PR to not put much code into this PR.

I have done the last todo item

That was fast! I'll take a close look today.

The rest looks good! Just one other thing: for the docs can you update the admin/build-doc script? To extract the docstrings sphinx imports the module, so that script builds a dummy version of the python rbd module just for the docstrings. Can you add the same thing for librados, and make sure the paths still work for rbd.so? Thanks!

BTW the rbd python tests pass as well.

Oh, and adding python3 packages in a separate PR would be great.

I have fixed your remarks.

Thanks!