setuptools usability - not good, what can be done?

15 July 2009   12 comments   Python

Powered by Fusion×

Gun to your head; what would it take to make setuptools as a package author easy to use?

I've spent far too long time today trying to create a package for a little piece of code I've written. Because I can never remember all the bizarre options and commands to I tried to do it by following Tarek Ziade's wonderful Expert Python Programming but I still got stuck.

Granted, I did not read the f**n manual. Why should I have to? I've got more important things to do such as eating cookies and watching tv.

Here's what I know:

Here's what I don't know:

If I was a setuptools hacker I would happily wrap up all these annoying questions into a single script that just sensibly guesses everything and removing any choices and replace them with suggestions. But I'm not.

Or am I wrong. Should I take the plunge and study the manual? Will having a all-singing-all-dancing script that just works just be another leaky abstraction?

PS. If you do comment, please don't answer the bullet point questions above. I know most of the answers at the moment (but will surely forget them till the next time). Asking the questions to prove a point that setuptools requires too much schooling.


Peter Bengtsson
Also, I noticed that (in python2.6) running: python --help-commands
it breaks and the traceback talks about turbogears and some sort of json thing. Too tired to investigate this further. Sigh.
Heikki Toivonen
Sorry, I couldn't resist answering your bullet points. I think it is ok since my answers are kind of non-answers:

* You probably don't need to create eggs, hence you probably don't need to know
* I don't really understand the second bullet. Of course you need to test everything, always. But I still don't understand what you mean by "with or test the normal way on the command line"
* Use none of those. Create a source tarball with python sdist, and if you can't remember the PyPI commands just upload the file with your browser.
* You probably don't need a

Finally, I bet more than half of your issues are with distutils and not setuptools specific.
Matt Chaput
Reading the manual will not help. It doesn't document half of what setuptools does and is hard to follow anyway. It doesn't give any examples of how to do common things either. It's a mess.
Stephan Dooley
Not too mention the manual is hosted on what is possibly the SLOWEST server on earth.
Damien just need to be in the same directory as, then you just need to set 'include_package_data' to true. From setuptools doc:

Accept all data files and directories matched by ```` or found
in source control*.

You can use "exclude_package_data = {'some_dir': 'some_file'}" to exclude some files tracked by your vcs*.

*only supporting svn by default I think.
The only useful options are:

from setuptools import setup, find_packages
name = "HelloWorld",
version = "0.1",
packages = find_packages(exclude=['ez_setup', 'bootstrap']),
include_package_data = True,
install_requires = [...],

Most of the other - distutils - options are for Meta-data used by pypi. And because they are a pain to remember, you should use PasteScript templates or your IDE template features to create the a new distribution layout.
Peter Bengtsson
PasteScript templates??? To create the as well?
Jonathan Hartley
Amen to this. I have friends who are dyed-in-the-wool piss-poor VB-only developers, who happily acknowledge that they don't really know very much at all about software engineering.

But when they see what I have to go through to figure out how to get my Python packaged up for deployment, they simply laugh at us. And rightly so.

It's clearly a difficult and nuanced problem, but even so - we're doing it wrong.

I don't know how to do it right. Does anyone currently claim to?
Peter Bengtsson
"I don't know how to do it right. Does anyone currently claim to?"
I think there are a handful of people who do it right. Perhaps in the same sense as there are a handful of people who understand the linux kernal development.

I'm sure the tools are good enough. They just need to mature on a higher level. I'm sure dpkg was a pain before apt and look where we're at now with Synaptic.
i go crazy every time i look up and i understand it’s only a fragment of what i am supposed to be in the know of. and that’s just the consumer side, right? i’ve also noticed quite a few blog posts over the months complaining about the sheer complexity of it. distutils, setuptools, easy_install—it should be easy but is definitely not.
Peter Bengtsson
And this is one of those blog posts.
I would personally find it helpful if there were a document that clearly outlined the relationship between setuptools and distutils.

Specifically: does distutils provide any features that setuptools does not? I think the lack of some options, like for example the --manifest-only option to `sdist`, might be an example of this, or it may be that setuptools obsoletes this in some way that, as a newcomer, I don't yet understand. I find this a bit confusing, since it seems like the absence of such features could break installation of ported packages containing such directives in their configuration files.

So it would be helpful to either have some assurance that yes, I don't have to worry about whether or not I need to write two install scripts and a guide to porting, or to have a nice clear indication of under what circumstances I'd want to still use distutils or to write a conditional script that does certain things in certain ways depending on which package it's using.

I'm also a bit confused about how to, or if it's possible to, enable testing as part of the deployment process. This last is a bit off-topic for this venue.

Your email will never ever be published

Related posts

Previous: - Transform CSS into line style attributes with lxml.html 11 July 2009
Getting uploadify to work 17 July 2009
Related by keywords:
To assert or assertEqual in Python unit testing 14 February 2009
COPYFILE_DISABLE and python distutils in python 2.6 12 April 2014
EditDistanceMatcher - NodeJS script for doing edit distance 1 matching 05 February 2011
Correction: running Django tests with MongoDB is NOT slow 30 May 2010
Mocking a Python standard library 14 March 2008 - Using (py)inotify to run commands when files change 20 July 2009
Django vs. Java 25 October 2008