pip で upgrade するときの不思議な挙動について

ここ最近pipで blockdiag をインストールするときの以下の不思議な挙動に悩んでいる。 (もしかすると正しい動きなのかもしれないが、ちとハマっている)

  • “pip install パッケージA -U” すると、依存するパッケージが install済みかどうか評価せずに、すでにあるパッケージをダウンロードの上、再 install しようとする。
  • 一旦パッケージAを uninstall した上で、install すると、依存するパッケージをきちんと評価して、すでに依存条件を満たすものについては再 install しない。

私の環境

Apple MacBook Pro の標準的な環境の python を利用し、以下のようなパッケージをインストールしている。

$ pwd
/Library/Python/2.6/site-packages

$ python
Python 2.6.1 (r261:67515, Aug  2 2010, 20:10:18)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Python/2.6/site-packages', '/Library/Python/2.6/site-packages/PIL', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']

$ sudo pip freeze
Jinja2==2.5.5
PIL==1.1.7
PyRSS2Gen==1.0.0
Pygments==1.4
Sphinx==1.0.7
Twisted==8.2.0
altgraph==0.6.8.dev
bdist-mpkg==0.4.3.dev
blockdiag==0.8.4
bonjour-py==0.3
cloud-sptheme==1.1
docutils==0.7
funcparserlib==0.3.5
macholib==1.2.1.dev
modulegraph==0.7.2.dev
numpy==1.2.1
nwdiag==0.2.5
py2app==0.4.2
pyOpenSSL==0.7
pyobjc-core==2.2b3
pyobjc-framework-AddressBook==2.2b3
pyobjc-framework-AppleScriptKit==2.2b3
pyobjc-framework-Automator==2.2b3
pyobjc-framework-CFNetwork==2.2b3
pyobjc-framework-CalendarStore==2.2b3
pyobjc-framework-Cocoa==2.2b3
pyobjc-framework-Collaboration==2.2b3
pyobjc-framework-CoreData==2.2b3
pyobjc-framework-CoreText==2.2b3
pyobjc-framework-DictionaryServices==2.2b3
pyobjc-framework-ExceptionHandling==2.2b3
pyobjc-framework-FSEvents==2.2b3
pyobjc-framework-InputMethodKit==2.2b3
pyobjc-framework-InstallerPlugins==2.2b3
pyobjc-framework-InstantMessage==2.2b3
pyobjc-framework-InterfaceBuilderKit==2.2b3
pyobjc-framework-LatentSemanticMapping==2.2b3
pyobjc-framework-LaunchServices==2.2b3
pyobjc-framework-Message==2.2b3
pyobjc-framework-PreferencePanes==2.2b3
pyobjc-framework-PubSub==2.2b3
pyobjc-framework-QTKit==2.2b3
pyobjc-framework-Quartz==2.2b3
pyobjc-framework-ScreenSaver==2.2b3
pyobjc-framework-ScriptingBridge==2.2b3
pyobjc-framework-SearchKit==2.2b3
pyobjc-framework-SyncServices==2.2b3
pyobjc-framework-SystemConfiguration==2.2b3
pyobjc-framework-WebKit==2.2b3
pyobjc-framework-XgridFoundation==2.2b3
python-dateutil==1.2
seqdiag==0.3.5
sphinxcontrib-blockdiag==0.8.3
sphinxcontrib-epydoc==0.4
sphinxcontrib-googleanalytics==0.1
sphinxcontrib-nwdiag==0.1.1
sphinxcontrib-seqdiag==0.1.1
sphinxjp.themecore==0.1.0
sphinxjp.themes.sphinxjp==0.1.0
webcolors==1.3.1
wsgiref==0.1.2
wxPython==2.8.8.1
wxPython-common==2.8.8.1
wxaddons==2.8.8.1
xattr==0.5
zope.interface==3.5.1

なお、PILは pip を通じてインストールしようとするとビルドエラーが発生するため、 homebrew で brew install PIL でインストールした。 そんなわけで、PILは以下のような感じでsite-packagesにシンボリックリンクを張っている。

$ pwd
/Library/Python/2.6/site-packages

$ ls -l PIL*
lrwxr-xr-x  1 root  admin  59  6 12 15:44 PIL -> /usr/local/Cellar/pil/1.1.7/lib/python2.6/site-packages/PIL
lrwxr-xr-x  1 root  admin  63  6 12 15:44 PIL.pth -> /usr/local/Cellar/pil/1.1.7/lib/python2.6/site-packages/PIL.pth

pip で –upgrade すると困ったことになる

このとき、たとえば pip の -U もしくは –upgrade オプションを利用してなにかパッケージをUpgradeするとすると以下のように、すでにある依存パッケージをインストールしょうとする。

$ sudo pip install sphinxcontrib-blockdiag -U
Downloading/unpacking sphinxcontrib-blockdiag
  Downloading sphinxcontrib-blockdiag-0.8.3.tar.gz
  Running setup.py egg_info for package sphinxcontrib-blockdiag
    warning: no files found matching 'CHANGES.*'
Downloading/unpacking blockdiag>=0.8.0 (from sphinxcontrib-blockdiag)
  Running setup.py egg_info for package blockdiag
Downloading/unpacking Sphinx>=0.6 (from sphinxcontrib-blockdiag)
  Downloading Sphinx-1.0.7.tar.gz

  .. 以下略

上記の例では、upgrade対象のパッケージ sphinxcontrib-blockdiag の依存として setup.py での requires = [‘blockdiag>=0.8.0’, ‘Sphinx>=0.6’] の設定のとおり、

  • blockdiag>=0.8.0
  • Sphinx>=0.6

に依存しているが、先ほどの pip freeze の結果からすると、私の環境ではすでに

  • blockdiag==0.8.4
  • Sphinx==1.0.7

となっているので、いちいちダウンロードする必要もなければ、これらの依存パッケージをInstallする必要もないはずだが、これをダウンロードしてインストールしようとする。(そしてblockdiagが依存しているPILをinstall使用とする段になってビルドに失敗する)

一方で、例えば以下のようにパケージをいったん uninstall してから install してみると 依存パッケージをきちんと検出して Requirement already satisfied と表示される。

$ sudo pip uninstall sphinxcontrib-blockdiag
Uninstalling sphinxcontrib-blockdiag:
  /Library/Python/2.6/site-packages/sphinxcontrib/blockdiag.py
  /Library/Python/2.6/site-packages/sphinxcontrib/blockdiag.pyc
  /Library/Python/2.6/site-packages/sphinxcontrib_blockdiag-0.8.3-py2.6-nspkg.pth
  /Library/Python/2.6/site-packages/sphinxcontrib_blockdiag-0.8.3-py2.6.egg-info
Proceed (y/n)? y
  Successfully uninstalled sphinxcontrib-blockdiag

$ sudo pip install sphinxcontrib-blockdiag
Downloading/unpacking sphinxcontrib-blockdiag
  Running setup.py egg_info for package sphinxcontrib-blockdiag
    warning: no files found matching 'CHANGES.*'
Requirement already satisfied (use --upgrade to upgrade): blockdiag>=0.8.0 in /Library/Python/2.6/site-packages (from sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): Sphinx>=0.6 in /Library/Python/2.6/site-packages (from sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Library/Python/2.6/site-packages (from blockdiag>=0.8.0->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): PIL in /usr/local/Cellar/pil/1.1.7/lib/python2.6/site-packages/PIL (from blockdiag>=0.8.0->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): funcparserlib in /Library/Python/2.6/site-packages (from blockdiag>=0.8.0->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): webcolors in /Library/Python/2.6/site-packages (from blockdiag>=0.8.0->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=0.8 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): Jinja2>=2.2 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-blockdiag)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.5 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-blockdiag)
Installing collected packages: sphinxcontrib-blockdiag
  Running setup.py install for sphinxcontrib-blockdiag
    warning: no files found matching 'CHANGES.*'
    Skipping installation of /Library/Python/2.6/site-packages/sphinxcontrib/__init__.py (namespace package)
    Installing /Library/Python/2.6/site-packages/sphinxcontrib_blockdiag-0.8.3-py2.6-nspkg.pth
Successfully installed sphinxcontrib-blockdiag
Cleaning up...

sphinxjp.themecoreでもやってみた

  • やはり –upgrade オプションだと setuptools をダウンロードしようとする。
  • 一旦 uninstall してから install すると、正しく依存をチェックし、すでに新しいパッケージはinstallダウンロードしない。
$ sudo pip install sphinxjp.themecore --upgrade
Downloading/unpacking sphinxjp.themecore
  Downloading sphinxjp.themecore-0.1.1.tar.gz
  Running setup.py egg_info for package sphinxjp.themecore
Downloading/unpacking setuptools (from sphinxjp.themecore)
  Running setup.py egg_info for package setuptools
Downloading/unpacking docutils (from sphinxjp.themecore)

依存パッケージのダウンロードが始まったのでいったん Ctrl-C で止める

$ sudo pip uninstall sphinxjp.themecore
Uninstalling sphinxjp.themecore:
  /Library/Python/2.6/site-packages/sphinxjp
  /Library/Python/2.6/site-packages/sphinxjp.themecore-0.1.0-py2.6.egg-info
Proceed (y/n)? y
  Successfully uninstalled sphinxjp.themecore

$ sudo pip install sphinxjp.themecore
Downloading/unpacking sphinxjp.themecore
  Running setup.py egg_info for package sphinxjp.themecore
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Library/Python/2.6/site-packages (from sphinxjp.themecore)
Requirement already satisfied (use --upgrade to upgrade): docutils in /Library/Python/2.6/site-packages (from sphinxjp.themecore)
Requirement already satisfied (use --upgrade to upgrade): sphinx in /Library/Python/2.6/site-packages (from sphinxjp.themecore)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=0.8 in /Library/Python/2.6/site-packages (from sphinx->sphinxjp.themecore)
Requirement already satisfied (use --upgrade to upgrade): Jinja2>=2.2 in /Library/Python/2.6/site-packages (from sphinx->sphinxjp.themecore)
Installing collected packages: sphinxjp.themecore
  Running setup.py install for sphinxjp.themecore
    Skipping installation of /Library/Python/2.6/site-packages/sphinxjp/__init__.py (namespace package)
    Installing /Library/Python/2.6/site-packages/sphinxjp.themecore-0.1.1-py2.6-nspkg.pth
Successfully installed sphinxjp.themecore
Cleaning up...

nwdiag でもやってみた

  • やはり –upgrade オプションだと setuptools , PIL をダウンロードし、PILのビルドに失敗する。
  • 一旦 uninstall してから install すると、正しく依存をチェックし、すでに新しいパッケージはinstallダウンロードしない。
$ sudo pip install nwdiag --upgrade
Downloading/unpacking nwdiag
  Downloading nwdiag-0.2.7.tar.gz
  Running setup.py egg_info for package nwdiag
Downloading/unpacking setuptools (from nwdiag)
  Running setup.py egg_info for package setuptools
Downloading/unpacking PIL (from nwdiag)
  Running setup.py egg_info for package PIL
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py
Downloading/unpacking funcparserlib (from nwdiag)
  Running setup.py egg_info for package funcparserlib
    warning: no files found matching '*' under directory 'tests'
Downloading/unpacking blockdiag>=0.8.1 (from nwdiag)
  Running setup.py egg_info for package blockdiag
Downloading/unpacking webcolors (from blockdiag>=0.8.1->nwdiag)
  Running setup.py egg_info for package webcolors
Installing collected packages: nwdiag, setuptools, PIL, funcparserlib, blockdiag, webcolors
  Found existing installation: nwdiag 0.2.5
    Uninstalling nwdiag:
      Successfully uninstalled nwdiag
  Running setup.py install for nwdiag
    Installing nwdiag script to /usr/local/bin
  Found existing installation: setuptools 0.6c12dev-r88846
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools
    Installing easy_install script to /usr/local/bin
    Installing easy_install-2.6 script to /usr/local/bin
  Found existing installation: PIL 1.1.7
    Uninstalling PIL:
      Successfully uninstalled PIL
  Running setup.py install for PIL
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py
    --- using frameworks at /System/Library/Frameworks
    building '_imaging' extension
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -DHAVE_LIBJPEG -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _imaging.c -o build/temp.macosx-10.6-universal-2.6/_imaging.o
    _imaging.c:3017: warning: initialization from incompatible pointer type
    _imaging.c:3077: warning: initialization from incompatible pointer type
    /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
    Installed assemblers are:
    /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
    /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
    _imaging.c:3017: warning: initialization from incompatible pointer type
    _imaging.c:3077: warning: initialization from incompatible pointer type
    _imaging.c:3281: fatal error: error writing to -: Broken pipe
    compilation terminated.
    _imaging.c:3017: warning: initialization from incompatible pointer type
    _imaging.c:3077: warning: initialization from incompatible pointer type
    lipo: can't open input file: /var/tmp//ccLE76AD.out (No such file or directory)
    error: command 'gcc-4.2' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/Library/Python/2.6/site-packages/build/PIL/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-B4nLVD-record/install-record.txt:
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py

running install

running build

running build_py

running build_ext

--- using frameworks at /System/Library/Frameworks

building '_imaging' extension

gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -DHAVE_LIBJPEG -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _imaging.c -o build/temp.macosx-10.6-universal-2.6/_imaging.o

_imaging.c:3017: warning: initialization from incompatible pointer type

_imaging.c:3077: warning: initialization from incompatible pointer type

/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed

Installed assemblers are:

/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64

/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386

_imaging.c:3017: warning: initialization from incompatible pointer type

_imaging.c:3077: warning: initialization from incompatible pointer type

_imaging.c:3281: fatal error: error writing to -: Broken pipe

compilation terminated.

_imaging.c:3017: warning: initialization from incompatible pointer type

_imaging.c:3077: warning: initialization from incompatible pointer type

lipo: can't open input file: /var/tmp//ccLE76AD.out (No such file or directory)

error: command 'gcc-4.2' failed with exit status 1

----------------------------------------
  Rolling back uninstall of PIL
Command /usr/bin/python -c "import setuptools;__file__='/Library/Python/2.6/site-packages/build/PIL/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-B4nLVD-record/install-record.txt failed with error code 1
Storing complete log in /Users/joey/.pip/pip.log
$ sudo pip uninstall nwdiag
Uninstalling nwdiag:
  /Library/Python/2.6/site-packages/nwdiag
  /Library/Python/2.6/site-packages/nwdiag-0.2.7-py2.6.egg-info
  /Library/Python/2.6/site-packages/nwdiag_sphinxhelper.py
  /Library/Python/2.6/site-packages/nwdiag_sphinxhelper.pyc
  /Library/Python/2.6/site-packages/sphinxcontrib_nwdiag.py
  /Library/Python/2.6/site-packages/sphinxcontrib_nwdiag.pyc
  /usr/local/bin/nwdiag
Proceed (y/n)? y
  Successfully uninstalled nwdiag

$ sudo pip uninstall sphinxcontrib-nwdiag
Uninstalling sphinxcontrib-nwdiag:
  /Library/Python/2.6/site-packages/sphinxcontrib/nwdiag.py
  /Library/Python/2.6/site-packages/sphinxcontrib/nwdiag.pyc
  /Library/Python/2.6/site-packages/sphinxcontrib_nwdiag-0.1.1-py2.6-nspkg.pth
  /Library/Python/2.6/site-packages/sphinxcontrib_nwdiag-0.1.1-py2.6.egg-info
Proceed (y/n)? y
  Successfully uninstalled sphinxcontrib-nwdiag

$ pip freeze |grep nwdiag
$

$ sudo pip install nwdiag
Downloading/unpacking nwdiag
  Running setup.py egg_info for package nwdiag
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Library/Python/2.6/site-packages (from nwdiag)
Requirement already satisfied (use --upgrade to upgrade): PIL in /usr/local/Cellar/pil/1.1.7/lib/python2.6/site-packages/PIL (from nwdiag)
Requirement already satisfied (use --upgrade to upgrade): funcparserlib in /Library/Python/2.6/site-packages (from nwdiag)
Requirement already satisfied (use --upgrade to upgrade): blockdiag>=0.8.1 in /Library/Python/2.6/site-packages (from nwdiag)
Requirement already satisfied (use --upgrade to upgrade): webcolors in /Library/Python/2.6/site-packages (from blockdiag>=0.8.1->nwdiag)
Installing collected packages: nwdiag
  Running setup.py install for nwdiag
    Installing nwdiag script to /usr/local/bin
Successfully installed nwdiag
Cleaning up...

$ sudo pip install sphinxcontrib-nwdiag
Downloading/unpacking sphinxcontrib-nwdiag
  Downloading sphinxcontrib-nwdiag-0.1.1.tar.gz
  Running setup.py egg_info for package sphinxcontrib-nwdiag
    warning: no files found matching 'CHANGES.*'
Requirement already satisfied (use --upgrade to upgrade): nwdiag>=0.2.3 in /Library/Python/2.6/site-packages (from sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): Sphinx>=0.6 in /Library/Python/2.6/site-packages (from sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Library/Python/2.6/site-packages (from nwdiag>=0.2.3->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): PIL in /usr/local/Cellar/pil/1.1.7/lib/python2.6/site-packages/PIL (from nwdiag>=0.2.3->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): funcparserlib in /Library/Python/2.6/site-packages (from nwdiag>=0.2.3->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): blockdiag>=0.8.1 in /Library/Python/2.6/site-packages (from nwdiag>=0.2.3->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=0.8 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): Jinja2>=2.2 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.5 in /Library/Python/2.6/site-packages (from Sphinx>=0.6->sphinxcontrib-nwdiag)
Requirement already satisfied (use --upgrade to upgrade): webcolors in /Library/Python/2.6/site-packages (from blockdiag>=0.8.1->nwdiag>=0.2.3->sphinxcontrib-nwdiag)
Installing collected packages: sphinxcontrib-nwdiag
  Running setup.py install for sphinxcontrib-nwdiag
    warning: no files found matching 'CHANGES.*'
    Skipping installation of /Library/Python/2.6/site-packages/sphinxcontrib/__init__.py (namespace package)
    Installing /Library/Python/2.6/site-packages/sphinxcontrib_nwdiag-0.1.1-py2.6-nspkg.pth
Successfully installed sphinxcontrib-nwdiag
Cleaning up...