Changelog

The purpose of this document is to list all of the notable changes to this project. The format was inspired by Keep a Changelog. This project adheres to semantic versioning.

Release 8.4 (2021-03-09)

Enhance deb_pkg_tools.package.update_conffiles() with exclude support: If an entry in the DEBIAN/conffiles starts with an exclamation mark (optionally followed by whitespace) that entry will be omitted from the final file.

Rationale: In general I like the automatic DEBIAN/conffiles updating but I’ve encountered circumstances in which it is really inconvenient not being able to exclude one or two specific files.

Release 8.3 (2020-05-11)

Minor improvements to the deb_pkg_tools.deb822 module:

Slightly relax deb822 parsing
Leading and trailing comment blocks and empty lines that directly precede or follow a paragraph of control fields are now silently ignored. This is intended to improve compatibility with python-debian.
Improve deb822 parse errors
Shortly after I started using deb-pkg-tools 8.0 it became apparent that deb_pkg_tools.deb822.parse_deb822() is quite a bit more strict than the previous usage of python-debian. While I don’t necessarily consider this a bad thing, it definitely highlighted a weak spot: The error messages didn’t include filenames or line numbers. This is now fixed.

Release 8.2 (2020-05-02)

Removed textwrap.indent() usage from deb_pkg_tools.deb822 module because this function isn’t available on Python 2.7 which deb-pkg-tools still supports. Also added a regression test.

Note

While I definitely intend to drop Python 2 support in my open source projects at some point, right now is not the time for that just yet.

Release 8.1 (2020-04-25)

Release 8.0 (2020-04-25)

Dropped GPL2 dependencies

The main purpose of this release was to resolve issue #20 by dropping two GPL2 dependencies to avoid having to change the deb-pkg-tools license from MIT to GPL2:

python-apt

This dependency was previously used for Debian version comparison. This functionality has now been implemented in pure Python, for more details please refer to the new deb_pkg_tools.version.native module.

Note

If this change introduces regressions for you, take a look at the deb_pkg_tools.version.PREFER_DPKG variable, it may help as a temporary workaround. Also please report the regression 😇.

python-debian
This dependency was previously used for Debian binary control file parsing. This functionality has now been implemented in pure Python, for more details please refer to the new deb_pkg_tools.deb822 module.
Updated Python compatibility
Python 3.8 is now officially supported, 3.4 is no longer supported.
Fixed deprecation warnings
Fixed humanfriendly 8.0 deprecation warnings and bumped requirements I authored that went through the same process. Also defined the first deprecated aliases in the deb-pkg-tools code base (in the process of implementing the functionality required to drop the GPL2 dependencies).
Quality boost for deb_pkg_tools.control module

The deb_pkg_tools.control module saw a lot of small changes to make the handling of case insensitivity and byte strings versus Unicode strings more consistent. The most important changes:

  • All functions that return dictionaries now return the same type of case insensitive dictionaries (see Deb822).
  • The complete module now expects and uses Unicode strings internally. Character encoding and decoding is only done when control files are read from and written to disk.

Release 7.0 (2020-02-07)

Code changes:

  • Make update_conffiles() optional (requested in #19) in the Python API.
  • Make find_object_files() use a builtin exclude list of filename patterns to ignore.
  • Start using __all__ to control what is exported:
    • This change is backwards incompatible in the sense that until now imports were exposed to the outside world, however for anyone to actually use this would imply not having read the documentation, so this doesn’t really bother me.
    • In theory this change could be backwards incompatible in a bad way if I omitted __all__ entries that should have been exported. I did double check but of course I can’t be 100% sure (the deb_pkg_tools.* modules currently span almost 6000 lines including whitespace and comments).
    • I decided to bump the major version number because of the potential for import errors caused by the introduction of __all__.

Documentation updates:

  • Simplified the overview of environment variables in the readme by properly documenting individual options and linking to their documentation entries. Over the years I’ve picked up the habit of treating my documentation just like my code: Make sure everything is defined in a single place (DRY), as close as possible to the place where it is used. Properly documenting all of the module variables that are based on environment variables and linking to those from the readme frees me from the burden of explaining things in more than one place. This is good because multiple explanations increase the chance of documentation becoming outdated or contradictoring itself, which are definitely problems to be avoided whenever possible.
  • Started using :man: role to link to Linux manual pages.
  • Changed Read the Docs URL (s/\.org$/.io/g).

Documented variables:

Module variable Environment variable
deb_pkg_tools.gpg.FORCE_ENTROPY $DPT_FORCE_ENTROPY
deb_pkg_tools.package.ALLOW_CHOWN $DPT_CHOWN_FILES
deb_pkg_tools.package.ALLOW_FAKEROOT_OR_SUDO $DPT_ALLOW_FAKEROOT_OR_SUDO
deb_pkg_tools.package.ALLOW_HARD_LINKS $DPT_HARD_LINKS
deb_pkg_tools.package.ALLOW_RESET_SETGID $DPT_RESET_SETGID
deb_pkg_tools.package.BINARY_PACKAGE_ARCHIVE_EXTENSIONS  
deb_pkg_tools.package.DEPENDENCY_FIELDS  
deb_pkg_tools.package.DIRECTORIES_TO_REMOVE  
deb_pkg_tools.package.FILES_TO_REMOVE  
deb_pkg_tools.package.PARSE_STRICT $DPT_PARSE_STRICT
deb_pkg_tools.package.ROOT_GROUP $DPT_ROOT_GROUP
deb_pkg_tools.package.ROOT_USER $DPT_ROOT_USER
deb_pkg_tools.repo.ALLOW_SUDO $DPT_SUDO

Release 6.1 (2020-02-05)

Implemented a feature requested from me via private email:

Problem: When filename parsing of *.deb archives fails to recognize a package name, version and architecture encoded in the filename (delimited by underscores) then deb-pkg-tools reports an error and aborts:

ValueError: Filename doesn't have three underscore separated components!

Solution: Setting the environment variable $DPT_PARSE_STRICT to false changes this behavior so that the required information is extracted from the package metadata instead of reporting an error.

For now the default remains the same (an error is reported) due to backwards compatibility and the principle of least surprise (for those who previously integrated deb-pkg-tools). This will likely change in the future.

Miscellaneous changes:

  • Use ‘console’ highlighting in readme (prompt are now highlighted).
  • Added license=MIT to setup.py script.
  • Bumped copyright to 2020.

Release 6.0 (2019-09-13)

  • Enable compatibility with newer python-apt releases:
    • The test suite has been modified to break on Travis CI when python-apt should be available but isn’t (when the Python virtual environment is based on a Python interpreter provided by Ubuntu, currently this applies to all build environments except Python 3.7).
    • The idea behind the test suite change is to verify that the conditional import chain in version.py always succeeds (on Travis CI, where I control the runtime environment).
    • This was added when after much debugging I finally realized why the new Ubuntu 18.04 build server I’d created was so awfully slow: The conditional import chain had been “silently broken” without me realizing it, except for the fact that using the fall back implementation based on dpkg --compare-versions to sort through thousands of version numbers was rather noticeably slow… 😇
  • Make python-memcached an optional dependency in response to #13.
  • Dropped Python 2.6 compatibility.

Release 5.2 (2018-11-17)

Promote python-debian version constraint into a conditional dependency.

Recently I constrained the version of python-debian to work around a Python 2.6 incompatibility. This same incompatibility is now biting me in the py2deb setup on Travis CI and after fighting that situation for a while I decided it may be better (less convoluted) to fix this in deb-pkg-tools instead (at the source of the problem, so to speak).

Release 5.1.1 (2018-10-26)

Bug fix for logic behind deb_pkg_tools.GPGKey.existing_files property: The configured directory wasn’t being scanned in combination with GnuPG < 2.1 even though the use of directory has become the preferred way to configure GnuPG < 2.1 as well as GnuPG >= 2.1 (due to the GnuPG bug mentioned in the release notes of release 5.1).

Release 5.1 (2018-10-26)

Added the deb_pkg_tools.gpg.GPGKey.identifier property that uses the gpg --list-keys --with-colons command to introspect the key pair and extract a unique identifier:

  • When a fingerprint is available in the output this is the preferred value.
  • Otherwise the output is searched for a key ID.

If neither of these values is available an exception is raised.

Note

While testing this I noticed that the old style gpg --no-default-keyring --keyring=… --secret-keyring=… commands don’t support the --list-keys command line option. The only workaround for this is to use the directory property (which triggers the use of --homedir) instead of the public_key_file and secret_key_file properties. This appears to be due to a bug in older GnuPG releases (see this mailing list thread).

Release 5.0 (2018-10-25)

GnuPG >= 2.1 compatibility for repository signing.

This release became rather more involved than I had hoped it would 😇 because of backwards incompatibilities in GnuPG >= 2.1 that necessitated changes in the API that deb-pkg-tools presents to its users:

  • The --secret-keyring option has been obsoleted and is ignored and the suggested alternative is the use of an ephemeral home directory which changes how a key pair is specified.
  • This impacts the API of the deb_pkg_tools.gpg.GPGKey class as well as the repos.ini support in deb_pkg_tools.repo.update_repository().

The documentation has been updated to explain all of this, refer to the deb_pkg_tools.gpg module for details. Detailed overview of changes:

  • The deb_pkg_tools.gpg.GPGKey class is now based on property-manager and no longer uses instance variables, because this made it easier for me to split up the huge __init__() method into manageable chunks.

    A side effect is that __init__() no longer supports positional arguments which technically speaking is backwards incompatible (although I never specifically intended it to be used like that).

  • The deb_pkg_tools.gpg.GPGKey class now raises an exception when it detects that the use of an isolated key pair is intended but the directory option has not been provided even though GnuPG >= 2.1 is being used. While this exception is new, the previous behavior on GnuPG >= 2.1 was anything but sane, so any thoughts about the backwards compatibility of this new exception are a moot point.

  • The deb_pkg_tools.gpg.GPGKey used to raise TypeError when a key pair is explicitly specified but only one of the two expected files exists, in order to avoid overwriting files not “owned” by deb-pkg-tools. An exception is still raised but the type has been changed to EnvironmentError because I felt that it was more appropriate. This is technically backwards incompatible but I’d be surprised if this affects even a single user…

  • The repository activation fall back test (that generates an automatic signing key in order to generate Release.gpg) was failing for me on Ubuntu 18.04 and in the process of debugging this I added support for InRelease files. In the end this turned out to be irrelevant to the issue at hand, but I saw no harm in keeping the InRelease support. This is under the assumption that the presence of an InRelease file shouldn’t disturb older apt-get versions (which seems like a sane assumption to me - it’s just a file on a webserver, right?).

  • Eventually I found out that the repository activation fall back test was failing due to the key type of the automatic signing key that’s generated during the test: As soon as I changed that from DSA to RSA things started working.

  • GnuPG profile directory initialization now applies 0700 permissions to avoid noisy warnings from GnuPG.

  • Added Python 3.7 to tested and and supported versions.

  • Improved update_repository() documentation.

  • Moved function result caching to humanfriendly.decorators.

  • I’ve changed Depends to Recommends in stdeb.cfg, with the following rationale:

    • The deb-pkg-tools package provides a lot of loosely related functionality depending on various external commands. For example building of Debian binary packages requires quite a few programs to be installed.
    • But not every use case of deb-pkg-tools requires all of these external commands, so demanding that they always be installed is rather inflexible.
    • In my specific case this dependency creep blocked me from building lightweight tools on top of deb-pkg-tools, because the dependency chain would pull in a complete build environment. That was more than I bargained for when I wanted to use a few utility functions in deb-pkg-tools 😅.
    • With this change, users are responsible for installing the appropriate packages. But then I estimate that less than one percent of my users are actually affected by this change, because of the low popularity of solutions like stdeb and py2deb 😇.
    • Only the python-apt package remains as a strict dependency instead of a recommended dependency, see 757286fc8ce for the rationale.
  • Removed python-apt intersphinx reference (for now).

  • Added this changelog to the repository and documentation.

Release 4.5 (2018-02-25)

Improved robustness of dpkg-shlibdeps and strip integration (followup to release 4.4).

Release 4.4 (2018-02-25)

Integrated support for dpkg-shlibdeps (inspired by py2deb).

I first started (ab)using dpkg-shlibdeps in the py2deb project and have since missed this functionality in other projects like deb-pkg-tools so have decided to move some stuff around :-).

Release 4.3 (2018-02-25)

  • Make mandatory control field validation reusable.
  • Include documentation in source distributions.
  • Restore Python 2.6 compatibility in test suite.

Release 4.2 (2017-07-10)

Implement cache invalidation (follow up to #12).

Release 4.1 (2017-07-10)

  • Merged pull request #11: State purpose of project in readme.
  • Improve dependency parsing: Add more Depends like fields (fixes #12).
  • Start using humanfriendly.testing to mark skipped tests.
  • Changed Sphinx documentation theme.
  • Add Python 3.6 to tested versions.

Release 4.0.2 (2017-02-02)

Bug fix for inheritance of AlternativeRelationship. This fixes the following error when hashing relationship objects:

AttributeError: 'AlternativeRelationship' object has no attribute 'operator'

I’d like to add tests for this but lack the time to do so at this moment, so hopefully I can revisit this later when I have a bit more time 😇.

Release 4.0.1 (2017-02-01)

  • Bug fix: Swallow unpickling errors instead of propagating them.

    In general I am very much opposed to Python code that swallows exceptions when it doesn’t know how to handle them, because it can inadvertently obscure an issue’s root cause and/or exacerbate the issue.

    But caching deserves an exception. Any code that exists solely as an optimization should not raise exceptions caused by the caching logic. This should avoid the following traceback which I just ran into:

    Traceback (most recent call last):
      File ".../lib/python2.7/site-packages/deb_pkg_tools/cli.py", line 382, in with_repository_wrapper
        with_repository(directory, \*command, cache=cache)
      File ".../lib/python2.7/site-packages/deb_pkg_tools/repo.py", line 366, in with_repository
        cache=kw.get('cache'))
      File ".../lib/python2.7/site-packages/deb_pkg_tools/repo.py", line 228, in update_repository
        cache=cache)
      File ".../lib/python2.7/site-packages/deb_pkg_tools/repo.py", line 91, in scan_packages
        fields = dict(inspect_package_fields(archive, cache=cache))
      File ".../lib/python2.7/site-packages/deb_pkg_tools/package.py", line 480, in inspect_package_fields
        value = entry.get_value()
      File ".../lib/python2.7/site-packages/deb_pkg_tools/cache.py", line 268, in get_value
        from_fs = pickle.load(handle)
    ValueError: unsupported pickle protocol: 3
    
  • Added property-manager to intersphinx mapping (enabling links in the online documentation).

Release 4.0 (2017-01-31)

  • Added support for parsing of architecture restrictions (#9).
  • Switched deb_pkg_tools.deps to use property-manager and removed cached-property requirement in the process:
    • This change simplified the deb-pkg-tools code base by removing the deb_pkg_tools.compat.total_ordering and deb_pkg_tools.utils.OrderedObject classes.
    • The introduction of property-manager made it easier for me to extend deb_pkg_tools.deps with the changes required to support ‘architecture restrictions’ (issue #9).
  • Add Build-Depends to DEPENDS_LIKE_FIELDS. I noticed while testing with the example provided in issue #9 that the dependencies in the Build-Depends field weren’t being parsed. Given that I was working on adding support for parsing of architecture restrictions (as suggested in issue #9) this seemed like a good time to fix this 🙂.
  • Updated generate_stdeb_cfg().

About backwards compatibility:

I’m bumping the major version number because 754debc0b61 removed the deb_pkg_tools.compat.total_ordering and deb_pkg_tools.utils.OrderedObject classes and internal methods like _key() so strictly speaking this breaks backwards compatibility, however both of these classes were part of miscellaneous scaffolding used by deb-pkg-tools but not an intentional part of the documented API, so I don’t expect this to be particularly relevant to most (if not all) users of deb-pkg-tools.

Release 3.1 (2017-01-27)

  • Merged pull request #8: Add support for *.udeb micro packages.
  • Updated test suite after merging #8.
  • Suggest memcached in stdeb.cfg.
  • Added readme target to Makefile.

Release 3.0 (2016-11-25)

This release was a huge refactoring to enable concurrent related package collection. In the process I switched from SQLite to the Linux file system (augmented by memcached) because SQLite completely collapsed under concurrent write activity (it would crap out consistently beyond a certain number of concurrent readers and writers).

Detailed changes:

  • Refactored makefile, setup script, Travis CI configuration, etc.
  • Bug fix: Don’t unnecessarily garbage collect cache.
  • Experimented with increased concurrency using SQLite Write-Ahead Log (WAL).
  • Remove redundant :py: prefixes from RST references
  • Fix broken RST references logged by sphinx-build -n.
  • Moved deb_pkg_tools.utils.compact() to humanfriendly.text.compact().
  • Fixed a broken pretty printer test.
  • Implement and enforce PEP-8 and PEP-257 compliance
  • Switch from SQLite to filesystem for package cache (to improve concurrency between readers and writers). The WAL did not improve things as much as I would have hoped…
  • Document and optimize filesystem based package metadata cache
  • Add some concurrency to deb-pkg-tools --collect (when more than one archive is given, the collection of related archives is performed concurrently for each archive given).
  • Re-implement garbage collection for filesystem based cache.
  • Improvements to interactive package collection:
    • Don’t use multiprocessing when a single archive is given because it’s kind of silly to fork subprocesses for no purpose at all.
    • Restored the functionality of the optional ‘cache’ argument because the new in memory / memcached / filesystem based cache is so simple it can be passed to multiprocessing workers.
  • Enable manual garbage collection (deb-pkg-tools --garbage-collect).
  • Updated usage in readme.
  • Improvements to interactive package collection:
    • A single spinner is rendered during concurrent collection (instead of multiple overlapping spinners that may not be synchronized).
    • The order of the --collect and --yes options no longer matters.
    • When the interactive spinner is drawn it will always be cleared, even if the operator presses Control-C (previously it was possible for the text cursor to remain hidden after deb-pkg-tools --collect was interrupted by Control-C).
  • Include command line interface in documentation.

Release 2.0 (2016-11-18)

Stop using the system wide temporary directory in order to enable concurrent builds.

Release 1.37 (2016-11-17)

Significant changes:

  • Prefer hard linking over copying of package archives from one directory to another.
  • Change Unicode output handling in command line interface. This revisits the ‘hack’ that I implemented in bc9b52419ea because I noticed today (after integrating humanfriendly.prompts.prompt_for_confirmation()) that the wrapping of sys.stdout disables libreadline support in interactive prompts (input() and raw_input()) which means readline hints are printed to stdout instead of being interpreted by libreadline, making interactive prompts rather hard to read :-s.

Miscellaneous changes:

  • Test Python 3.5 on Travis CI.
  • Don’t test tags on Travis CI.
  • Use pip instead of python setup.py install on Travis CI.
  • Uncovered and fixed a Python 3 incompatibility in the test suite.

Release 1.36 (2016-05-04)

Make it possible to integrate with GPG agent ($GPG_AGENT_INFO).

Release 1.35 (2015-09-24)

Include Breaks in control fields parsed like Depends.

Release 1.34.1 (2015-09-07)

Bug fix: Invalidate old package metadata caches (from before version 1.31.1).

Should have realized this much sooner of course but I didn’t, for which my apologies if this bit anyone like it bit me 😇. I wasted two hours trying to find out why something that was logically impossible (judging by the code base) was happening anyway. Cached data in the old format! 😒

Release 1.34 (2015-07-16)

Automatically embed usage in readme (easier to keep up to date 😇).

Release 1.33 (2015-07-16)

Added deb_pkg_tools.control.create_control_file() function.

Release 1.32.2 (2015-05-01)

Bug fixes for related package archive collection.

Release 1.32.1 (2015-05-01)

  • Include Pre-Depends in control fields parsed like Depends:.
  • Updated doctest examples with regards to changes in bebe413dcc5.
  • Improved documentation of parse_filename().

Release 1.32 (2015-04-23)

Improve implementation and documentation of collect_related_packages().

The result of the old implementation was dependent on the order of entries returned from os.listdir() which can differ from system to system (say my laptop vervsus Travis CI) and so caused inconsistently failing builds.

Release 1.31 (2015-04-11)

  • Extracted installed version discovery to re-usable function.
  • dpkg-scanpackages isn’t used anymore, remove irrelevant references.

Release 1.30 (2015-03-18)

Added deb_pkg_tools.utils.find_debian_architecture() function.

This function is currently not used by deb-pkg-tools itself but several of my projects that build on top of deb-pkg-tools need this functionality and all of them eventually got their own implementation. I’ve now decided to implement this once, properly, so that all projects can use the same tested and properly documented implementation (as simple as it may be).

Release 1.29.4 (2015-02-26)

Adapted pull request #5 to restore Python 3 compatibility.

Release 1.29.3 (2014-12-16)

Changed SQLite row factory to “restore” Python 3.4.2 compatibility.

The last Travis CI builds that ran on Python 3.4.1 worked fine and no changes were made in deb-pkg-tools since then so this is clearly caused by a change in Python’s standard library. This is an ugly workaround but it’s the most elegant way I could find to “restore” compatibility.

Release 1.29.2 (2014-12-16)

Bug fix: Don’t normalize Depends: lines.

Apparently dpkg-scanpackages and compatible re-implementations like the one in deb-pkg-tools should not normalize Depends: fields because apt can get confused by this. Somehow it uses either a literal comparison of the metadata or a comparison of the hash of the metadata to check if an updated package is available (I tried to find this in the apt sources but failed to do so due to my limited experience with C++). So when the Depends: line in the Packages.gz file differs from the Depends: line in the binary control file inside a *.deb apt will continuously re-download and install the same binary package…

Release 1.29.1 (2014-11-15)

Moved coerce_boolean() to humanfriendly package.

Release 1.29 (2014-10-19)

Merged pull request #4: Added $DPT_ALLOW_FAKEROOT_OR_SUDO and $DPT_CHOWN_FILES environment variables to make sudo optional.

Release 1.28 (2014-09-17)

Change location of package cache when os.getuid() == 0.

Release 1.27.3 (2014-08-31)

Sanitize permissions of DEBIAN/{pre,post}{inst,rm} and etc/sudoers.d/*.

Release 1.27.2 (2014-08-31)

Improve Python 2.x/3.x compatibility (return lists explicitly).

Release 1.27.1 (2014-08-31)

  • Bug fix for SQLite cache string encoding/decoding on Python 3.x.
  • Bug fix for check_package() on Python 3.x.
  • Bug fix for obscure Python 3.x issue (caused by mutating a list while iterating it).
  • Make collect_related_packages() a bit faster (actually quite a lot when dpkg --compare-versions is being used 🙂).
  • Make deb_pkg_tools.control.* less verbose.

Release 1.27 (2014-08-31)

  • Added command line interface for static checks (with improved test coverage).
  • Made collect_related_packages() a bit faster.
  • “Refine” entry collection strategy for Travis CI.

Release 1.26.4 (2014-08-30)

Restore Python 3.x compatibility (failing build).

Release 1.26.3 (2014-08-30)

Still not enough entropy on Travis CI, let’s see if we can work around that…

I tried to fix this using rng-tools in 3c372c3097f but that didn’t work out due to the way OpenVZ works. This commit introduces a more general approach that will hopefully work on OpenVZ and other virtualized environments, we’ll see…

Release 1.26.2 (2014-08-30)

  • Restore Python 3 compatibility.
  • Improve test coverage.
  • Try to work around lack of entropy on Travis CI.

Release 1.26 (2014-08-30)

Add static analysis to detect version conflicts.

Release 1.25 (2014-08-30)

Make collect_related_packages() 5x faster:

  • Use high performance decorator to memoize overrides of Relationship.matches().
  • Exclude conflicting packages from all further processing as soon as they are found.
  • Moved the dpkg comparison cache around.
  • Removed Version.__hash__().

Release 1.24.1 (2014-08-26)

Bug fix for unused parameter in 442d67cf4dd.

Release 1.24 (2014-08-26)

Normalize setgid bits (because dpkg-deb doesn’t like them).

Release 1.23.4 (2014-08-04)

Merged pull request #2: Improve platform compatibility with environment variables.

  • Added user-name and user-group overrides ($DPT_ROOT_USER, $DPT_ROOT_GROUP) for systems that don’t have a root group or when root isn’t a desirable consideration when building packages.
  • Can now disable hard-links ($DPT_HARD_LINKS). The cp -l parameter is not supported on Mavericks 10.9.2.
  • Replaced du -sB with du -sk (not supported on Mavericks 10.9.2).
  • Can now disable sudo ($DPT_SUDO) since it’s sometimes not desirable and not required just to build the package (for example on MacOS, refer to pull request #2 for an actual use case).

Release 1.23.3 (2014-06-27)

Bug fix for copy_package_files().

Release 1.23.2 (2014-06-25)

Further improvements to collect_packages().

Release 1.23.1 (2014-06-25)

Bug fix: Don’t swallow keyboard interrupt in collect_packages() wrapper.

Release 1.23 (2014-06-25)

Added group_by_latest_versions() function.

Release 1.22.6 (2014-06-22)

Try to fix cache deserialization errors on older platforms (refer to the commit message of 8b04dfcd4d3 for more details about the errors I’m talking about).

Release 1.22.5 (2014-06-22)

Preserving Python 2.x and Python 3.x compatibility is hard 😞.

Release 1.22.4 (2014-06-22)

Bug fix: Encode stdout/stderr as UTF-8 when not connected to a terminal.

Release 1.22.3 (2014-06-19)

Bug fix for Python 3 syntax compatibility.

Release 1.22.2 (2014-06-19)

Make the package cache resistant against deserialization errors.

Today I’ve been hitting zlib decoding errors and I’m 99% sure my disk isn’t failing (RAID 1 array). For now I’m inclined not to dive too deep into this, because there’s a very simple fix (see first line :-). For future reference, here’s the zlib error:

File ".../deb_pkg_tools/cache.py", line 299, in control_fields
  return self.cache.decode(self['control_fields'])
File ".../deb_pkg_tools/cache.py", line 249, in decode
  return pickle.loads(zlib.decompress(database_value))

error: Error -5 while decompressing data

Release 1.22.1 (2014-06-16)

  • Change clean_package_tree() to clean up __pycache__ directories.
  • Improved test coverage of check_duplicate_files().

Release 1.22 (2014-06-09)

Proof of concept: duplicate files check (static analysis).

Release 1.21 (2014-06-09)

Implement proper package metadata cache using SQLite 3.x (high performance).

I’ve been working on CPU and disk I/O intensive package analysis across hundreds of package archives which is very slow even on my MacBook Air with four cores and an SSD. I decided to rip the ad-hoc cache in scan_packages() out and refactor it into a more general purpose persistent, multiprocess cache implemented on top of SQLite 3.x.

Release 1.20.11 (2014-06-08)

Improve performance: Cache results of RelationshipSet.matches().

Release 1.20.10 (2014-06-08)

Make deb_pkg_tools.utils.atomic_lock() blocking by default.

Release 1.20.9 (2014-06-07)

Make it possible to ask a RelationshipSet for all its names.

Release 1.20.8 (2014-06-07)

Bug fix for Python 3.x compatibility.

Release 1.20.7 (2014-06-07)

Sanitize permission bits of root directory when building packages.

Release 1.20.6 (2014-06-07)

Switch to executor 1.3 which supports execute(command, fakeroot=True).

Release 1.20.5 (2014-06-05)

Added deb_pkg_tools.control.load_control_file() function.

Release 1.20.4 (2014-06-01)

Minor optimization that seems to make a major difference (without this optimization I would sometimes hit “recursion depth exceeded” errors).

Release 1.20.3 (2014-06-01)

Bug fix for Python 3.x compatibility (missed compat.basestring import).

Release 1.20.2 (2014-06-01)

Bug fix for Python 3.x incompatible syntax in newly added code.

Release 1.20.1 (2014-06-01)

Automatically create parent directories in atomic_lock class.

Release 1.20 (2014-06-01)

Re-implemented dpkg-scanpackages -m in Python to make it really fast.

Release 1.19 (2014-06-01)

Added function deb_pkg_tools.package.find_package_archives().

Release 1.18.5 (2014-05-28)

Bug fix for find_latest_version() introduced in commit 5bf01b0 (build failure on Travis CI).

Release 1.18.4 (2014-05-28)

Disable pretty printing of RelationshipSet objects by default.

Release 1.18.3 (2014-05-26)

  • Fixed sort order of deb_pkg_tools.package.PackageFile (changed field order)
  • Sanity check given arguments in deb_pkg_tools.package.find_latest_version().
  • Documented the exception that can be raised by deb_pkg_tools.package.parse_filename().

Release 1.18.2 (2014-05-26)

Change deb_pkg_tools.deps.parse_depends() to accept a list of dependencies.

Release 1.18.1 (2014-05-25)

  • Bug fix for last commit (avoid AttributeError on apt_pkg.version_compare).
  • Changed documentation of deb_pkg_tools.compat module.
  • Made doctest examples Python 3.x compatible (print() as function).
  • Integrate customized doctest checking in makefile.

Release 1.18 (2014-05-25)

Extract version comparison to separate module (with tests).

I wanted to re-use version sorting in several places so it seemed logical to group the related code together in a new deb_pkg_tools.version module. While I was at it I decided to write tests that make sure the results of compare_versions_with_python_apt() and compare_versions_with_dpkg() are consistent with each other and the expected behavior.

Release 1.17.7 (2014-05-18)

Made collect_related_packages() faster (by splitting inspect_package()).

Release 1.17.6 (2014-05-18)

Re-implemented dpkg_compare_versions() on top of apt.VersionCompare().

Release 1.17.5 (2014-05-18)

Moved Python 2.x / 3.x compatibility functions to a separate module.

Release 1.17.4 (2014-05-18)

  • Made pretty print tests compatible with Python 3.x.
  • Removed binutils and tar dependencies (these are no longer needed since the inspect_package() function now uses the dpkg-deb command).

Release 1.17.3 (2014-05-18)

  • Cleanup pretty printer, remove monkey patching hack, add tests.
  • Dedent string passed to deb822_from_string() (nice to use in tests).

Release 1.17.2 (2014-05-18)

  • Bug fix for output of deb-pkg-tools --inspect ... (broken in Python 3.x compatibility spree).
  • Monkey patch pprint so it knows how to ‘pretty print’ RelationshipSet (very useful to verify docstrings containing doctest blocks).
  • Improved test coverage of deb_pkg_tools.package.PackageFile.__lt__().

Release 1.17.1 (2014-05-18)

  • Bug fix for deb_pkg_tools.deps.parse_relationship().
  • Bug fix for inspect_package() (hard links weren’t recognized).
  • Added deb_pkg_tools.control.deb822_from_string() shortcut.
  • Various bug fixes for Python 2.6 and 3.x compatibility:
    • Bumped python-debian requirement to 0.1.21-nmu2 for Python 3.x compatibility
    • Changed logger.warn() to logger.warning() (the former is deprecated).
    • Fixed missing str_compatible decorator (Python 3.x compatibility).

Release 1.17 (2014-05-18)

Added collect_related_packages() function and deb-pkg-tools --collect command line interface.

Release 1.16 (2014-05-18)

  • Added relationship parsing/evaluation module (deb_pkg_tools.deps.*).
  • Bug fix for deb_pkg_tools.generate_stdeb_cfg().
  • Test suite changes:
    • Skip repository activation in test_command_line_interface() when not root.
    • Added an improvised slow test marker.

Release 1.15.2 (2014-05-16)

  • Added deb_pkg_tools.package.parse_filename() function.
  • Properly document deb_pkg_tools.package.ArchiveEntry named tuple.
  • Improved test coverage by testing command line interface.
  • Changed virtual environment handling in Makefile.

Release 1.15.1 (2014-05-10)

  • Bug fix for Python 3 compatibility.
  • Moved deb_pkg_tools.cli.with_repository() to deb_pkg_tools.repo.with_repository().
  • Submit test coverage from travis-ci.org to coveralls.io, add dynamic coverage statistics to README.rst.
  • Run more tests on travis-ci.org by running test suite as root (this gives the test suite permission to test things like apt-get local repository activation).
  • Improved test coverage of deb_pkg_tools.repository.update_repository() and deb_pkg_tools.gpg.GPGKey().

Release 1.15 (2014-05-10)

  • Merge pull request #1: Python 3 compatibility.
  • Document supported Python versions (2.6, 2.7 & 3.4).
  • Start using travis-ci.org to avoid dropping Python 3 compatibility in the future.
  • Update documented dependencies in README.rst.

Release 1.14.7 (2014-05-04)

Refactored deb_pkg_tools.utils.execute() into a separate package.

Release 1.14.6 (2014-05-03)

Bug fix for globbing support.

Release 1.14.5 (2014-05-03)

Added support for deb-pkg-tools --patch=CTRL_FILE --set="Name: Value".

Release 1.14.4 (2014-05-03)

Make update_repository() as “atomic” as possible.

Release 1.14.3 (2014-05-03)

Support for globbing in configuration file (repos.ini).

Release 1.14.2 (2014-04-29)

Bug fix: Typo in readme (found just after publishing of course 😉).

Release 1.14.1 (2014-04-29)

Added support for the system wide configuration file /etc/deb-pkg-tools/repos.ini.

Release 1.14 (2014-04-29)

  • Make repository generation user configurable (~/.deb-pkg-tools/repos.ini).
  • Test GPG key generation (awkward but useful, make it opt-in or opt-out?).
  • Make Python >= 2.6 dependency explicit in stdeb.cfg (part 2 :-).
  • Documentation bug fix: Update usage message and README.rst.

Release 1.13.2 (2014-04-28)

Bug fix: Respect the build_package(copy_files=False) option.

Release 1.13.1 (2014-04-28)

  • Try to detect removal of *.deb files in update_repository().
  • Bring test coverage back up to >= 90%.

Release 1.13 (2013-11-16)

Make inspect_package() report package contents. This was added to make it easier to write automated tests for deb-pkg-tools but may be useful in other circumstances and so became part of the public API 😇.

Release 1.12.1 (2013-11-03)

Make Python >= 2.6 dependency explicit in stdeb.cfg.

Release 1.12 (2013-11-03)

Make copy_package_files() more generally useful.

Release 1.11 (2013-11-02)

  • Improve deb_pkg_tools.gpg.GPGKey and related documentation.

Release 1.10.2 (2013-11-02)

Bug fix: Make update_repository() always remove old Release.gpg files.

Release 1.10.1 (2013-11-02)

Bug fix: Make update_repository() fully aware of apt_supports_trusted_option().

Release 1.10 (2013-11-02)

Use the [trusted=yes] option in sources.list when possible:

With this we no longer need a generated GPG key at all; we just skip all steps that have anything to do with GPG :-). Unfortunately we still need to be backwards compatible so the code to generate and manage GPG keys remains for now…

Release 1.9.9 (2013-10-22)

Remove automatic dependency installation (too much magic, silly idea).

Release 1.9.8 (2013-10-22)

Bug fixes for last commit (sorry about that!).

Release 1.9.7 (2013-10-22)

New deb-pkg-tools --with-repo=DIR COMMAND... functionality (only exposed in the command line interface for now).

Release 1.9.6 (2013-10-21)

Workaround for old and buggy versions of GnuPG 😞.

Release 1.9.5 (2013-10-20)

Bug fix for update_repository().

Release 1.9.4 (2013-10-20)

Change update_repository() to only rebuild repositories when contents have changed.

Release 1.9.3 (2013-10-20)

Make update_conffiles() work properly in Python < 2.7.

Release 1.9.2 (2013-10-20)

Enable overriding of GPG key used by the deb_pkg_tools.repo.* functions.

Release 1.9.1 (2013-10-20)

Made it possible not to copy the files in the build directory (build_package()).

Release 1.9 (2013-10-20)

Extracted GPG key generation into standalone function.

Release 1.8 (2013-10-20)

Automatic installation of required system packages.

Release 1.7.2 (2013-10-19)

Make copy_package_files() compatible with schroot environments.

Release 1.7.1 (2013-10-18)

Enable callers of update_repository() to set fields of Release files.

Release 1.7 (2013-10-16)

Change build_package() to automatically update DEBIAN/conffiles.

Release 1.6.2 (2013-10-13)

Bug fix: Make deb-pkg-tools -u and deb-pkg-tools -a compatible with schroot environments.

Release 1.6.1 (2013-10-12)

Added stdeb.cfg to MANIFEST.in.

Release 1.6 (2013-10-12)

  • Improved documentation of deb_pkg_tools.utils.execute().
  • Improved deb_pkg_tools.utils.execute(), implemented optional sudo support.

Release 1.5 (2013-10-12)

Automatically generate a GPG automatic signing key the first time it’s needed.

Release 1.4.3 (2013-10-12)

  • Made log messages more user friendly.
  • Made Debian package dependencies available from Python.

Release 1.4.2 (2013-10-12)

Make it possible to delete fields using patch_control_file().

Release 1.4.1 (2013-08-13)

Improved update_installed_size() (by using patch_control_file()).

Release 1.4 (2013-08-13)

Normalize field names in control files (makes merging easier).

Release 1.3.2 (2013-08-13)

Make build_package() sanitize file modes:

I was debating with myself for quite a while how far to go in these kinds of “sensible defaults”; there will always be someone who doesn’t want the behavior. I decided that those people shouldn’t be using deb-pkg-tools then :-) (I wonder how long it takes though, before I find myself in that group of people ;-).

Release 1.3.1 (2013-08-11)

Improved clean_package_tree() (better documentation, more files to ignore).

Release 1.3 (2013-08-11)

Added clean_package_tree() function.

Release 1.2 (2013-08-10)

Added patch_control_file() function.

Release 1.1.4 (2013-08-10)

Removed as much manual shell quoting as possible.

Release 1.1.3 (2013-08-10)

  • Silenced deb_pkg_tools.utils.execute()
  • Simplified deb_pkg_tools.package.inspect_package().

Release 1.1.2 (2013-08-07)

Started using coloredlogs.increase_verbosity().

Release 1.1.1 (2013-08-07)

Loosen up the requirements (stop using absolute version pinning).

Release 1.1 (2013-08-05)

Automatically run Lintian after building packages.

Release 1.0.3 (2013-08-04)

Improved wording of readme, fixed typo in docs.

Release 1.0.2 (2013-08-04)

Got rid of the use of shell pipes in order to detect “command not found” errors.

Release 1.0.1 (2013-08-04)

Brought test suite coverage up to 96% 🎉.

Release 1.0 (2013-07-26)

Initial commit with a focus on:

  • Building of Debian binary packages.
  • Inspecting the metadata of Debian binary packages.
  • Creation of trivial repositories based on collected package metadata.