You are viewing sharkcz

Previous Entry | Next Entry

dhorak1
If you are involved as a package maintainer in Fedora you have probably already heard "do the fix in primary and koji-shadow will pick it up". The procedure is that all builds in primary Fedora (x86) are replicated in the secondary Fedoras (ARM, PPC, s390) using as close buildroots as possible. Generally only when a build doesn't exist in secondary, because it fails to build, it is replaced by a newer build. The 2 levels of architectures exist because the secondary architectures are not so widespread among Fedora users and a failing build on secondary architecture doesn't affect the package flow in primary. I was told that the original koji-shadow wasn't meant for building secondary architectures and this functionality (the ability to include newer builds in buildroots) was only later added there by Dennis Gilmore. Another useful feature is importing of noarch builds instead of their rebuilding, it saves important portion of the buildsystem resources.

The koji-shadow tool can run in 2 modes - doing a single build (all missing dependencies will be also built) or scanning a tag and queue all missing builds. I will now describe the improvement koji-shadow received in the previous months and also some existing features.

  • koji-shadow supports 2 lists for packages that won't be built, these lists are merged together and the packages listed there are skipped, but it allows to use one list generated automagically from the ExcludeArch/ExclusiveArch tags in source RPMs and the second list can be manually maintained and contain packages where queueing a build is waste of resources, but it still wasn't decided whether it will be fixed or properly excluded,

  • you can use globs in the ignore list, so I can have "nodejs-*" on s390 and nothing from the nodejs stack will be included on s390 because it depends on v8 and v8 is exclusive for x86 and
    ARM architectures

  • originaly koji-shadow required all builds to be rebuilt to finish, which is not always the situation in the prefer-new mode, some of them can be skipped, now it always finishes when there are no more packages to built,

  • the single build mode wasn't compatible with the prefer-new option, it required all dependencies to be present, now when a tag is specified on command line, it is able to run in prefer-new mode, the tag is here to allow searching the replacements for the missing builds


  • running in the prefer-new mode originaly resulted in including the newest build when the exact build wasn't present, often causing dependency problem when a too new build was included, now the closest newer build is included

  • this is an existing capability, but became more important now, sometimes a build in secondary Koji is successful, but causes additional broken dependencies or simply doesn't work, this is the use-case for "substitute", so the bad build can be replaced by an older (or newer), but good one

    And what you can expect in the future? Karsten prepared a patch to print information which failed builds are blocking the most other build and also has logging improvements in development, right now I use the logging feature of screen to capture the output. In the longer term work is being done to integrate koji-shadow with fedmsg and start builds when they finish in primary and not to rely on periodical scans. Naturally additional ideas are welcome, when patches will be attached the better :-)
  • Profile

    dhorak1
    sharkcz
    sharkcz

    Latest Month

    July 2014
    S M T W T F S
      12345
    6789101112
    13141516171819
    20212223242526
    2728293031  
    Powered by LiveJournal.com
    Designed by Lilia Ahner