This is a pirate version of Modularity (TM) module build

<<<<< Brainstorm dump
we'd like to ask for a help with a best way to write "module build" prototype.

As of now think of module as rpm repo with extra metadata (yaml file defining core packages of given module and also it's build requires (requires are just other modules), + module version)

so think of module webserver-1.0 (which has httpd, lighttpd) which build-requires e.g. module devel-2.0 (glibc ...) + core-1.0 (coreutils, bash) and perhaps httpd itself (webserver is probably wrong example for requiring itself but yeah ...)

What "module build webserver" means:

build package list from webserver (httpd) the sources would be referenced by their commit IDs

this will need to generate buildroot which consists of devel-2.0 and core-1.0 and will chainbuild (since some packages within module might depend on other packages from module ... nss, nspr ...). There might be some overrides involved, even when we didn't really think of them (perhaps they might not be necessary by design)

so now we have all builds ... then pungi-createrepo (or perhaps koji signed repo) takes it job and generates repository out of these build packages.

the receipt for module (yaml file) would be stored in dist-git. We don't have proper module depsolving yet but think of module as rpm metapackage ... which has all rpm depsolving capabilities.

It's not from my head so I hope that I translated it properly.

The idea is to work on some mock-based content-generator for quick prototyping. Basically we need chainbuild and generation of tags/buildroots/targets based on receipt if that makes sense.
   Check out skvidal's `mockchain`.  It may help?



RPM Build separate from module build


Architecture of build

User story

As a module creator, (referenced as The Creator) I want to rule all modules, but especially I want to create module core for some unknown reason ...

fedpkg co modules/core
cd core.git
cat > core.yaml
document: modulemd
version: 0
        name: core
        version: 0-1
        summary: The core module
        description: >
                The minimal set of packages for modularity.
                        - MIT
                content: [] # This should be populated by other tools
                        dependencies: True
                        fulltree: False
                            bash: ~ # Fundamental to the Linux CLI experience, scripting
                            bzip2: ~ # Standard CLI tool
                            chrony: ~ # Plumbing for timekeeping
                            coreutils: ~ # Fundamental to the Linux CLI experience, scripting
                            dnf: ~ # Plumbing for package management
                            dracut: ~ # Required to boot hardware and virt guests
                            filesystem: ~ # Basic directories
                            glibc: ~ # Plumbing required for all applications
                            grub2: ~ # Required to boot hardware and virt guests
                            gzip: ~ # Standard CLI tool
                            initscripts: ~ # support for third-party initscripts
                            iproute: ~ # Required for network initialization
                            kernel-core: ~ # Required to boot hardware and virt guests
                            less: ~ # Standard CLI tool
                            openssh-clients: ~ # Fundamental to the Linux CLI experience, scripting
                            procps-ng: ~ # Fundamental to the Linux CLI experience, scripting
                            rpm: ~ # Plumbing for package management
                            sed: ~ # Fundamental to the Linux CLI experience, scripting
                            sudo: ~ # Fundamental to the Linux CLI experience, scripting
                            systemd: ~ # Plumbing required for all non-container install types
                            tar: ~ # Standard CLI tool
                            util-linux: ~ # Fundamental to the Linux CLI experience, scripting
                            xz: ~ # Standard CLI tool
                            zlib: ~ # Standard CLI tool and library
fedpkg commit -pm "Initial commit"
fedpkg module-build --nowait # died as a virgin while waiting several years for kojira to regen all buildroot repos

card idea: what will be installed in buildroot?

A) whole module?
B) just core set of packages defined in koji by rel-eng (koji list-groups $BUILDROOT) + build deps of packages (current behavior for rpm builds)
C) just the build deps of the packages

C is winner! (note we'll most probably need to have at least empty build and srpm-build needs to contain minimal set of default packages ... for actually being able to do rpmbuild -bs file.spec) 

card idea: order of required modules (issue one package provided by multiple modules)
Module (build)requirements should be an ordered-set to ensure order of  required modules.  Build system is expected to respect such order.
Undefined is a winner? (can yaml guarantee order that will be passed to ordered dict?)
    I think yaml can do that, yes?

card idea: define where to read default set of arches for module build
big requirement here is set of arches in koji build-tag which needs to be set to non-zero value

$ koji taginfo f24-build | grep -i arches
Arches: armv7hl i686 x86_64


card idea: how to rebuild module without bumping rpm specs

Note: changing dist-tag (.el6, .fc24) requires:

1) rebuild of system-release (fedora-release, redhat-release ...) witch changed dist_tag macro in system-release rpm-spec
2) trigger build tag it to specific buildroot that we want to affect
3) wait for createrepo to finish.
4) rebuild desired package

idea: have $module-build metapackage containing macro affecting dist-tag in this particular module
Example disttag: httpd-2.4.18-1.webserver@2.4.18@11

card idea: automated generation/maintenance of targets
validation of module name/version (typos could result into generating multiple otherwise unusable targets)

automated part would be:
1. creation of target (if it doesn't exist)
2. maintenance of buildroot depenedencies (defined in yaml)

requirement of package-version which is supposed to be built as part of "building this module" would be solved by chainbuild?

do we need overrides

fedpkg module-build
validate name / version of module (name should be same as name of the repo)
check whether build target exists
check whether build tag inheritance matches module build requirement
check whether arches are matching expectations (card define where to read default set of arches for module build)

if needed create/change koji build-tag arches or inheritance
if needed create koji target (it's more "kojira effective" to create target after the build-tag-inheritance is in place ... triggers fewer tasks)

wait for koji createrepo tasks to finish (generates repodata above set of packages in build-tag)
perform (chain) build