Fedora Modularity Documentation

Wiki pages are great for collaboration. But they are not that great in getting people’s attention. They can also become pretty messy and hard to navigate trough when using multiple pages that are related to each other – like documentation – which was what we had there. We needed something better. Something that would make it easy to go trough multiple pages of documentation. Something that would have a simple landing page explaining what we do. And having a simple way to review the changes people make before publishing them would be also great.

I knew we wanted something better, but I didn’t know what exactly. I also didn’t want to invent yet another way to build docs. So I looked around, and found the Fedora Release Engineering documentation. It’s hosted in Pagure Docs, it’s built with Python Sphinx, and it also used to be a wiki. And I got inspired!

So I created some drafts, made a proposal, and convinced people from the Modularity Working Group that we need a cool website. And then I just built it. Using the same tech as the Fedora Release Engineering team – Python Sphinx to build, Pagure Docs to host.

But because I’m a wanna-be designer, I also created a logo, and wrote a custom Sphinx template including a simple landing page that helps people quickly understand what Modularity is about.

And I’m happy to announce that the new Fedora Modularity documentation website has been published today!

To edit the documentation, please send pull-requests against our source repository. And maybe get in touch if you like the project!

Early module development with Fedora Modularity

So you like the Fedora Modularity project – where we separate the lifecycle of software from the distribution – and you want to start with module development early? Maybe to have it ready for the modular Fedora 26 Server preview? Start developing your modulemd on your local system now, and have it ready for later when the Module Build Service is in production!

Defining your module

To have your module build, you need to start with writing a modulemd file which is a definition of your module including the components, API, and all the information necessary to build your module like specifying the build root and a build order for the packages. Let’s have a look at an example Vim module:

document: modulemd
version: 1
    summary: A ubiquitous text editor
    description: >
        Vim is a highly configurable text editor built to make creating and
        changing any kind of text very efficient.
            - MIT
        content: []
    xmd: ~
            generational-core: master
            generational-core: master
        community: http://www.vim.org/
        documentation: http://www.vim.org/docs.php
        tracker: https://github.com/vim/vim/issues
                - vim-enhanced
                - vim-minimal
                - vim-X11
            - vim-minimal
            - vim-enhanced
            - vim-X11
        rpms: ~
                rationale: The minimal variant of VIM, /usr/bin/vi.
                rationale: The enhanced variant of VIM.
                rationale: The GUI variant of VIM.
                rationale: Common files needed by all VIM variants.
                rationale: The directory structure used by VIM packages.

Notice that there is no information about the name or version of the module. That’s because the build system takes this information from the git repository, from which the module is build:

  • Git repository name == module name
  • Git repository branch == module stream
  • Commit timestamp == module version

You can also see my own FTP module for reference.

To build your own module, you need to create a Git repository with the modulemd file. The name of your repo and the file must match the name of your module:

$ mkdir my-module
$ touch my-module/my-module.yml

The core idea about modules is that they include all the dependencies in themselves. Well, except the base packages found in the Base Runtime API – which haven’t been defined yet. But don’t worry, you can use this list of binary packages in the meantime.

So the components list in your modulemd need to include all the dependencies except the ones mentioned above. You can get a list of recursive dependencies for any package by using repoquery:

$ repoquery --requires --recursive --resolve PACKAGE_NAME

When you have this ready, you can start with building your module.

Building your module

To build a modulemd, you need to have the Module Build Service installed on your system. There are two ways of achieving that:

  1. Installing the module-build-service package with all its dependencies.
  2. Using a pre-built docker image and a helper script.

Both options provide the same result, so choose whichever you like better.

Option 1: module-build-service package

On Fedora rawhide, just install it by:

$ sudo dnf install module-build-service

I have also created a Module Build Service copr repo for Fedora 24 and Fedora 25:

$ sudo dnf copr enable asamalik/mbs
$ sudo dnf install module-build-service

To build your modulemd, run a command similar to the following:

$ mbs-manager build_module_locally file:////path/to/my-module?#master

An output will be a yum/dnf repository in the /tmp directory.

Option 2: docker image and a helper script

With this option you don’t need to install all the dependencies on your system, but it requires you to setenforce 0 before the build. :-(

You only need to clone the asamalik/build-module repository on GitHub and use the helper script as follows:

$ build_module ./my-module ./results

An output will be a yum/dnf repository in the patch you have specified.

What’s next?

The next step would be installing your module on the Base Runtime and testing if it works. But as we are doing this pretty early, there is no Base Runtime at the moment I’m writing this. However, you can try your module in a container using a pre-build fake Base Runtime image.

To handcraft your modular container, please follow the Developing and Building Modules guide on our wiki which provides you all the necessary steps while showing you a way how modular containers might be built in the future infrastructure!

DevConf.cz 2017

Are you visiting DevConf.cz in Brno? There is a talk about Modularity and a workshop where you can try building your own module as well. Both can be found in the DevConf.cz 2017 Schedule.

  • Day 1 (Friday) at 12:30 – Fedora Modularity – How does it work?
  • Day 2 (Saturday) at 16:00 – Fedora Modularity – Build your own module!

See you there!


Copr in the Modularity World

Copr is a community build service in Fedora that enables anyone with a FAS account to build their own RPM repositories. This post explains how we can use Copr in modularity, what is already supported, and what needs to be done.

Almost everything in this post starts with a view from the UX perspective. However, technical details are also provided as they are the crucial part of this post.

I have also created a YouTube video showing building modules in Copr + installing.

What I want to achieve

Build systems in Fedora

There are two build systems in Fedora currently:

  1. Koji is the official Fedora build service, that builds everything officially released in Fedora.
  2. Copr is a community build service, available to anyone to build their own RPM repositories. Packages here are not reviewed. Copr in Fedora is something like AUR in Arch Linux.

The purpose of these two build systems should remain unchanged in the modular world. Fedora modules would be built in Koji, and community modules would be built in Copr.

Producing modules

A simplified examples of workflows to build modules in both build systems could be as follows:

  • Koji:
    1. By submitting a modulemd yaml. The whole module including its packages will be built automatically in the pipeline.
  • Copr:
    1. By submitting a modulemd yaml. The whole module including its packages will be built automatically in the pipeline.
    2. By “handcrafting” the module manually. User would submit the builds one by one – as they would in Copr today – and then build a module out of the project. This workflow is described in a more detail below.

Copr could be used to build the initial versions of modules that would eventually end up in Fedora. The “Handcrafting” feature will allow packagers to build and develop their module gradually, and submit it for review in Fedora later when it is ready.

Consuming modules

Official/production modules in Fedora would be identified by ‘name-branch-version’. To install a LAMP stack from Fedora, user would do something like:

$ module-thing install LAMP-something-42

Community modules from Copr would be identified by ‘username/name-branch-version’. To install a LAMP stack from Copr created by user asamalik, user would do something like:

$ module-thing install asamalik/LAMP-something-42

Important detail in both scenarios is using of the same tools to consume both official and community versions.

Technical – modules in Koji vs. modules in Copr

This table is a summary of the main differences. It is also described in a bit more detail below.

Koji Copr
Modules are represented by Tags Projects
Build roots are defined by RPMs with a specific tag External RPM repositories
Finished module builds are RPMs with a specific tag RPM repository
Multiple packages with the same NVR + dist-tag in the build system No Yes – every project has its own isolated namespace for packages.
Multiple modules with the same name in the build system No Yes – every user has its own isolated namespace for modules.


Koji uses tags to work with modules, where tag is basically a group of RPM packages. Each module has a ‘build tag’ with all RPMs defining the build root, and a ‘module tag’ with all the built RPMs that are included in the module as components.

A single RPM can be tagged with multiple tags. The problem is, that there can not be multiple RPMs in Koji with the same NVR and dist-tag. To workaround this problem, we modify the dist-tag of all RPMs in a module to include the name of the module.

Koji can/will be able to create an RPM repository out of every ‘module tag’ using a ‘signed repo’ function.


Copr would use projects to work with modules, where project is something like a directory. One project can define a build root by selecting a predefined mock chroot. This build root can be modified by linking external RPM repositories. To build modules, Copr would introduce an empty mock chroot, and the build root could be defined only by the RPM repository.

A single RPM can be included in only one project. To include one RPM in more projects, it needs to be rebuilt in them. However, there can be multiple packages with the same NVR and dist-tag in Copr. One in every project.

Copr also supports namespaces which is done by usernames. For example, two users can create modules with the same name, stream, release, metadata, … and include an RPM package with the same NVR + dist-tag. These will be completely separated.

Copr automatically creates a signed RPM repository out of every project. “Modularization” of the repository with a yaml spec file is already supported.

Building modules in Copr – conceptual view

This section describes steps that needs to be performed in Copr to build a module. The following steps can be done by a user or an automated tool like the orchestrator.

Copr currently supports everything except empty chroots and releasing a module.

Building a module

  1. Create a project with empty chroot
  2. Define build root by linking external repository
  3. Build all your RPM packages
  4. Release a module, two ways doing that:
    1. Define the module API, install profiles, and package filters by selecting binary packages from a list. [supported soon]
    2. Add the modulemd yaml to the repository. [already supported]

Building a stack

  1. Create a project with empty chroot – let’s call it the main project
  2. Create projects for all the “submodules” in your stack – let’s call them submodule projects
  3. Define build root by linking external repository to all projects you have created
  4. Build all your RPM packages of your “submodules” in their projects
  5. Use the “fork” function in Copr to copy the built RPMs from all your submodule project to the main project
  6. Delete all the submodule projects
  7. Release a module, two ways doing that:
    1. Define the module API, install profiles, and package filters by selecting binary packages from a list. [supported soon]
    2. Add the modulemd yaml to the repository. [already supported]

Handcrafting projects in Copr

This method of building modules is meant for the community members that use Copr today. The main benefits are:

  • Users will not need to write the yaml spec – Copr will automatically generate it for them.
  • User’s workflow of creating a module will be very similar to how they create RPM repositories in Copr today.

However, Copr will also support rebuilding modules from Fedora or building custom modules by submitting the yaml spec.


  1. User creates a project
  2. User defines build root by selecting a “base runtime module” from Fedora – roughly the same way as they would select chroots today
  3. User builds all their RPM packages
  4. User defines the module API, install profiles, and package filters by clicking checkboxes (see mockup below)
  5. User then click on “release module”. Copr will generate the yaml spec for them automatically and create a modular RPM repository.

The picture below shows the UI in Copr to create a module out of a project.

Producing traditional repo and module at the same time

Copr supports multiple chroots in one project. That means I can build the same thing for example for Fedora 23, Fedora 24, Epel 6, and Epel 7. We could add “module” as another chroot, so user could build the same thing for old and new systems. Traditional repositories could be consumed anytime – including installation and updates. Modules would be released every time project enters a release milestone. Modules would remain unchanged forever – the same way as in Fedora.

Roles of PDC and BPO in Modularity – please give me feedback

There has been some confusion about the roles of PDC and BPO in Modularity as the roles might seem overlapping. This document will briefly explain the roles of both services and highlight the main differences.

Long story short:

  • PDC is a database that stores module metadata and dependency graphs
  • BPO is a web UI to display module build state and probably to browse some metadata

Main overlaps:

  • BPO has its own database. However, the database acts as a cache only to make the UI fast. All the data can be lost and recovered from other services such as Orchestrator and PDC.
  • PDC has its own web UI. However, browsing and displaying modules has not been implemented. And even if it was, PDC does not/should not store data such as build state.

From my understanding, this design has been already valid on the Modularity Infra wiki page: https://fedoraproject.org/wiki/Modularity/Architecture/Infra?rd=Modularity/Infra

PDC – Product Definition Centre

PDC is the primary database for module metadata such as:

  • Name, stream, release – or however are these going to be called
  • Dependencies and components
  • Koji tags
  • Maybe information such as SLA?

It provides a REST API to access and manipulate this data.

A part of PDC is also a web UI that will not be used in Modularity.

BPO – Build Pipeline Overview

BPO is a single web UI that will watch over the whole pipeline. Its primary function is to show the build state of each module.

Users will be also able to browse module metadata such as:

  • Dependencies
  • Components
  • Install profiles

BPO uses its own database as a cache only to make the UI faster. The database:

  • Will be updated by fedmsg and also queries of other services as reactions to some messages.
  • Can be deleted and recovered from other services.