Minutes from 11/8/2007 Distribution Constructor Brainstorming Meeting

Attendees:

Karen, Jack, Jean, Sarah, Dave, Sanjay, Frank, Barbara (full day)

Eric, Erik, Channing, Alok, Ginnie, Stefan Teleman (morning)

Summary of Overall Distribution Constructor design

It's been observed that many of tasks performed by the Distribution constructor is very similar to the tasks that need to be performed by the Installer. So, we want to have a way to share common components.

In order to have sharable components between Distribution Constructor and other install technologies, some parts of the Distro Constructor will be implemented as modules and some other parts will be available as utilities. The Distro Constructor will call different modules as appropriate. The diagram below illustrates the overall architecture:

Note:

• Yellow indicates the step is optional

• circle indicates components are designed to be re-usable by other Install projects

Quick Notes about each component, and how it fits into the flow of Distribution Construction.

Component Selection Module

• Will have either GUI or CLI interface

• Should be able to take an existing manifest as a starting point for changes.

• Used for selection of components that needs to be included into the image. Example of choices are 32 bit or 64 bit or both, sparc or x86, what packages/features to include in the image, size of the resulting image, path to the proto area ...etc..

• The output of this module is a manifest file recording all the selections made in the module. Format of this manifest file is TBD. One format to consider is the Extended SMF format.

• Calling this module is optional in the Distribution Constructor, if the user already has a manifest that they want to use.

Target Instantiation Module

• Prepares the proto area for constructing the image. The proto area could be just a regular directory or a ZFS file system.

Transfer Module

• Based on the manifest, retrieve bits from the IPS repository and place the bits into the proto area.

Image Specific Processing

• Depending on the type of output image, special processing might be needed. These are modules that's supplied by the project teams that are responsible for the specific image type to be produced. They can pretty much manipulate the bits that's in the proto area.

• The Distribution Constructor will design an interface for these special processing modules to plug in.

Distribution Constructor Utilities

• Some utilities are provided by the Distribution constructor to do common things. For example, creating a bootable ISO image, creating a bootable USB image, etc...

• Depending on the image type, these utilities might not get used.

Outputs

• Given the same manifest, different image types can be produced by the Distribution Constructor. To produce different images from the same manifest, only the “Image Specific Processing” and the optional Distribution Constructor Utilities steps need to be repeated for the different images.

Summary of features to be supported in March 2008 release

The March 2008 release will provide very similar feature compared to the October 2007 Preview release. The major difference is the fact that these features will be implemented using the new Distro Constructor architecture as outlined above. Because of the new architecture, most of the code for the Oct 2007 preview most likely can't be reused.

What will be supported:

• The ability to specify what packages, or cluster will be included in the image. (also available in Oct 2007 preview)

• Your requirements will be specified in the manifest file (new for March 2008)

• Generate an image for X86 based Slim Live CD/USB (also available in Oct 2007)

• Interface to plug in special processing for different type of images. (new for March 2008)

What MIGHT be supported:

• provide some pre-defined useful manifests. (Depending on time. If we do decide to do this, will discuss what people think are “useful”)

• GUI to generate the manifest (most likely will not have time for this, but we will see)

Other topics discussed

Notes on some topics brought up during the meeting as well as topics posted on the Distro Constructor SketchPad wiki, but not covered in the other discussions above.

Fedora Revisor provides some pre-configured contents for images. That seemed to be very helpful. Are we going to do that with the Distro Constructor?

We will provide some pre-defined manifest for people to start with when constructing their own distro. We will also investigate how we can setup a space somewhere on the web for people to share their useful manifest so other users can also benefit.

Ubuntu Reconstructor seemed to have a lot of focus on making it easy for people to do custom branding. Are we going to support that in the Distro Constructor at some point.

The problem with supporting custom branding in the Distro constructor is the fact that many things that needs to be branded does not have public interfaces. If all the brandable components can be pulled out into one or more packages, this will be as simple as including the particular package(s) that have the branding information in the Distro Constructor. So, we don't think this is a problem the Distro Constructor should solve at this time.

Item 1 from Bart's notes about ensuring that the file system laid down during initial image construction is the one that gets installed on the target drive.

The suggestion Bart has in steps 1-6 is great. However, that greatly relies on using ZFS. ZFS works great on hard disks and most likely USB sticks. However, there has not been any experiment with using ZFS on CDs. Further investigation needs to be made to determine how well ZFS behaves on CD, which are read only and not random-access.

The topic of support multiple packaging systems.

We are only planning to support IPS.

Programming languages for the Distro Constructor

There's no decision as to what programming language(s) will be used for the Distro Constructor implementation. We will investigate different languages and pick one that suites Distro Constructor's purpose. Investigation will start with: Python, ksh93, C, Java.

Better approach to handling pre-configuration tasks such as Gnome's postrun. Using chroot seems too fragile.

One approach is to setup the proto area in a zone or a xVM. Then, boot up the zone, and have all the pre-configuration tasks can be executed in the isolated env.

Will Distro Constructor support different locales?

It makes sense to have this support in the Distro Constructor. If a user picks a set of locales, and indicates a default locale. The Distro Constructor can setup the default locale for the image.

How to make debugging LiveCD generated by Distro Constructor easier?

One idea is to create the image in a ZFS file system. That way, we can boot up the ZFS file system, and debug. This is quick and we have a “writable” disk in case there's any core dumps or debugging data that needs to be saved. To do this, we might need to modify some of the LiveCD SMF scripts that currently only works with LiveCD.

Topics that needs to be researched

Based on discussions in the meeting, the following items needs further research. Members of the Distro Constructor team should sign up to research one or more of the following. Inputs and help from others are of course welcomed. The following table lists topics, and what I think could be things to start the research with. In the Distribution Constructor meeting next Thurs, 11/15/2007, we will discuss some of the findings.