This document lays out a problem statement, requirements, and constraints according to the Open Decision Framework. The aim is to arrive at a transparent decision about the future of a git forge for the communities that represent the platforms that the Community Platform Engineering (CPE) team manages. Those communities are the CentOS and Fedora platforms and also include the Red Hat Enterprise Linux (RHEL) platform from a tooling and integration perspective. This document is the first in a series of documents capturing the conversation about the problems we face and driving the conversation to implement the decisions captured.
The problem statement for this ODF can be broken down into a number of disctinct problems. They are listed in no particular order or priority:
The Community Platform Engineering (CPE) team have a mission statement to support the infrastructure and services that build and deliver platform artifacts. The mission statement aims to focus the team’s work from overcommitting to work, running multiple projects in a disconnected manner and to ultimately provide focus and value for our Communities. The remit of the team needed to be defined to allow the team to both manage the workload and the expectations.
Using this definition, the current git forge, Pagure, does not align with what CPE can focus on from both a roadmap development perspective and the operational requirements that such a service demands long term. While Pagure was historically driven by the CPE team, developing a gitforge is outside of building and delivering platform artifacts. [See the later sections for more focus on this.] A self-hosted and self-developed git forge is not required to build and deliver platform artifacts.
While we can make exceptions to the mission statement, we first need to know why we should consider a specific exception. This helps justify the inclusion and subsequent prioritisation of work. We recognise that Pagure is deeply integrated into our daily workflows and is used extensively by the Community. This is the reason the CPE team has not re-examined this commitment, and is a primary driver to openly discuss the team’s and community’s requirements for a git forge.
The CPE team has been unable to commit a development team to Pagure for several months now. This situation is unlikely to change based on our current known priorities.
Historically, Pagure was maintained by individuals (including members of the Fedora community who aren’t on the CPE team) where spare cycles allowed. However, CPE’s mode of working is now that of feature teams, removing the siloed contributions in favour of building sustainable team practices. The feature roadmap for Pagure, however, cannot be executed solely by the CPE team, since the feature requests need to be weighed against the team’s other priorities.
Pagure represents one app out of our current ownership of 100+ code bases. Therefore progression of the roadmap is not guaranteed and certainly not at the pace seen previously. Similarly, bug fixes and enhancements are currently on a best-effort basis. The code is therefore frozen from a functionality perspective pending the outcome of this ODF.
Every line of code and application CPE supports as a team has a cost burden for maintenance and uptime. Pagure is highly-connected to numerous services that are critical to successfully running services that CPE and the community need and support. Therefore, the team must look at long term maintenance including bug fixes and server maintenance as requirements.
At the same time, integrations that already exist in Pagure may need to be created for another git forge, which is a cost as well. This also needs to be fully considered by the team as part of the requirement gathering.
Another major consideration to operationally own an application is its performance and scalability. A git forge may have key requirements for uptime, availability and responsiveness for end users. The current scalability profile of Pagure is unlikely to substantially change as it is resourced today – while the consumption profile of the user base and interconnected applications is likely to increase.
TThe original purpose of Pagure was to mirror the functionality of popular git forge solutions that were available at the time (when most were nascent). Pagure’s feature enhancements were driven by community needs and the team’s viewpoint of where a git forge should go. The team has not solicited requirements in a holistic way from its users and the community, and its internal customers have mainly consisted of the team itself.
However, we also recognize that the feature gap between Pagure and some other forges is substantial and growing. Without either a dedicated development team, or stealing resources from other priority initiatives, it will be almost impossible to close those gaps. Depending on the consumers’ requirements, we recognize this could put Pagure in an untenable situation versus other solutions.
This makes gathering a full set of requirements even more critical. If we fail to capture requirements completely, this discussion is very likely to happen again, only more urgently and with less time for the team to plan and react.
This conversation does not focus on whether Feature X exists in Forge Y or Forge Z. Instead it focuses on functional and non functional requirements for a git forge in general.
This conversation does not focus on how the CPE team invest their time and limited resources. That is not a factor in this discussion.
This document does not focus on the CPE Mission Statement or whether a git forge should fit that Mission Statement.
The decision will be made by the management of the CPE team with careful consideration of the requirements for both the Fedora and CentOS communities as well as the needs of the team. The CPE team is the group that manages the integration of services and tooling with a git forge solution on behalf of our communities, and will choose the most sustainable, functional, and scalable option to improve our workflows long term.
There exist three choices for such a solution. Github, Gitlab and Pagure. There are no other forges that we could find that had both the product maturity and standing in open source communities, therefore no other solutions are under consideration as the three choices here represent the main git forge solutions on the market. The team will use the requirements gathered to make an informed decision on which of the 3 to pursue.
Please see the section on Stakeholders below.
The goal is to outline what is needed from the day to day perspective of:
Requirements are welcome from members of the CPE team and the groups identified as being impacted by such a decision.
Examples of non-functional requirements include but are not limited to performance, scalability, availability, reliability, maintability, and capacity. The goal here is to include considerations of this nature from any group impacted by this decision.
Upon gathering the requirements of a git forge solution, the intention is to:
To be clear, the outcome here may be a decision to invest heavily in Pagure to meet the requirements or it may be to opt for another git forge to meet the requirements. No option is off the table.
While we apprecaited that all individual voices matter, for a more sane approach to gathering requirements we will identify key stakeholders to collate and present a singular view of their representation.
It is recommended that both Fedora Council and CentOS Board gather input and present their concerns in a manner that is consistent with how their communities work. The RHEL and CPE requirements will be gathered through Red Hat communication mechanisms and presented publicly via a HackMD file to ensure transparency in their source. This will be published and distributed in due course. Additionally, a live video call and associated IRC meetings will be held and advertised in advance to discuss the requirements, talk about concerns and address any questions. We want transparency to be at the heart of this decision.
On behalf of the Board, a group of us is working on an update to the CentOS Project goals that were originally laid out in 2014 and are online at centos.org/about. We’re hosting informal user and contributor interviews in a room throughout the day at the CentOS Dojo later this month in Brussels.
Please join us and share your open and honest experiences with CentOS the project, technologies, community, and so forth. We’d like to hear from you and, ultimately, see how your input can inform the goal-setting process and outcome. You are welcome to bring your questions about community, governance, project direction, other strategic thoughts, and so forth.
If you're interested in participating in this informal opinion-gathering, please come see Karsten or Rich at the Dojo, or at the CentOS table at FOSDEM.
/signed Karsten Wade on behalf of CentOS Board and other co-authors
On 2020-01-08 the CentOS Board of Directors held the first meeting of 2020, welcoming guest Rich Bowen, Community Architect for the CentOS Project.
The group talked through some background for each-other as part of the framework for updating the project goals. The Board is drafting a process that is for refreshing the project's goals openly and transparently. More details including a timeline should start rolling out in the middle of January to the centos-devel mailing list and announced on blog.centos.org.
The Board then heard from Jim Perrin as head of Community Platform Engineering (CPE) about the ongoing work around the EL 8 rebuild, SIG needs, and what the path forward might look like. He covered how the teams have been making realtime changes to build systems due to the differences in how CentOS Linux 8 and CentOS Stream need to be built. Regarding some open requests, discussions on the build root are coming soon, as the team raises their heads from work that has been underway. In general, Jim reported that tooling around the build systems are improving by force. He identified Aofie Maloney as a key contact to work with Rich Bowen on highlighting the ongoing work from CPE that affects various CentOS Project constituencies. We’re all hoping this communication helps raise visibility and focus questions about work to keep everyone better informed.
As a participant in the discussion, Rich Bowen agreed to write up a post for the community that covers the current situation of CentOS Linux 8 builds. This has been subsequently been posted:
In support of these efforts, the Board came to the following decisions, resolutions, and agreements:
Present at the meeting:
We wanted to update you on what is happening, largely out of sight to most of the community, on the CentOS Linux 8 front. We have appreciated the patience of the community, but we understand that your patience won’t last forever.
A lot of the work in rebuilding RHEL sources into CentOS Linux is handled by automation scripts. Due to the changes between RHEL 7 and RHEL 8, many of these scripts no longer work, and had to be fixed to reflect the new layout of the buildroot. This work has been largely completed, allowing us to pull the source from Red Hat without a lot of manual work. This, in turn, should make the process of rebuilding RHEL 8.2 go much more smoothly than RHEL 8.0 and 8.1 have done.
Once 8.1 has been released, work will begin on bringing this new codebase, along with CentOS Stream, in to CBS (https://wiki.centos.org/HowTos/CommunityBuildSystem) so that SIGs can build packages for CentOS Linux 8 and CentOS Stream.
We will discuss this, and give updates of progress, on the centos-devel mailing list over the coming weeks. Some of you have observed that the CentOS team tends to prioritize doing the work over talking about it. While that’s not all bad, it does tend to leave most of you in the dark as to what it is that is being worked on, and we’re committed to greater transparency going forward.
Once again, we appreciate your patience as we work through the growing pains of the 8 branch. We hope to share a more detailed (projected) timeline in the days to come, with the caveat that timelines always change as they are being worked.
To build and distribute the Origin 3.x rpm packages to CentOS repository.
Happy New Year to all CentOS community!
As of 2020, the CentOS PaaS SIG wants to make a step towards a new endeavor to help and provide OKD 4.x as part of a wider community. However, for the time being we would like to announce that CentOS PaaS SIG charter will only be to mantain the existing Origin 3.x rpm packages published in the CentOS mirrors while we transition to the new OKD Working group (where all the development is taking place as we speak) which will ship the next version of OpenShift community.
As many of you already know, the OpenShift 4.x brought in a lot of innovation and changes in terms of the architecture, deployment and packaging compared with OpenShift 3.x and with that there been some changes with regards to the development relation between OCP/ OKD 4 which was very well covered in .
And last but not least, we would like to address the 1 mil $ question: Will there be an OKD 4.x based on CentOS as base OS ?
This topic was very much discussed in the OpenShift Working Group kick off meeting as well as the OpenShift dev mailer and the conclusion (captured in the roadmap  ) was:
The initial deliverable of OKD 4.x will be based on Fedora CoreOS as base OS since is the only distribution close to Red Hat CoreOS (rpm-ostree based system driven by Ignition) however should there be any community formed/ willing to develop/ create a CentOS rpm-ostree based system driven by Ignition, the OpenShift Working Group would welcome them (please join the meeting to discuss and maybe create a sub project as mentioned in the cahrter  )
Note, the CentOS PaaS SIG doesn't have the expertise in building / creating a new CentOS distribution nor the knowledge of any other initiative in the CentOS community.
We would like to send kudos to all our members who helped us with the SIG activities:
To find out more information about the OKD Working Group, please visit  where you can find out the charter  as well as the approved roadmap for OKD 4.x . Please do get involved  and if you find issues please open them in  (Bugzilla locations coming soon). You can also contact us on Slack in the #origin-users channel on openshiftcommons.slack.com and #openshift-dev on kubernetes.slack.com.
Dear CentOS enthusiast,
For those of you who celebrate various things at this time of year, we wish you a wonderful time with family and friends.
December, as usual, was very slow around here, with many people taking some time off around the end of the year. As such, I don't have much news to report this time.
Red Hat engineering continues to work towards on the tooling necessary to have an active CentOS Stream, and we hope to have an announcement about that this time next month.
Continuing the push for greater transparency and community participation, the Board of Directors has published the minutes from the December board meeting.
We issued the following CEEA (CentOS Errata and Enhancements Advisories) during December:
We issued the following CESA (CentOS Errata and Security Advisories) during December:
We issued the following CEBA (CentOS Errata and Bugfix Advisories) during December:
The following releases also happened during December:
December was very quiet, as it is in most years. If you represented CentOS at an event in December, please do let us know!
We have published a number of interviews from the Student Cluster Competition at the recent SuperComputing event in Denver:
In just under a month, we will, as usual, have a table at the annual FOSDEM conference in Brussels, Belgium. This will be held on the first weekend in February, which is the 1st and 2nd of February, 2020. We'll be sharing the space with our friends from Fedora. Please drop by and see us.
And, on the day before FOSDEM starts, we'll be having our annual Dojo at the Marriott Grand Place. That's Friday, January 31st, 2020. The agenda is on the event listing page, and we would love to have you there.
We'll be having a lightning talks section this year, so if you have something you'd like to present about, but don't have enough for a full presentation, bring your notes and your ideas! Tell us about your favorite projects, your interesting discoveries, or your perplexing problem.
Attendance is free, but we would appreciate it if you register, so that we know how many people to plan for. We have limited space, so register soon before we are all full.
See you in Brussels!
If your University, company, or research organization, wants to host a CentOS Dojo, we would love to hear from you. You'll need a space where 100-200 people can attend technical talks, and someone who is able to work with us on logistics and talk acquisition. We'll help promote the event, and work with you to craft the schedule of talks. Drop us a note on the CentOS-Promo mailing list - https://lists.centos.org/mailman/listinfo/centos-promo - with your proposal.
The SIGs - special interest groups - are where most of the interesting stuff in CentOS happens. They are communities packaging and testing layered projects on top of CentOS, and ensuring that they work reliably.
The PaaS SIG has provided their report as a separate blog post, and the Virtualization and Software Collections SIG reports are provided below.
Packaging and maintaining different FOSS based virtualization
applications that one can install and run natively on CentOS.
We are always looking for new members.
omachace__ joining Virt SIG for oVirt project volunteering for providing
ansible-runner related and mod_wsgi into Virt SIG
Welcoming Miguel Barroso mbarroso to Virt SIG for oVirt
* upstream released oVirt 4.3.7
* Working on getting oVirt CentOS Stream packages, particularly oVirt 4.4
* Xen 4.12.1 available on CentOS 7
* Regular updates to 4.8, 4.10, 4.12 for security updates
* Upstream Xen 4.13 nearing release
The Virtualization SIG remains fairly healthy. All the projects within
the SIG are updating regularly on biweekly meetings.
oVirt had a conference in Rome on 4 Oct.
oVirt also now has a new driver installer for Windows. If you have a VM
with the old drivers, it is recommended to uninstall them before
installing new ones.
The Xen Developer Summit was held 9-11 July in Chicago.
After an online discussion / survey, it was decided that the "primary
supported" version of Xen would always be the most recent version of
Xen-1. The current "primary" version is 4.8; once Xen 4.13 comes out
upstream (probably next week) we'll move this to 4.12.1. After that,
when 4.14 comes out, we'll update to the latest version of 4.13, and so on.
Both Xen and oVirt waiting for CentOS 8 support in the CBS. oVirt using
copr as a work-around for now.
The Software Collections SIG will provide an upstream development area for various software collections and related tools. Developers can build on and extend existing SCLs, so they don't need to re-invent the wheel or take responsibility for packaging unnecessary dependencies.
The successfully rebuilt collections are in process of being tested and released, and should be available on public mirrors shortly after this report is published.
As with any open source project, there's a lot more than just code. If you want to get involved, but you're not a programmer or packager, there's still a ton of places where you can plug in.
We look forward to hearing from you, and helping you figure out where you can fit in.
On Wednesday 08 January 2020, the CentOS Board of Directors will hold its first meeting of the decade and 2020 calendar year. Below is the agenda for that meeting that can be shared with the community and wider public.
On 2019-12-18 the CentOS Board of Directors held the final meeting of the 2019 calendar year.
The meeting was focused primarily on how the Board can lead the project further into being a contributor-centric open source project while continuing to deliver value to our community of users. Of particular interest is growing participation in CentOS Streams in addition to ongoing efforts around CentOS SIGs.
As a centralizing effort, the Board agreed to revisiting the goals document created five years ago, and to undergo an effort to refresh those goals in the light of the project’s evolution. The Board will be inviting various stakeholders into these discussions as we undergo a public revision of the goals at the start of 2020.
In support of these efforts, the Board came to the following decisions, resolutions, and agreements:
On Wednesday 18 December 2019, the CentOS Board of Directors will hold it's last meeting of the 2019 calendar year. Below is the agenda for that meeting that can be shared with the community and wider public.
At the recent SuperComputing event in Denver, I spoke with several of the teams at the Student Cluster Competition. One of them was the team from the Shanghai Jiao Tong University. You can listen to the full interview on YouTube at https://youtu.be/HpJRyF5S_4U
Rich: I'm with the team from Shanghai Jiao Tong University. They have just finished participating in the Student Cluster Competition. I wonder if you can tell me about your experience.
Shangai Jiao Tong: We think the competition was quite challenging for us. We're a first-time participant in the SC competition. We think we learned a lot about the competition, as well as other teams - we made a lot of friends. It was a pretty good experience.
R: How do you feel you did?
SJT: We think we did fine within our capabilities. Maybe not state of the art but pretty good, for us.
R: If somebody from another university was interested in participating, what advice would you give them?
SJT: Read the rules carefully before you participate, because we missed some of the points, and that cost us something. But, it's still fine. Just have fun.
R: I was wondering why you chose CentOS for your base operating
SJT: Well, because it's well tested, stable, and performance is good. Mainly because it's well tested. Because we all use that in our test clusters back home.
R: Thank you so much for your time, and good luck.