LWN.net Weekly Edition for May 16, 2024 (2024)

Welcome to the LWN.net Weekly Edition for May 16, 2024

This edition contains the following feature content:

  • Portable LLMs with llamafile: A new tool that generates binaries with weights from large language models in order to locally run a chatbot.
  • Another push for sched_ext: The extensible scheduler class is a way to change the kernel's scheduler using BPF but it has run into some headwinds.
  • Some 6.9 development statistics: The usual look at developers and companies involved in creating the 6.9kernel, plus a new subscribers-only feature.
  • The start of coverage from the Linux Storage, Filesystem, Memory Management, and BPF Summit (LSFMM+BPF):
    • The state of the page in 2024: An update on the process of converting the kernel to use folios.
    • Lots more to come ...
  • Debian dismisses AI-contributions policy: Unlike Gentoo, the Debian project does not seem to be interested in an outright ban on AI-generated contributions.
  • Managing expectations with a contributions and credit policy: A new guide to developing a policy that outlines the expectations that a project has with regard to contributions and the credit those contributions will receive.

This week's edition also includes these inner pages:

  • Brief items: Brief news items from throughout the community.
  • Announcements: Newsletters, conferences, security updates, patches, and more.

Please enjoy this week's edition, and, as always, thank you for supporting LWN.net.

Comments (none posted)

Portable LLMs with llamafile

By Daroc Alden
May 14, 2024

Large language models (LLMs) have been the subject of much discussion andscrutiny recently. Of particular interest to open-source enthusiasts are theproblems with running LLMs on one's own hardware — especially when doing sorequires NVIDIA's proprietary CUDA toolkit, which remains unavailable in manyenvironments.Mozilla has developedllamafile as apotential solution to these problems. Llamafile can compile LLM weightsinto portable, native executables for easy integration, archival, ordistribution. These executables can take advantage of supported GPUs whenpresent, but do not require them.

Portability

Llamafile is based on the MIT-licensedllama.cpp library, aC/C++ implementation of the code necessary to evaluate LLMs of several differentarchitectures. Llamafile maintains its own copy of llama.cpp, with someadditional MIT-licensed changes. The code for llamafile itself is made availableunder the Apache 2.0 license. Llamafile's value comes from buildingllama.cpp in a way that works seamlessly across many different environments.On April 25, Mozilla postedan update about llamafile's progress that called it "the easiestand fastest way to run a wide range of open large language models".

The lead developer of llamafile, Justine Tunney, chose to re-use herprevious work onCosmopolitan Libc. That project implements a C standard library andassociated compiler infrastructure to compile programs written in C as multi-formatexecutables that can run on Linux, macOS, Windows, several BSDs, or without anoperating system.

In the blog post announcing the startof Cosmopolitan Libc, Tunney said:

I like the idea of having the freedom towrite software without restrictions that transcends traditional boundaries. Mygoal has been helping C become a build-once run-anywhere language, suitable forgreenfield development, while avoiding any assumptions that would preventsoftware from being shared between tech communities.

In service of that goal, Cosmopolitan Libc uses clevertechniques to create executable files that can be simultaneously interpreted asseveral different formats. The executables it produces start with a shellscript that behaves differently on different operating systems, which theprogram exploits to select a pre-compiled static binary suitable to theoperating system and architecture at run time.These files (along with the weights of the LLM, in llamafile's case),are bundled into a zip file. Since zip files store their metadata at the end ofthe file, the same file can serve as shell script,executable, and zip archive.

This approach is of questionable use for most programs, but for Mozilla, itrepresents an important way to democratize access to LLMs. There are anincreasing number of repositories, most prominentlyHuggingFace, that distributeraw LLM weights — but raw weights aren't actually enough to use the models. Usersalso rely on inference code and software that providesan API to actually access the resultsof the model. To make things worse, that inference code is often specific to aparticular brand of GPU or machine-learning toolchain, which makes LLMs hard to runexcept in specific environments.

Llamafile applies Cosmopolitan Libc's philosophy of selecting anappropriate implementation at run time to machine learning. It automaticallydetects whether the user has AMD or NVIDIA's GPU toolchains available, and ifso, uses those. If not, it uses a new open-source linear-algebra library calledtinyBLAS. The library supports using the APIs madeavailable by existing graphics drivers to take advantage of GPU accelerationwithout requiring an installed toolchain. This is less performant than lettingNVIDIA's CUDA or AMD's ROCm compile a native programfor a specific model of graphics card, but stilluseful for users who don't have the GPU SDKs but do have the hardware.

TinyBLAS doesn't work with all drivers, however. If no GPU acceleration isavailable, llamafile falls back to CPU implementations of the core linearalgebra libraries — versions that are specialized for particular microarchitectures andspecific hardware.On March 31, Tunney published a detailedblog post discussing how she improved CPU inference performance across awide variety of hardware, often by hand-writing a matrix multiplicationkerneltuned for that exact hardware.

There's another trick that llamafile uses to speed up matrixmultiplication, though, which is much more specific to its purpose as a platformfor running LLMs. Generic linear algebra libraries likeBLAS need tobe able to multiply arbitrary matrices with unknown dimensions, possiblytransposed or weighted in some way. LLM inference, because it proceeds one tokenat a time, spends a lot of time doing matrix-vector multiplications that canbe written in a simpler form.

Even when LLMs do generalized matrix multiplications (duringinitialization), the models are architected such that the matrices are usuallyof a known size — often a multiple of 64. This lets a hand-unrolledimplementation specific to those sizes outperform a more generic algorithm.Tunney benchmarked the multiplication of a 513x512 matrix with a 512x512one (a size llamafile uses frequently),finding that her code outperformed Intel's proprietaryMath Kernel Library (MKL) — on that specific size. TheMKL is still faster on other sizes.Since llamafile controls the size of the batches used during LLMinitialization, however, that's still a clear performance improvement.

Using llamafile

Using an LLM packaged by llamafile is fairly straightforward.The project's READMElinks to several examples of different sizes.Downloading a file, marking it as executable, and running it is all thatshould be required in the vast majority of cases. Users who havebinfmt_misc registrations for WINE might need to add a more specificrule to prevent WINE from being used as the program's interpreter.Running the program with no arguments will open llama.cpp's simple chat interface:

The built-in web server also offers an OpenAI-compatible API, so tools thatexpect to talk to the proprietary service can be seamlessly re-directed, as cantools that use OpenAI's API design as a de-facto standard for LLM inference.Users who are more comfortable on the command line can pass parameters andinstructions as arguments instead.

Parameters can also be baked into a llamafile executable.As mentioned above, the files areactually valid zip files; adding a file named .args to the executablewill make it treat those arguments as additional command line parameters.The procedure for turning the llamafile binary produced by building theproject into a LLM-specific llamafilefor distribution is actually the same: addthe weights and any required arguments to the zip file.

For performance reasons, however, it's important to add the weights withoutcompression, and ideally aligned to a 4K boundary. This allowsllamafile to map the weights directly into memory, which is substantially fasterthan decompressing them into non-disk-backed memory.For this purpose, the project also provides a utility called zipalignthat adds files to a zip archive in the correct way.

On my laptop, which lacks any relevant GPUs but does have a spiffy 12thgeneration Intel i7 processor, theMeta-Llama-3-8B-Instruct.Q5_K_M.llamafile download provided as anexample evaluates provided prompts at a rate of about 16 tokens per second. Theactual answer itself is evaluated at about 3.5 tokens per second. The differenceis attributable to the factthat during prompt evaluation, the model can use matrix-matrixmultiplications, instead of matrix-vector multiplications. But this level ofperformance — while perhaps too slow to process many large documents — seems entirelyadequate for local use.

With LLMs becoming increasingly integrated into other software,efforts to make them easy to run onexisting, consumer hardware are an important part of making sure users canbenefit from the technology without sending their data to third parties.The ultimate test of whether a format is suitable for widespread use is whetherit is actually adopted; with llamafile being only a few months old,it's too soon to say for sure whether the project has achieved its goals. Itdoes, however, seem to be well on the way.

Comments (18 posted)

Another push for sched_ext

By Jonathan Corbet
May 9, 2024

The extensible scheduler class (or "sched_ext") is a comprehensiveframework that enables the implementation of CPU schedulers as a set of BPFprograms that can be loaded at run time. Despite having attracted a fairamount of interest from the development community, sched_ext has run intoconsiderable opposition and seems far from acceptance into the mainline.The posting by Tejun Heo of a newversion of the sched_ext series at the beginning of May has restartedthis long-running discussion, but it is not clear what the end result willbe.

As a quick refresher: sched_ext allows the creation of BPF programs thathandle almost every aspect of the scheduling problem; these programs can beloaded (and unloaded) at run time. Sched_ext is designed to safely fallback to the completely fair scheduler should something go wrong (if aprocess fails to be run within a time limit, for example). It has beenused to create a number of special-purpose schedulers, often withimpressive performance benefits for the intended workload. See this 2023 article for a more detailed overviewof this work.

Heo lists a number of changes that have been made to sched_ext since theprevious version was posted in November. For the most part, theseappear to be adjustments to the BPF API to make the writing of schedulerseasier. There is also a new shutdown mechanism that, among other things,disables the BPF scheduler during power-management events like systemsuspend. There is now support for CPU-frequency scaling, and somedebugging interfaces have been added to make developing schedulers easier.The core design of sched_ext appears to have stabilized, though.

Increasing interest

Even before getting to the changes, though, Heo called attention to theincreasing interest in sched_ext that is being shown across the communityand beyond. Valve is planning to use sched_ext for better game schedulingon the Steam Deck. Ubuntu isconsidering shipping it in the 24.10 release. Meta and Google areincreasing their use of it in their production fleets. There is alsoevidently interest in using it in ChromeOS, and Occulus is looking at it aswell. Heo concludes that section with:

Given that there already is substantial adoption which continues togrow and sched_ext doesn't affect the built-in schedulers or therest of kernel in an invasive manner, I believe it's reasonable toconsider sched_ext for inclusion.

Whether that inclusion will happen remains an open question, though. Theposting of version4of the patch set in July 2023 led to a slow-burning discussion on themerits of this development. Scheduler maintainer Peter Zijlstra rejectedthe patches outright, saying:

There is not a single doubt in my mind that if I were to mergethis, there will be Enterprise software out there that will mandateits own BPF sched thing, or else it won't work.

They will not care, they will not contribute, they might even pulla RedHat and only share the code to customers.

He added that he saw no value in merging the code, and dropped out of theconversation. Mel Gorman also expressedhis opposition to merging sched_ext, echoing Zijlstra's concern thatenterprise software would start requiring the use of special-purposeschedulers. He later addedthat, in his opinion (one shared with Zijlstra), sched_ext would workactively against the improvement of the current scheduler:

I generally worry that certain things may not have existed in theshipped scheduler if plugging was an option including EAS,throttling control, schedutil integration, big.Little, adapting tochiplets and picking preferred SMT siblings for turbo boost. Ineach case, integrating support was time consuming painful and apluggable scheduler would have been a relatively easy out thatwould ultimately cost us if it was never properly integrated.

Heo, naturally, disagreedwith a lot of the concerns that had been raised. There are, he said,scheduling problems that cannot be addressed with tweaks to the currentscheduler, especially in "hyperscaling" environments like Meta. Hedisagreed that sched_ext would impose a maintenance burden, arguing thatthe intrusion of BPF into other parts of the kernel has not had thatresult. Making it possible for users to do something new is beneficial,even if there will inevitably be "stupid cases" resulting from howsome choose to use the new feature. In summary, he said, opponents arefocused on the potential (and, in his opinion, overstated) costs ofsched_ext without taking into account the benefits it would bring.

Restarting the conversation

That message, in October, was the end of the conversation at the time. Heois clearly hoping for a better result this time around, but Zijlstra's responsewas not encouraging:

I fundamentally believe the approach to be detrimental to the scheduler eco-system. Witness the metric ton of toy schedulers written for it, that's all effort not put into improving the existing code.

He said that he would not accept any part of this patch series until"the cgroup situation" has been resolved. That "situation" is aperformance problem that affects certain workloads when a number of controlgroups are in use. Rik van Riel had put together a patchseries to address this problem in 2019, but it never reached the pointof being merged; Zijlstra seems to be insisting that this work be completedbefore sched_ext can be considered, and he gave little encouragement thatit would be more favorably considered even afterward.

Heo expresseda willingness (albeit reluctantly) to work on the control-group problemif it would clear the way for sched_ext. He strongly disagreed withZijlstra's characterization of sched_ext schedulers as "toy schedulers" andthe claim that working on sched_ext will take effort away from the mainlinescheduler, though. There is, he said, no perfect CPU scheduler, so themainline scheduler has to settle for being good enough for all users. Thatmakes it almost impossible to experiment with "radical ideas", andseverely limits the pool of people who can work on the scheduler. Much ofthe energy that goes into sched_ext schedulers, he said, is otherwiseunavailable for scheduler development at all.

There is, he said, value in some of those radical ideas:

Yet, the many different ways that even simple schedulers candemonstrates sometimes significant behavior and performancebenefits for specific workloads suggest that there are a lot of lowhanging fruits in the area. Low hanging fruits that we can'teasily reach from our current local optimum. A singleimplementation which has to satisfy all users all the time isunlikely to be an effective vehicle for mapping out such landscape.

Igalia developer Changwoo Min, who is working with Valve on gaming-orientedscheduling, supportedHeo's argument, saying that: "The successfulimplementation of sched_ext enriches the scheduler community withfresh insights, ideas, and code".That, as of this writing, is where this conversation stands.

What next?

Sched_ext is on the schedule for the BPF track of the Linux Storage,Filesystem, Memory-Management, and BPF Summit, which begins onMay13. That discussion will cover the future development ofsched_ext but, most likely, will not be able to address the question ofwhether this work should be merged at all. That discussion couldcontinue, on the mailing lists and elsewhere, for some time yet.

Sometimes, when a significant kernel development stalls in this way,distributors that see value in it will ship the patches anyway, as Ubuntu,Valve, and ChromeOS are considering doing. While shipping out-of-tree codeis often discouraged, it can also serve to demonstrate interest in afeature and flush out any early problems that result from its inclusion.If things go well, this practice can strengthen the argument for mergingthe code into the mainline, albeit with the ever-present possibility ofchanges that create pain for the early adopters.

Whether that will be the path taken for sched_ext remains to be seen. Whatis certain is that this work has attracted a lot of interest and isunlikely to go away anytime soon. Sched_ext has the potential to enable anew level of creativity in scheduler development, even if it remains out ofthe mainline — but that potential will be stronger if it does end up beingmerged. Significant scheduler patches are not merged quickly even whenthey are uncontroversial; this one will be slower than most if it isaccepted at all.

Comments (40 posted)

Some 6.9 development statistics

By Jonathan Corbet
May 13, 2024

The 6.9 kernel was releasedon May12 after a typical nine-week development cycle. Once again,this is a major release containing a lot of changes and new features. Ourmerge-window summaries (part1, part2) covered those changes; now thatthe development cycle is complete, the time has come to look at where allthat work came from — and to introduce a new and experimental LWN featurefor readers interested in this kind of information.

A total of 2,028 developers contributed to the 6.9 kernel; 285 ofthem made their first kernel contribution during this cycle. The mostactive contributors to 6.9 were:

Most active 6.9 developers
By changesets
Uwe Kleine-König 3442.4%
Kent Overstreet 2591.8%
Christoph Hellwig 2061.4%
Krzysztof Kozlowski 2011.4%
Johannes Berg 1751.2%
Ricardo B. Marliere 1721.2%
Eric Dumazet 1611.1%
Andy Shevchenko 1270.9%
Dmitry Baryshkov 1230.8%
Thomas Gleixner 1160.8%
Andrew Davis 1080.7%
Jiri Slaby 1000.7%
Jani Nikula 990.7%
Sean Christopherson 970.7%
Darrick J. Wong 970.7%
Randy Dunlap 960.7%
Ard Biesheuvel 930.6%
Masahiro Yamada 880.6%
Takashi Iwai 810.6%
Matthew Wilcox 800.6%
By changed lines
Hamza Mahfooz 721449.1%
Hawking Zhang 669978.5%
Matthew Sakai 587137.4%
Matthew Wilcox 311923.9%
Ian Rogers 184562.3%
Darrick J. Wong 123561.6%
Neil Armstrong 97071.2%
Dmitry Baryshkov 83001.0%
Kent Overstreet 80871.0%
Johannes Berg 77791.0%
Ping-Ke Shih 68890.9%
Mike Snitzer 65470.8%
Rob Clark 56540.7%
Christoph Hellwig 55890.7%
Geert Uytterhoeven 55350.7%
Shinas Rasheed 53100.7%
Krzysztof Kozlowski 52180.7%
Rajendra Nayak 52110.7%
Stefan Herdler 50170.6%
Yazen Ghannam 49950.6%

Following what has become a longstanding tradition, Uwe Kleine-König wasthe biggest contributor of changesets this time around. This work, whichis mostly focused on low-level device-driver refactoring, has brought about2,500 changesets into the kernel since 6.3 was released in April, 2023.Kent Overstreet continued the work of completing and stabilizing thebcachefs filesystem. Christoph Hellwig kept on with his extensiverefactoring work in the block layer and XFS filesystem. KrzysztofKozlowski worked extensively with drivers and devicetrees for mobilesystems, and Johannes Berg did a lot of work within the kernel's WiFisubsystem.

In the "lines changed" column, Hamza Mahfooz and Hawking Zhang kept upanother apparent tradition: adding huge files with lots of amdgpu registerdefinitions. Matthew Sakai, instead, added the new dm-vdodevice-mapper target. Matthew Wilcox removed the old NTFS filesystemimplementation, and Ian Rogers added event definitions for Intel CPUs.

The top testers and reviewers this time around were:

Test and review credits in 6.9
Tested-by
Daniel Wheeler 1197.5%
Michael Kelley 784.9%
Sohil Mehta 714.5%
Helge Deller 472.9%
Philipp Hortmann 342.1%
Shan Kang 342.1%
Pucha Himasekhar Reddy 322.0%
Dapeng Mi 311.9%
Carl Worth 241.5%
Babu Moger 231.4%
Dietmar Eggemann 231.4%
Shaopeng Tan 231.4%
Peter Newman 231.4%
Geert Uytterhoeven 221.4%
Randy Dunlap 221.4%
Guenter Roeck 211.3%
Nicolin Chen 211.3%
Juergen Gross 201.3%
K Prateek Nayak 201.3%
Zhang Rui 191.2%
Reviewed-by
Simon Horman 2002.2%
Christoph Hellwig 1711.9%
Krzysztof Kozlowski 1611.8%
Konrad Dybcio 1431.6%
AngeloGioacchino Del Regno 1291.4%
Andy Shevchenko 1151.3%
Ilpo Järvinen 1121.2%
Andrew Lunn 1121.2%
Darrick J. Wong 981.1%
Dmitry Baryshkov 981.1%
Kees Cook 951.0%
Linus Walleij 921.0%
Geert Uytterhoeven 891.0%
Neil Armstrong 881.0%
Jiri Pirko 881.0%
Rob Herring 871.0%
Greg Kroah-Hartman 850.9%
Gregory Greenman 780.9%
Hawking Zhang 770.8%
David Sterba 690.8%

The top testers continue, by all appearances, to be people who do that workas a primary job focus. On the review side, there are19developers who reviewed at least one patch every day duringthis development cycle, and five of those reviewed more than two each day.

There are 227 companies that were identified as having supported work onthe 6.9 kernel, the highest number (by a small margin) since 6.4 wasreleased. The most active employers were:

Most active 6.9 employers
By changesets
Intel186712.9%
(Unknown)10727.4%
Google10317.1%
(None)9796.8%
Linaro9246.4%
AMD8205.7%
Red Hat8075.6%
SUSE4683.2%
Meta4132.9%
Pengutronix3722.6%
Huawei Technologies3452.4%
Oracle3132.2%
Qualcomm3112.1%
IBM3012.1%
(Consultant)2872.0%
Renesas Electronics2471.7%
NVIDIA2411.7%
Texas Instruments2101.5%
Arm1761.2%
Microsoft1591.1%
By lines changed
AMD17187721.7%
Red Hat9144811.5%
Intel708008.9%
Google511046.5%
Oracle479066.0%
(Unknown)443005.6%
Linaro414925.2%
(None)283883.6%
Qualcomm178122.2%
Meta173882.2%
Renesas Electronics170512.2%
Realtek138621.7%
SUSE119531.5%
NVIDIA101621.3%
Huawei Technologies91001.1%
(Consultant)71400.9%
IBM67770.9%
Collabora67600.9%
Arm67120.8%
Marvell65870.8%

As usual, there are not a lot of surprises here; these results do notchange greatly from one release to the next — or even from one year to thenext.

One last note

It has been over 17 years since Who wrote2.6.20? was published here. Back in 2007, it was still widely saidthat the kernel was mostly developed and maintained by volunteers; bytaking the time to map commits in the kernel repository to employers, weshowed that the reality was rather different, and that most kerneldevelopers were paid for their work.

After all these years, it sometimes seems that these articles contain aboutas much news as a tide table. The information found there might be useful,but it is not generally surprising. There is still interest in thesearticles, though, as wefound out when we skipped a few development cycles some years back. Giventhe ongoing interest and the generally mechanical nature of putting thisinformation together, it perhaps makes sense to delegate more of the workto a machine.

Thus, we are happy to launch the LWN Kernel SourceDatabase as an experimental, subscriber-only feature. Much of theinformation found in these articles is available there, along with quite abit more. We encourage readers to play with the system and to let us knowwhat they think. To be clear: there is no plan to stop publishing thesearticles anytime soon, but now there is a resource for readers who wouldlike to dig deeper.

Comments (11 posted)

The state of the page in 2024

By Jonathan Corbet
May 15, 2024

LSFMM+BPF

The advent of the folio structure todescribe groups of pages has been one of the most fundamentaltransformations within the kernel in recent years. Since the foliotransition affects many subsystems, it is fitting that the subject wascovered at the beginning of the 2024 Linux Storage,Filesystem, Memory Management, and BPF Summit in a joint session of thestorage, filesystem, and memory-management tracks. Matthew Wilcox used thesession to review the work that has been done in this area and to discusswhat comes next.

The first step of this transition, he began, was moving much of theinformation traditionally stored in the kernel's page structureinto folios instead, then converting users of struct page to usethe new structure. Theinitial goal was to provide a type-safe representation for compound pages, but the scope has expandedgreatly since then. That has led to a bit of ambiguity: what, exactly, isa folio in current kernels? For now, a folio is still defined as "thefirst page of a compound page".

By the end of the next phase, the plan is for struct pageto shrink down to a single, eight-byte memory descriptor, the bottom fewbits of which describe what type of page is being described. Thedescriptor itself will be specific to the page type; slab pages will havedifferent descriptors than anonymous folios or pages full of page-tableentries, for example.

Among other motivations, a key objective behind the move to descriptors isreducing the size of the memory map — the large array of pagestructures describing every physical page in the system. Currently, thememory-map overhead is, at 1.6% of the memory it describes, too high. Onsystems where virtualization is used, the memory map is also found inguests, doubling the memory consumed by the memory-map. By moving todescriptors, that overhead can be reduced to 0.2% of memory, which can savemultiple gigabytes of memory on larger systems.

Getting there, though, requires moving more information into thefolio structure. Along the way, concepts like the pin count for apage can be clarified, cleaning up some longstanding problems in thememory-management subsystem. This move will, naturally, increase the sizeof the folio structure, to a point where it will be larger thanstruct page. The advantage, though, is that only onefolio structure is needed for all of the base pages that make upthe folio. For two-page folios, the total memory use is about the same;for folios of four pages or more, the usage is reduced. If the kernel iscaching the contents of a 1GB file, it currently needs60MB 16MB of page structures. If that caching isdone entirely with base pages, that overhead will increase to 23MB in thefuture. But, if four-page folios are used instead, it drops to 9MB total.

Some types of descriptors, including those for slab pages and page-table entries, have already beenintroduced. The page-table descriptors are quite a bit smaller thanfolios, since there are a number of fields that are not needed. Forexample, these pages cannot be mapped into user space, so there is no needfor a mapping count.

Wilcox put up a plot showing how many times struct page andstruct folio are mentioned in the kernel since 2021. On the orderof 30% of the page mentions have gone away over that time. Heemphasized that the end goal is not to get rid of struct pageentirely; it will always have its uses. Pages are, for example, thegranularity with which memory is mapped into user space.

Since last year's update, quite a lot ofwork has happened within the memory-management subsystem. Many kernelsubsystems have been converted to folios. There is also now a reliable wayto determine whether a folio is part of hugetlbfs, the absence of whichturned out to be a bit of a surprising problem. The adoption of large anonymous folios has been a welcomeimprovement.

The virtual filesystem layer has also seen a lot of folio-related work.The sendpage() callback has been removed in favor of a better API.The fs-veritysubsystem now supports large folios. The conversion of the buffer cache is proceeding, but has run into asurprise: Wilcox had proceeded with the assumption that buffer heads arealways attached to folios, but it turns out that the ext4 filesystemallocates slab memory and attaches that instead. That usage isn'twrong, Wilcox said, but he is "about to make it wrong" and does not want tointroduce bugs in the process.

Avoiding problems will require leaving some information in structpage that might have otherwise come out. In general, he said, hewould not have taken this direction with buffer heads had he known whereit would lead, but he does not want to back it out now. All is well fornow, he said; the ext4 code is careful not to call any functions onnon-folio-backed buffer heads that might bring the system down. But thereis nothing preventing that from happening in the future, and that is a bitfrightening.

The virtual filesystem layer is now allocating and using large foliosthrough the entire write path; this has led to a large performanceimprovement. Wilcox has also added an internal function, folio_end_read(),that he seemed rather proud of. It sets the up-to-date bit, clears thelock bit, checks for others waiting on the folio, and serves as a memorybarrier — all with a single instruction on x86 systems. Various otherhelpers have been added and callbacks updated. There is also a newwriteback iterator that replaces the old callback-based interface; amongother things, this helps to recover some of the performance that was takenaway by Spectre mitigations.

With regard to individual filesystems, many have been converted to foliosover the last year. Filesystems as a whole are being moved away from thewritepage() API; it was seen as harmful, so no folio version wascreated. The bcachefs filesystem can now handle large folios — somethingthat almost no other filesystems can do. The old NTFSfilesystem was removed rather than being converted. The "netfs" layer hasbeen created to support network filesystems. Wilcox put up a chart showingthe status of many filesystems, showing that a lot of work remained to bedone for most. "XFS is green", he told the assembled developers, "yourfilesystem could be green too".

The next step for folios is to move the mapping and index fields out ofstruct page. These fields could create trouble in the filesystemsthat do not yet support large folios, which is almost all of them. Ratherthan risk introducing bugs when those filesystems are converted, it isbetter to get those fields out of the way now. A number of page flags arealso being moved; flags like PageDirty and PageReferencedrefer to the folio as a whole rather than to individual pages within it,and thus should be kept there. There are plans to replace thewrite_begin() and write_end() address-space operations,which still use bare pages.

Beyond that, there is still the task of converting a lot of filesystems,many of which are "pseudo-maintained" at best. The hugetlbfs subsystemneeds to be modernized. The shmem and tmpfs in-memory filesystems shouldbe enhanced to use intermediate-size large folios. There is also a desireto eliminate all higher-order memory allocations that do not usecompound pages, and thus cannot be immediately changed over to folios; thecrypto layer has a lot of those allocations.

Then, there is the "phyr" concept. A phyris meant to refer to a physical range of pages, and is "what needsto happen to the block layer". That will allow block I/O operations towork directly on physical pages, eliminating the need for the memory map tocover all of physical memory.

It seems that there will be a need for a khugepaged kernel threadthat will collapse mid-size folios into larger ones. Other types of memoryneed to have special-purpose memory descriptors created for them. Thenthere is the KMSANkernel-memory sanitizer, which hasn't really even been thought about.KMSAN adds its own special bits to struct page, a usage that willneed to be rethought for the folio-based future.

An important task is adding large-folio support to more filesystems. Inthe conversions that Wilcox has done, he has avoided adding that supportexcept in the case of XFS. It is not an easy job and needs expertise inthe specific filesystem type. But, as the overhead for single-page foliosgrows, the need to use larger folios will grow with it. Large folios alsohelp to reduce the size of the memory-management subsystem's LRU list,making reclaim more efficient.

Ted Ts'o asked how important this conversion is for little-used filesystems;does VFAT need to be converted? Wilcox answered that it should be done forany filesystem where somebody cares about performance. Dave Chinner addedthat any filesystem that works on an NVMe solid-state device will needlarge folios to perform well. Wilcox closed by saying that switching tolarge folios makes compiling the kernel 5% faster, and is also needed tosupport other desired features, so the developers in the room should wantto do the conversion sooner rather than later.

Comments (11 posted)

Debian dismisses AI-contributions policy

By Joe Brockmeier
May 10, 2024

In April, the Gentoo Linux project banned the use ofgenerative AI/ML tools due to copyright, ethical, and qualityconcerns. This means contributors cannot use tools like ChatGPT or GitHub Copilot tocreate content for the distribution such as code, documentation,bug reports, and forum posts. A proposal for Debian to adopt a similarpolicy revealed a distinct lack of love for those kinds of tools,though it would also seem few contributors supportbanning them outright.

Tiago Bortoletto Vaz startedthe discussion on the Debian project mailing list on May 2, with thesuggestion that the project should consider adopting a policy on the use ofAI/ML tools to generate content. Vaz said that he feared that Debian was"already facing negative consequences in some areas" as aresult of this type of content, or it would be in a short time. He referenced the GentooAI policy, and Michał Górny's argumentsagainst AI tools on copyright, quality, and ethical grounds. He saidhe was in agreement with Górny, but wanted to know how other Debian contributors felt.

Ansgar Burchardt wrotethat generative AI is "just another tool". He noted that Debian doesn't ban Tor, eventhough it can be used to violate copyright or for unethical things,and it doesn't ban human contributions due to quality concerns: "Idon't see why AI as yet another tool should be different."

Others saw it differently. Charles Plessy respondedthat he would probably vote for a general resolutionagainst "the use of the current commercial AI for generating Debian packaging, native, orinfrastructure code". He specified "commercial AI" because "these systems arecopyright laundering machines" that abuse free software, and foundthe idea that other Debian developers would use them discouraging. He was notagainst generative AI technology itself, however, as long as it was trained oncontent that the copyright holders gave consent to use for that purpose.

Russ Allbery wasskeptical of Gentoo's approach of an outright ban, since "it is(as they admit) unenforceable". He also agreed with Burchardt,"we don't make policies against what tools people use locally fordeveloping software". He acknowledged that there are potentialproblems for Debian if output from AI tools infringes copyright. Evenso, banning the use of those tools would not make much difference:"we're going to be facing that problem with upstreams aswell, so the scope of that problem goes far beyond" directcontributions to Debian. The project should "plan to be reactive [rather] thanattempt to be proactive". If there are reports that AI-generated content is acopyright violation, he said, then the project should deal with it asit would with any DebianFree Software Guidelines (DFSG) violation. The project may need tomake judgment calls about the legal issues then, but "hopefully thiswill have settled out a bit in broader society before we're forced tomake a decision on a specific case".

Allbery said his primary concern about the impact of AI is itspractical impact:

Most of the output is low-quality garbage and, because it's now automated,the volume of that low-quality garbage can be quite high. (I amrepeatedly assured by AI advocates that this will improve rapidly. Isuppose we will see. So far, the evidence that I've seen has just led meto question the standards and taste of AI advocates.)

Ultimately, Allbery said he saw no need for new policies. If thereis a deluge of junk, "we have adequate mechanisms tocomplain and ask that it stop without making new policy". The onlystatement he wanted to convey so far is that "anyone relying on AIto summarize important project resources like Debian Policy or theDevelopers Guide or whatnot is taking full responsibility for anyresulting failures".

A sense of urgency

In reply to Allbery, Vaz concededthat Gentoo's policy was not perfect but, despite the difficulty inenforcing it, he maintained there was a need to do somethingquickly.

Vaz, who is an applicationmanager (AM) for the Debian new maintainer process, suggested that Debian was already seeingproblems with AI output submitted during the new maintainer (NM)process and as DebConf submissions, but declinedto provide examples. "So far we can't [prove] anything, and even ifwe could, of course we wouldn't bring any of the involved to thepublic arena". He did, however, agree that a statement was a moreappropriate tool than a policy.

Jose-Luis Rivas repliedthat Vaz had more context than the rest of the participants in thediscussion and that "others do not have this same information andcan't share this sense of urgency". He inferred that an NMapplicant might be using a large-language model (LLM) tool during the NM process, but in thatscenario there was "even less point" in making policy or astatement about the use of such tools. It would be hard to prove that an LLM wasin use, and "ultimately [it] is in the hands of those judging"to make the decisions. "I can't see the point of 'somethingneeds to be done' without a clear reasoning of the expectations out ofthat being done".

Vaz arguedthat having a policy or statement would be useful, even in the absence of proofthat an LLM was in use. He made a comparison to Debian's code ofconduct and its diversity statement: "They might seem quite obvious to some, andless so to others." Having an explicit position on the useof LLMs would be useful to educate those who are "getting to use LLMs intheir daily life in a quite mindless way" and "could help us bothavoid and mitigate possible problems in the future".

The NM scenario Vaz gave was not convincing to Sam Hartman, who repliedthat the process would not benefit from a policy. It is up tocandidates to prove to their application manager (AM), advocates, andreviewers that they can be trusted and have the technicalskills to be a Debian Developer:

I as an AM would find an applicant using an LLM as more than apossibly incorrect man page without telling me would violate trust. Idon't need a policy to come to that conclusion.

He said he did not mind if a candidate used an LLM to refresh theirmemory, and saw no need for them to cite the use of the LLM. But ifthe candidate didn't know the material well enough to catch badinformation from an LLM, then it's clear they are not to be trusted tochoose good sources of information.

On May 8, after the conversation had died down, Vaz wrotethat it was apparent "we are far from a consensus on an officialDebian position regarding the use of generative AI as a whole in theproject". He thanked those who had commented, and said that hehoped the debate would surface again "at a time when we betterunderstand the consequences of all this".

It is not surprising to see Debian take a conservative,wait-and-see approach. If Debian is experiencing real problems from AI-generated content,they are not yet painful or widespread enough to motivate support for aban or specific policy shift. A flood of AI gibberish, or a successfullegal challenge to LLM-generated content, might turn the tide.

Comments (147 posted)

Managing expectations with a contributions and credit policy

May 13, 2024

This article was contributed by Valerie Aurora

Maintainers of open-source projects sometimes have disagreements withcontributors over how contributions are reviewed, modified, merged, andcredited. A written policy describing how contributions are handled canhelp maintainers set reasonable expectations for potential contributors.In turn, that can make the maintainer's job easier because it can helpreduce a source of friction in the project. A guide to help create thiskind of policy for a project has recently been developed.

People sometimes have rather different expectations about how open-sourceprojects function with regard to contributions. For example, a recent discussionabout how to credit a Linux kernel patch that had two authors attractedmore than 600comments, covering a wide range of opinions from "theoriginal author should have sole credit" to "the original author should getno credit at all". Another kind of disagreement is over which types of contributions are welcome: some projects don't want external contributions, or any new features, but contributors keep sending them anyway.

In the absence of a written policy, contributors will make assumptions about the way a project operates—and may start an argument if they don't get the expected response. A written policy describing how contributions are processed and credited can prevent conflicts from even starting.To help maintainers create their own policies, I co-authored a credits and contribution policy development guide with Maria Matějka, Martin Winter, Marcos Sanz, and other members of the RIPE Open Source Working Group.

Coverage

A contributions and credit policy is most useful when it focuses on areas where maintainers and contributors frequently have different, incompatible expectations, such as: what contributions are welcome, how reviews and changes will be handled, and how credit will be assigned. A policy can also include step-by-step instructions for maintainers and contributors on how to handle complicated situations, such as when a contributor won't make requested changes.

A policy can and should cover all kinds of contributions. Melissa Mendonça, a maintainer for NumPy, SciPy, and napari, said: "Often, things like docs, community, design work are not credited as contributions in the usual sense (i.e., don't give you green squares on GitHub)." A policy can help surface these contributions and create a standard process for acknowledging them.

A policy can help resolve three common dilemmas: assigning credit for multiple direct contributors, revising contributions, and attracting the right contributions.

Deciding on who deserves credit for a work is a difficult and unsolvedphilosophical question. Practically speaking, people assign credit for manyreasons, such as rewarding people for contributing, establishing ownershipof intellectual property, or tracking down the source of bugs or backdoors.

As the authors of the policy development guide, we took a practical approach to this question with our own contributions and credit policy. We asked ourselves what kind of behaviors we wanted to encourage in our project, and then assigned credit in ways that make those outcomes more likely. Since we want to attract new contributors, we give primary author credit to the new contributor even if a maintainer had to completely rewrite the contribution. Other projects can make similarly pragmatic decisions without solving the general problem of who gets credit.

Sometimes people will try to take advantage of a formal creditpolicy. Mendonça says that credit for open-source work is "always a systemthat you can game (for example, making just enough contributions to get acertain credit/position and then moving on)". However, she says that she is"willing to err on the side of giving credit", and take corrective actionafterward. Any policy can have an explicit exception for the maintainer totake whatever action they think necessary.

Multi-contributor credit

Some policies for assigning credit are easy to implement, such as the current de facto policy for many projects: "whoever merges the code decides who gets credit". However, for more complex credit policies, some difficult situations quickly arise.Matějka, team leader for the BIRDrouting daemon (and a co-author of the policy guide) said:

We sometimes get contributions with good ideas and poor quality. It'salways a discussion how much the specific patch is more an idea (to bethanked for in the commit message) or a patch (where we keep the author butadd a note that it was updated by the committer).

Conversely, credit for a multi-author contribution can also turn intoblame; if the original author cannot review the change, crediting the changes to them can end up blaming them for the editor's mistakes.Solving complicated questions of credit can use hours of a maintainer'stime or result in a contribution hanging in limbo for months. A standardpolicy lets maintainers decide how to handle these situations once and thensimply refer to the policy in the future.

Many contributions need some revision before they can be merged into the main project. This can go wrong in many ways: maybe no one has time to review, contributors don't respond to requests, or the contribution no longer merges by the time it is approved.A policy can help with these situations in two ways: it can give the maintainer a playbook for how to handle difficult situations (e.g., after two weeks of no response, make the edits and merge the contribution), and it can give contributors guidelines for when to ping the maintainers (e.g., after two weeks of no response, email the mailing list again).

Attracting the right contributions

Many people assume that "open source" means "open to outside contributions and willing to mentor new contributors". In reality, some projects don't want new contributors or features, others will accept outside contributions but only after they have been totally rewritten, and only some maintainers have the time and interest to mentor new contributors.An explicit policy about what contributions are desired helps maintainers avoid conflict from would-be contributors. It also helps both potential contributors and potential users decide whether they want to depend on a project with that particular contribution policy.

One of my early open-source contributions was guided by a contributionspolicy of sorts. The xjackscreensaver prints "All work and no play makes Jack a dull boy" overand over, with a variety of mistakes and typos, inspired by a scene in TheShining. A commentin the source code tells would-be contributors not to bother sending in apatch to change the words it prints. However, while I was doing exactlythat, by changing it to print "All work and no play makes Val a dull girl", I found a bug in the code that generated typos. I sent in a patch to fix that bug, which was merged (without credit).

Policy resources

The credits and contribution policy development guide includes several variations on each section of the policy, as well as a list of policies in use by open-source projects. We request (but do not require) giving the authors credit on any derivative work.The policy guide has its own contributions and credit policy and new contributors are welcome. Currently we are especially interested in additional examples of choices for each section of the policy, as well as links to existing policies.

Comments (12 posted)

LWN.net Weekly Edition for May 16, 2024 (2024)
Top Articles
Latest Posts
Article information

Author: Otha Schamberger

Last Updated:

Views: 5924

Rating: 4.4 / 5 (75 voted)

Reviews: 82% of readers found this page helpful

Author information

Name: Otha Schamberger

Birthday: 1999-08-15

Address: Suite 490 606 Hammes Ferry, Carterhaven, IL 62290

Phone: +8557035444877

Job: Forward IT Agent

Hobby: Fishing, Flying, Jewelry making, Digital arts, Sand art, Parkour, tabletop games

Introduction: My name is Otha Schamberger, I am a vast, good, healthy, cheerful, energetic, gorgeous, magnificent person who loves writing and wants to share my knowledge and understanding with you.