{"version":"https://jsonfeed.org/version/1.1","title":"Web log on dee.underscore.world","home_page_url":"https://dee.underscore.world/blog/","feed_url":"https://dee.underscore.world/blog/feed.json","icon":"https://dee.underscore.world/favicon.png","authors":[{"name":"Dee","url":"https://dee.underscore.world"}],"language":"en-US","items":[{"id":"https://dee.underscore.world/blog/y2k-as-seen-on-tv-as-seen-now/","url":"https://dee.underscore.world/blog/y2k-as-seen-on-tv-as-seen-now/","title":"Y2K, as seen on TV, as seen now","date_published":"2024-01-26T21:29:31.000Z","date_modified":"2024-01-26T21:29:31.000Z","content_html":"
HBO recently released a documentary called Time Bomb Y2K. Several places—including IMDb—offer a short blurb about the film as such:
\n\n\nAn immersive, all-archival retelling of the "Y2K" millennium bug and the mass hysteria that changed the fabric of modern society.
\n
In some corners of the Internet, this summary caused the particular reaction that seems to have become common whenever the year 2000 problem is mentioned. Although the 1990s were in the ancient times of over 24 years ago, there are still people alive who were alive back then. Some of those people were also, at the time, involved in year 2000 problem mitigations. Some of those people take issue with framing of the problem as "mass hysteria", pointing out that the issue was actually real, and it was work like theirs that ensured the issue was addressed before it became a more serious problem.
\nIndeed, this seems to be the trajectory the year 2000 problem has taken in popular perceptions. Previously, people—either those who remembered that time, or those who learned about it later—often associated it with the prevalence of irrational fears of impending literal apocalypse. Recently, though, it is more common to see people vehemently arguing on the other end of the spectrum, and defending the position that the year 2000 problem was actually very serious
\nBlurb notwithstanding, the film actually does not attempt to make the point that the year 2000 problem was a lie. Nevertheless, the broader shifting of sentiment about the year 2000 problem, as manifested in the reaction to the film, does make sense if we consider how serious, global problems are talked about today. A useful way to approach the film, then, is with an eye on how the 1990s and the 2020s differ in how they deal with their problems.
\nDirected by Marley McDonald and Brian Becker, and first shown at a film festival in early 2023, Time Bomb Y2K consists entirely of contemporary footage from the 1990s, with no extra narration. The film splices news reports, snippets of documentaries, assorted B-roll, less-professionally published tapes, and even home movies into a chronological narrative.
\nThe film starts in the mid-1990s, and moves forward from there, with larger portions of the film devoted to the times closer to the year 2000. It tries to convey the general mood of the times, or at least the general mood of the times as seen on American television. It also includes some coverage of more fringe elements of society, and their reaction to the year 2000 problem and its possible consequences. At the end, we find out that (spoiler alert) civilization did not end on January 1st, 2000, and get to watch the people of the year 2000 express their relief and joy.
\nAt times, the film is cheeky, referencing our current times in the same way that a prequel from a popular franchise may reference later canon. Elon Musk, Jeff Bezos, and Osama bin Laden show up at various points. At one point, someone mentions an example of a car that may refuse to start after the year 2000, because it believes it has not been serviced for a hundred years, which seems directed at the people of 2023 and their cars that refuse to start due to a failed over-the-air software update.
\nFor inhabitants of 2024 who are looking back at the year 2000 problem, the COVID-19 pandemic and the climate crisis both come to mind. All three are examples of global problems, which require considerable efforts to address. All three are things that people could have been better prepared for earlier on.
\nIt is these comparisons that are likely why current day discussions of the year 2000 problem often involve strong condemnations of seeing the problem as exaggerated. The hypothetical individual who is the target of condemnation here is one who believes that the climate crisis and the COVID-19 pandemic are also exaggerated problems, and the efforts already spent on them were excessive. This individual would then presumably point at the year 2000 problem as another example of something that generated more concern than it should have—a mass hysteria, if you will.
\nIt is true that the year 2000 problem could have been addressed earlier. Even before the 1990s, people were aware of the fact that the year 2000 may happen in the future, and that some of their computer systems were incapable of handling it properly. Eventually, though, enough awareness of the problem was raised in the mainstream, and successful mitigations were applied, which is something many wish would happen with problems of today. There are, however, some fundamental differences between then and now.
\nOne difference between the 2020s and the 1990s that Time Bomb Y2K highlights is in how people interacted with news, and information in general.
\nA point that frequently comes up in news reports and interviews shown in the film is that the looming problem may affect even those who do not own a computer. This made things more scary for people of the 1990s, but to the people of 2020s, it also serves as a reminder that back then, it was less common for people to get their news through a computer.
\nFrom the vox pop interviews included in the film, one gets the impression that people mostly had only a vague idea of what the year 2000 problem was, how it would affect them, and what was being done to fix it. These people have the idea that someone, somewhere is doing something to fix it, but they do not know the details, and they do not know how well that is going.
\nThis sort of mass anxiety stands in contrast with the modern day. In the 2020s, people are far more confident in expressing opinions and providing explanations about what is happening, regardless of whether those are based in fact or not. It may be tempting to think that misinformation in the 1990s spread the same way it spreads now, but it could not have, and so things worked differently.
\nThe film also covers the more fringe responses to the impending year 2000. These range from the more ordinary individualist preppers, to right wing militias, and apocalyptic religious movements.
\nSome people shown take reasonable precautions, ensuring they have enough supplies to survive a possible disaster, in concert with their community. Some take a more extreme approach, stockpiling a remote residence in the mountains with food and guns for the coming societal breakdown. Some quit their jobs and take up homesteading, since the whole civilization thing is going to end anyway.
\n\nThere are also religious leaders who preach about the coming apocalypse, connecting the end of the world predictions associated with the year 2000 problem with more traditional apocalyptic prophesies. Depicted, too, are right-wing militias, who believe that the year 2000 will usher both a societal collapse, and prompt the federal government of the United States to seize dictatorial levels of control over the nation.
\n\nRight-wingers with a fondness for AR-15s and conspiracy theories regarding a coming New World Order are also a familiar feature of 2024. Their modern-day iteration is, however, more frequently associated with denialism. A modern day right-wing militia may believe that efforts to combat the climate crisis are a plot to gain power (by whatever group the militia is bigoted against), but they will also believe that the climate crisis is made up in the first place.
\nFrom the film, one can get the impression that fringe right-wing and conservative movements in the year 2000 problem era were far closer to the shared reality of everyone else. They kept one foot outside of that reality, as manifested by their theories about conspiracies or religious apocalypses, but they built their narratives on the understanding of the world shared by the broader population. This stands in contrast with COVID-19 or the climate crisis, where the fringe right-wing elements often advance the idea that those things are not real, or if they are, the mitigation efforts are not real.
\nIt is tempting to use the year 2000 problem as an example of how we should do things now. On the surface, it seems like the good old times, when people would actually manage to solve major problems collectively. But, as Time Bomb Y2K reminds us, it was a different time, and it was a different problem. There is no going back to the culture of the 1990s—not that this would be such a good idea in the first place—and there is no going back to the way people interacted and communicated with each other back then.
\nWe are also not facing the year 2000 problem today. Ours are not looming apocalypses with set deadlines. Our problems are not addressed by a bunch of work done behind the scenes, in a world that keeps going on as usual. These problems are not the kind that stop banks from being able to count their money, spurring them to action; they are problems that result in poor people dying, and the 1990s were not that much better about dealing with those.
\nThe ending of the film depicts the sort of relived, happy, and optimistic mood present during the early year 2000. There is a shot of a sanitation worker disposing of a discarded sign that says "THE END IS NEAR", as if thus getting rid of the gloom and fear of the recent times and ushering in a hopeful future. The very last scenes in the film are interviews with kids, sharing their hopes for a better future. This is the sort of a moment in time that people are likely to feel nostalgia for. A moment where people just got through a problem together, and were standing there looking towards the future, hopeful to be likewise unified in dealing with the problems of the coming century.
\n\nBut, the story has, of course, been spoiled for anyone living in the year 2024. We know how the 2000s ended up, with their persisting bigotries, wars, crises, and suffering. With the benefit of hindsight, that view from 2000 seems awfully naive.
\nIt is true that the year 2000 problem was real. It is true that a lot of effort went into ensuring that it would not have serious consequences once the year 2000 actually rolled around. On the other hand, it is also true that, at the time, there were reactions among certain people that exaggerated the problem to advance their conspiracy theories and agendas. This particular sort of hysteria is useful to acknowledge, as echoes of it persist to 2024. However, it is important that, when trying to counteract the misconceptions or denialism about how this particular bit of history went down, we do not erase all the nuance. It is useful to take inspiration from past successes, but it is less useful to turn them into myths of glorious past times.
\nStills included here are from Time Bomb Y2K, copyright to which is held by Home Box Office, Inc.
\nCurrently, The Unicode Standard sees one major release per year (barring unusual exceptions), with an occasional additional point release outside of that normal pace. 2023 has been one of the exceptions, in that there was only minor release: 15.1.
\nBoth minor and major version updates can come with new emojis. These new emojis subsequently become available in various pieces of software... eventually, one hopes. To understand how new emojis make their way to actual software in use, it helps to understand what the data behind emojis is, where it comes from, and how it is used.
\nWhat is often referred to as Unicode is, more precisely, The Unicode Standard. The Unicode Standard, published by the Unicode Consortium, consists of tables of codepoint assignments, as well as a number of documents describing how the standard's data should be used, and broader practices for handling Unicode text. The Standard includes machine-readable data files, the Unicode Character Database (UCD), which describes codepoint assignments, plus some related extra data.
\nA codepoint (or code point) is essentially a number, and so The Unicode Standard determines what—if any—character is assigned to a given number. Plenty of codepoints are yet unassigned, and so new assignments can be introduced in new versions of the standard. Each character also has certain additional information associated with it, in the form of character properties. These include things like the identifying name, general category, and directionality (some characters are associated with left-to-right scripts, some are right-to-left, and some are more complicated than that). The UCD consists of a number of files which contain this data.
\nSome—but not all—emojis are single characters, and these a get a single codepoint each. UnicodeData.txt
is the file within the UCD that lists codepoint assignments, and so we can find such emojis in it:
The UCD also includes files which specify which characters are actually emoji, and which should have emoji presentation (that is, which should show up in color). Furthermore, The Unicode Standard comes with additional emoji-related, computer-readable data tables that are not part of the UCD proper.
\nNot all emojis are single characters—some are sequences. For example, the astronaut emoji (🧑🚀) is assembled from the adult emoji (🧑) and the rocket emoji (🚀), plus some glue characters. Similarly, the trans flag emoji (🏳️⚧️) involves combining the white flag emoji (🏳) and the trans symbol emoji (⚧), plus some glue characters. Such sequences do not require new codepoint assignments, and could theoretically be devised by vendors on an ad-hoc basis. The standard, however, includes a number of such sequences which are expected to be widely deployed—or, as Unicode puts it, recommended for general interchange (RGI). These sequences are one of the things specified in the emoji data tables that exist outside of the UCD, in a file that looks like this:
\n\nEmoji additions to The Unicode Standard are handled by the Emoji Subcommittee (ESC), which operates under the Unicode Technical Committee. Proposals for new emojis (either codepoint or sequence) are open to the public, although they also sometimes originate from within the Consortium. For proposals that make it to the later stages of the approval process, the ESC will generally make the proposal public, and report on its progress. It is therefore possible to tell, ahead of time, what will end up in the next Unicode Standard release.
\nThe Common Locale Data Repository (CLDR) is a project aimed at maintaining a standard repository of a variety of locale data. It is a project maintained by the Unicode Consortium, though it is not part of The Unicode Standard.
\nThe CLDR contains locale data for a wide range of language, region, and script combinations. This includes some of the more usual locale data, such as the way numbers or dates are formatted, or how the days of the week are written. The CLDR also contains some less obvious locale data, such as information on how person names in a given culture are usually collated, or how many different plural forms a language uses.
\nAmong the less obvious CLDR data are character annotations. These are used to assign a short name, and any number of keywords to every emoji. The short name is the main name that an emoji will have, and may be used for text-to-speech systems that need to read the emoji out loud. The keywords are any additional terms that may be useful when searching for that particular emoji. While The Unicode Standard gives each codepoint an English name, that name is intended more as an internal identifier, suitable for use in source code, and such names are only given to codepoints, not to emoji sequences. CLDR annotations, on the other hand, are for both single character emojis and sequences, and are different in every language, making them more suitable for the end user.
\nAnnotation data is in XML, and looks like this:
\n\nThe CLDR is managed by the CLDR Technical Committee (CLDR-TC) within the Unicode Consortium. The project is maintained by Consortium members, as well as affiliated institutions and individuals with relevant language expertise. Unnaffiliated public can also participate in a limited way, by filing requests for changes, to be reviewed by the members. The CLDR is released on its own six month schedule, independent of The Unicode Standard.
\nIn order to render new emojis, software will generally require new graphics that depict them, in addition to possibly needing the latest version of the Unicode Character Database. Standards changes that would require code changes to the rendering stack are less common.
\nAs the UCD describes which characters are emojis, updating it may be necessary to inform software about which characters it should treat as emojis. When it comes to emoji handling by the operating system, this generally means a lower-level component or library, which may either vendor the UCD, take it in at build time, or vendor an already pre-processed version of the UCD.
\nThe other thing needed to display new emojis is the graphics to actually display them. While the Unicode Character Database does not contain any instructions on what a given emoji should look like, the proposals and other documentation on the Unicode website generally do include examples. These are also usually available prior to the formal release date for the given Unicode Standard version, so it is technically possible for vendors to have graphics ready before that date.
\nTwo popular permissively licensed emoji sets are Google's Noto Emoji, and Twitter's Twemoji. Both of these projects often put out a release that supports the new version of The Unicode Standard within about a month after The Unicode Standard's release, though some outliers do happen. In particular, Twemoji was affected by a petulant billionaire buying Twitter, which resulted in several people involved in handling of the project being fired, and apparent halt to Twitter's (now called "X") releasing of emojis under a permissive license. The Twemoji project now continues as a fork, maintained by some of the original authors.
\nBoth Noto Emoji and Twemoji are shipped as a series of vector graphics. These graphics can be built into font files, and such fonts are generally how new emojis end up making it into desktop and mobile operating systems. Google ships the tooling needed to turn Noto Emojis into an OpenType font, and also makes pre-built font files available; Twemoji fonts can be built with a third party tool twemoji-color-font, with pre-built fonts likewise distributed by that project.
\nOther times, the graphics are used more directly. Web applications, for example, may replace emoji characters by images loaded via HTML. In such situations, there is often a dedicated library which either brings its own graphics, or can be configured to use a separately provided graphics set It is these components which need to be updated in such a situation.
\nIt is technically possible to input emojis the same way as any other arbitrary Unicode character: through character pickers, numerical codepoint entry, or copy-and-paste from an outside source. These methods are not very practical, though, which is why emoji pickers, and other emoji input methods exist.
\nEmoji pickers can source their data from multiple places, but the Common Locale Data Repository is often among them. Like with the UCD, the CLDR data is often vendored, and is often pre-processed in some way prior to being vendored. As such, building the picker with an updated CLDR may require an extra step to update the CLDR-derived data files within the picker's source code.
\nEmoji pickers also often use an intermediate dependency as a way to access emoji data. While with the UCD, software will generally get its Unicode data just from the UCD, emoji pickers may use CLDR partially, indirectly, or even not at all. Projects such as emoji-data or emojibase maintain their own lists of emoji shortcodes which may be derived from CLDR data, but are not always a one-to-one mapping. Such projects may take the CLDR into consideration, but the shortcodes and other keywords are ultimately assigned by the project, and so subject to individual editorial control. This means updates are not automated.
\nAs with displaying emojis, emoji input also ends up implemented on multiple layers: there are OS-level emoji input methods, and individual apps or websites can have emoji input methods of their own. Of note is the fact that, while a website replacing emojis with its own images means the original glyphs are not rendered, a website providing shortcode expansion usually does not prevent an OS-level picker from inserting emojis like any other Unicode character.
\nWhen writing software that may require emojis data from the UCD or the CLDR, there are some things to consider.
\nParsing UCD or CLDR data directly at runtime may be impractical, and so pre-processing it into a format that can be included at build time often makes sense. In this case, it is also common to vendor the data, so that the UCD or CLDR data does not have to be supplied as a build-time dependency.
\nNew emojis are generally introduced in the UCD once a year, and the CLDR is updated twice a year. This represents an update that has to be performed on a regular basis, albeit rarely. When vendoring this data directly, it can be useful to set up procedures for updating it in the future. If possible, it can also be useful to set up a way for packagers to patch in the latest UCD or CLDR data at build time, even if it is otherwise vendored.
\nWhen implementing emoji pickers on app level, it is good to remember that the user may actually have a functioning method for inputting emojis at the operating system level. Most of the time, when dealing with text input fields, this doesn't require any extra handling. There are, however, situations where the user is forced to input an emoji through the picker, like when adding a reaction to an item in a chat app. In such cases, the user may still wish to, for example, select one of the recent emojis from their phone's keyboard, which is something that the picker should facilitate.
\nBoth the UCD, and the CLDR, including the parts excerpted here, are available under the Unicode Data Files and Software License. More details are available on the Unicode website.
\nOne of the fun things about running your own personal website is that you can add things to it that are not strictly needed, but might be interesting to add anyway. While I try to not actually break the basic functionality of the website, I feel more free to add miscellaneous new stuff.
\nThis is how I ended up deciding to add Pagefind-powered search to my website. I assume that people do not regularly visit my website and then find themselves wishing it had a search feature, so that they could use it to locate a particular web log article, considering I have published fewer than 25 of those over the entire time this website has existed. But, it was possible for me to add such a feature, and so I did.
\nSearch engine implementations for static websites can generally be divided into two categories: server-side and client-side. With server-side, something has to build an index of all the pages within the site, and then something has to handle search-related HTTP requests by consulting that index. Unlike a static website, such functionality cannot be hosted off a server that simply serves static files over HTTP—the thing that consults the index needs to be able to execute code on the back end, which makes hosting more complicated. This kind of search functionality often ends up outsourced to a third party.
\nThe other option for static website search is moving the search to the client. In the simplest implementation of this idea, the client downloads a search index of the website, along with some JavaScript code capable of using that index. These can be static files, so the server does not have to do anything fancy, but it also means the client has to do more work. While extra CPU cycles and memory use by the client can be an acceptable trade-off, the bigger problem can be transferring the index. Since such an index is essentially the whole website, but in a different shape, it can involve transferring quite a lot of data to the client.
\nPagefind solves the index size problem by splitting the index into chunks. While the index is still mostly the whole website in a different shape, the whole index does not have to be downloaded to perform a search. Instead, the client ends up downloading only the chunks it needs to search for the given terms, and to display the relevant results.
\nPagefind is implemented as a command line tool, written in Rust. The tool is designed to be operated on a local copy of the site's directory tree of HTML files, and it emits a subdirectory of files which implement the search engine, into that copy. It creates an index of all the pages in the site, while also gathering metadata about each page. It then splits that search index into chunks, and saves those index chunks, as well as the metadata for each page, into individual, compressed files. The tool also emits the client API code—a combination of JavaScript and WebAssembly—as well as an implementation of a search user interface that uses the API.
\nSince I run Eleventy directly, and not from a build system or task runner, I wanted to run Pagefind as part of the Eleventy build process. After version 1.0, Pagefind comes with a Node.js interface, which runs the Pagefind binary, and talks to it over standard input and output. I, however, started working on this problem before Pagefind 1.0 was released, and so I went with the simpler method of invoking the Pagefind binary directly.
\nEleventy emits a number of events during a build process. One of those events is eleventy.after
, which happens after a build is done (predictably enough). We can hook into this event to run Pagefind on the site, after it's done building:
The npm package for Pagefind pulls in a pre-compiled binary (at least for the supported platforms), so we are calling it through npx
. Since we do not use Pagefind's Node.js API, we could also provision the Pagefind binary in some other way, get it into PATH
, and call it directly. The npm approach is easier, but might be undesirable for situations where building such things locally is preferable.
Pagefind can store arbitrary key–value pairs of metadata for each page it indexes, and the client API can then be used to access that data for each result it finds. Pagefind stores some metadata by default—for example, the title of a result—and then uses that metadata when rendering results with the default results interface.
\nSince Pagefind indexes the finished HTML output of a static site, this is also where it gets its metadata from. To indicate to Pagefind where this metadata is, specific data-*
attributes can be used. For example, if we wanted to store the datetime
attribute of a <time>
element under the key date
, we would add data-pagefind-meta="date[datetime]"
to that <time>
element:
The Pagefind documentation has detailed instructions on how these attributes are used.
\nPagefind comes with a pre-made search interface, which can be included in a page by loading one of the JavaScript files from the Pagefind binary's output. This interface is fine, but I wanted to build my own, partly because it'd be fun, and partly because I wanted the interface to better integrate with my website.
\nBuilding a custom search interface requires use of the Pagefind API. Fortunately, the API is fairly simple. A basic query can look like this:
\n\nBoth the init()
call and the search()
call involve network requests. init()
causes Pagefind to load general metadata and configuration relating to the search database, as well as the relevant WebAssembly bytecode. Once init()
is done, Pagefind can figure out which chunks of index it should download for a given query, and these are the requests that search()
makes.
The object returned (within a promise) by the search()
call does not, however, contain all the metadata for each result. Each potential result—which is to say, each page—has its own file containing both the metadata which Pagefind gathered, as well as the full searched text of the page. For each result, Pagefind can be asked to make a request for the corresponding file. The downside of this approach is the need to potentially make more requests to build a search results page, but the upsides are the fact that the client does not have to download metadata for pages that are not relevant. When using pagination, the client can also delay loading data for further pages, until it is needed.
Fetching the data for a single result looks like this:
\n\nPagefind returns all the metadata fields as strings, which means something like the date field might need to be parsed back into a Date
object.
When listing web log posts, my website uses a template fragment to render a post strip—a card which includes the publish date, the title, as well as a short summary of what the post is about. I use this on the front page, as well as on the page which lists all the published web log posts. It looks something like this:
\n\nI wanted to reuse the post strip for search results that are blog posts. Using post strips there would both provide consistent design, and also supply some relevant information about what each result is—an excerpt highlighting relevant terms does not always provide that. Pagefind's metadata support provided an obvious means to supply the relevant fields needed for rendering the strip.
\nI was writing the search interface in vanilla JavaScript (to make it more fun), and ideally wanted to reuse the same partial template on the server side and the client side. This could be done with LiquidJS—which is one of the template engines that comes with Eleventy, and which is what I use for most of the site—but that would require the client to load the whole LiquidJS library (or at least a large portion of it), and that feels like overkill for a relatively simple partial template.
\nEleventy also supports EJS. Since EJS supports pre-compiling templates, a template can potentially be used both from Eleventy at build time, and can also be bundled into the client-side JavaScript code for use there. The problem with doing this in practice is that a template written for use with Eleventy will end up using a bunch of Eleventy-supplied names and functions, and so it will not be easy to use from places outside Eleventy—like the client-side search script.
\nPlain JavaScript is another template format supported by Eleventy, however. Thus, one solution for creating a reusable EJS template is to create a JavaScript wrapper which loads a generic, reusable template, takes a bunch of data from Eleventy, and renders the template by putting that data into a shape that the template can use. This is the solution I decided to go with, and since I was no longer using Eleventy's built in EJS support anyway, I decided to employ Eta instead (which is supposed to be like EJS, but better). I added an Eleventy filter for rendering Eta templates:
\n\nA wrapper can then look something like this:
\n\nI can then use renderFile
to call the wrapper, from a place like the web log listing page:
For the client side, I use Rollup to pre-compile the Eta template, and then have a function which takes a single Pagefind search result object, and renders a post strip out of it:
\n\nFor a more detailed example I have a branch of my Eleventy example repository that contains enough code to roughly reproduce what my actual website does for Pagefind.
\n"},{"id":"https://dee.underscore.world/blog/ingesting-secrets-as-a-daemon/","url":"https://dee.underscore.world/blog/ingesting-secrets-as-a-daemon/","title":"Ingesting secrets as a daemon","date_published":"2023-09-15T15:35:01.000Z","date_modified":"2023-09-15T15:35:01.000Z","content_html":"Server software and other long-running daemons sometimes need to have access to local secrets. These can include things like private keys for a X.509 certificates, passwords for talking to an SMTP server, or access tokens for communicating with other services. In some setups, secrets can be obtained from a remote service over network, but other times there is a need for the secret to already be present on the machine.
\nAn administrator may wish to keep such secrets separate from other kinds of configuration the daemon needs. An example of such approach is NixOS, where generated configuration files often end up in the world-readable Nix store, and so it is desirable for them to not contain secrets. It is also a useful practice in other deployments. For example, one may wish to make a configuration file public, and if such a file contains secrets, this can lead to their inadvertent disclosure.
\nThe ways that secrets can be kept separate from other configuration depends on a the particular software's approach to handling configuration. Some implementations anticipate the need to separate secrets out, while others necessitate the use of workarounds.
\nThe simplest approach to handling secrets is to have them specified in a configuration file, just like every other configuration directive. For daemon authors, this makes things easier—there are libraries for loading settings from common serialization formats, like TOML or YAML, and using them means less need to write extra logic.
\nThe way NixOS modules deal with such situations is either string replacement, or processing structured data. In either case, a config file is first written out to the Nix store without the secrets in it. Before the daemon starts up, the config file is copied to ephemeral storage (such as tmpfs), and the secrets are read from elsewhere and inserted into it. With the string replacement approach, the input config file has placeholders in it, and these are searched for and replaced with actual secrets. When the file uses a common format like YAML or JSON, the other approach is to use tools like yq (the one written in Python), yq (different one, written in Go), or jq to parse the config file, and insert the secret key–value pairs where they are needed. Once the secrets are in the config file copy, the daemon can be launched, and—if needed—given the path to the modified config file copy as a command line argument.
\nOne thing to keep in mind when pre-processing config files in this manner is that command line arguments for running processes are usually readable by any user on the system. This means that any secrets provided on the command line—such as with sed
replacements—could be intercepted. With NixOS, workarounds for this problem include use of --rawfile
(with jq and the Python yq); use of the load_str
operator (with the Go yq); or, in place of sed
, using replace-secret
, a small utility written for the purpose. All of these means look for secrets in separate files, instead of command line arguments, enabling easier access control.
Some daemons—especially ones with bespoke configuration file formats—have the option of reading multiple configuration files. This can include config.d/
-type solutions, where the daemon reads every file inside some specific directory, or support for include
-type directives, where a configuration file can specify other configuration files to read.
In this situation, a separate config file can be created with just the sensitive directives. Such config files can, likewise, be placed in ephemeral storage, and then included in the rest of the configuration hierarchy, either though the use of symlinks, or through include
directives given the secret file's path. Such separation is useful even setups not facilitated by tools like NixOS modules, since it decreases the chance of accidentally committing secrets to version control, or publishing them alongside the rest of the configuration.
A common way for daemon software to support configuration—advocated by the Twelve-Factor App methodology—is via environment variables. Environment variables are handy for deployments with Docker and similar platforms, as feeding environment variables to the inside of a container is often easier than handling configuration files. Implementations usually merge configuration sources, constructing the effective configuration from the loaded config files, overlaid with the supplied environment variables. Environment variables thus provide a way to insert secrets into a separately specified configuration.
\nThere are multiple options for getting secrets into environment variables. If the secrets are in their own files, an expedient solution is a wrapper script that reads the secrets into environment variables before launching the main daemon. Sops (by itself, without sops-nix) has an exec-env
subcommand, for loading a Sops file into an environment, and then executing a process in that environment. systemd units files have the Environment
and EnvironmentFile
directives. The latter directive is frequently used in NixOS modules, with the whole environment file treated as a secret.
There are some problems with sticking secrets in environment variables. The situation is not entirely dire: unlike with command line arguments, by default, a process's environment is not trivially readable to any user on the system. On the other hand, using environment variables for secrets makes the variables sensitive, and so they have to be handled more carefully—for example, debug logs dumping the entire environment could leak secrets. Child processes also inherit the parent's environment, even if they end up running as a different user, which means they could gain access to secrets they should not have access to. systemd subscribes to the idea that environment variables are a poor way to supply secrets to processes, and so it does not have native support for putting credentials obtained via LoadCredential
in environment variables. systemd's credential functionality is happy to provide the path to a file containing the secret in an environment variable, but not that secret's actual value.
Separate files, each containing a secret, are a common interface for providing secrets to software. This is how systemd credentials are exposed inside systemd services, and the way NixOS secret tools such as agenix or sops-nix work.
\nSometimes, this approach is supported natively by the daemon itself. A common case is PEM-encoded private keys, used for TLS (Transport Layer Security), which are generally awkward to handle inline in configuration files, since multiline strings are frustrating to input with many config file formats. Some software goes further, and allows any configuration option to be specified as the path to a file, where the contents of the file are what the option will be set to.
\nTo handle this approach, secret files have to be written out to some predictable location, and assigned permissions that allow the daemon to read them. Secret management tools generally provide a way to write a secret out to a file, or to standard output, which makes separate secret files a good baseline interface. systemd prefers this approach, and additionally provides a way to specify the path to a secret in an environment variable; it considers this safer than storing the actual secret in the variable, as the secret files are not world-readable.
\nWhen starting a greenfield project, especially one without any aspirations for high complexity, it is often easy to drop in an existing config-parsing library that uses a common file format, and call it a day. While using a common file format makes it easier to generate the config file with external tools (such as NixOS modules), the lack of specific support for secrets makes handling the config file potentially more difficult.
\nFortunately, there are often Twelve-Factor–themed libraries which make it easy to add support for overlaying of configuration via environment variables. This provides a simple option for specifying secrets in a more out-of-band manner.
\nReading each secret from an individual file is the preferred method for some systems, and so that is also a nice thing to support. This tends to be less readily available in config libraries, although it is not unheard of.
\nThere are tool-specific approaches that can be specifically supported, but these simpler, generic interfaces form a good baseline.
\nManaging secrets needed by daemons running under NixOS can be tricky. The Nix store is world-readable (any process on the machine can read it), and so it is not a good idea to write config files or scripts to it, if they include secrets. Fortunately, solutions like sops-nix and agenix exist. The idea behind both of those tools is to have the secrets stored in an encrypted format, with the key outside of the store. At system activation, the secrets are decrypted, and each is placed in its own file with a predictable path, inside a ramfs mount. Services that need the secrets can read them from that path.
\nThis is all very nice, but things get more complicated once we add NixOS containers into the mix. NixOS containers share the Nix store with the host system, but the rest of their directory tree is (by default) isolated, so they do not see the decrypted secrets under the host's /run
. Getting secrets from the host into the container by copying or mounting is non-trivial, and giving the container its own key means having to maintain a separate set of encrypted secrets, specifically for that container. An alternative exists: systemd credentials.
The systemd approach to handling secrets—which, in systemd land are generally referred to as credentials—is similar to the approach that sops-nix and agenix take. Short, sensitive pieces of data are made available under a filesystem path, while only existing in ephemeral storage.
\nIn a systemd unit definition, the LoadCredential
directive can be used to—predictably—load a credential. Each credential will be placed in a file, and the path to the directory with these files will be passed to the unit's processes using the CREDENTIALS_DIRECTORY
environment variable. systemd will ensure that the credentials are only readable by the unit's user.
While the systemd's credentials facility may seem a bit redundant with agenix and sops-nix, the added benefit here is the automated handling of permissions on the credential files. sops-nix and agenix can both set the owners of the secrets they provision, but this has to be kept in sync with the service's user, and becomes more of a problem if the service uses dynamic users. On the other hand, if sops-nix or agenix leave the secrets owned as root, systemd's LoadCredential
can still load them, and ensure the service's processes can read them.
As an example of this pattern, assuming the configuration has sops-nix properly set up, could look something like this:
\n\nLet's assume we've set the greetings_target
secret to "Earth". We can check that it worked:
Loading credentials into systemd units has some uses, but how does it help with NixOS containers? Turns out, systemd's credentials concept extends not only to loading credentials into systemd services, but also to loading credentials into the system itself.
\nWhen running a virtual machine or a container, the host machine can pass any number of credentials to the init system inside the guest, at the time of the latter's startup. The guest init system can subsequently pass some of those credentials on to the services it launches.
\nsystemd supports a number of methods of passing credentials into the guest system. One is SMBIOS OEM strings, which QEMU can set, and which systemd will read at startup, looking for ones starting with io.systemd.credential
. Another method—more useful for containers—is similar to that used inside systemd services: give the PID 1 init process an environment variable called CREDENTIALS_DIRECTORY
, containing the path to a directory containing credential files.
systemd-nspawn
uses the latter method internally. It will set up a ramfs mount inside the container's namespace, write out the secrets to it, and set CREDENTIALS_DIRECTORY
to its path. It can be told to do this by using the --load-credential
option.
systemd-nspawn
is also what NixOS containers are based on. Although, as of writing, there is no NixOS option for passing credentials into NixOS containers, the freeform extraFlags
can be used to just pass the relevant --load-credential
argument to systemd-nspawn
:
Since systemd-nspawn
containers implement systemd's container interface, we can peek at the container's journals using the --machine
flag:
Use of systemd credentials is not without its problems.
\nUnder both agenix and sops-nix, secrets have predictable paths, and those paths are available at NixOS configuration build time. For a systemd service, the path is provided dynamically. The path is passed in by systemd using an environment variable, and while in practice it is technically predictable, this is not documented or supported, so ideally, we do not want to guess the path ahead of time.
\nAccording to systemd, the ideal situation would be for the daemon itself to understand the CREDENTIALS_DIRECTORY
environment variable, and go looking for the relevant secrets in there. More likely, some daemons might support being passed paths to a particular secret via a specific environment variable, in which case those variables can be made to point at the relevant credentials, using the %d
specifier (as in the hello-sayer
examples).
For software where these solutions do not work, we have to resort to other tricks. Fortunately, the common NixOS pattern of replacing strings in config files before service start is easy to adapt for systemd credentials. With systemd 252 or later, the credentials directory is available in ExecStartPre
, which is where we usually end up doing credential replacement:
Another way secrets are often handled in NixOS modules is through environment variables. systemd has no provisions for setting an environment variable to the content of a credential, since systemd does not want us doing that (which is a story for another day), but system-level credentials are under a predictable path: /run/credentials/@system
. We can commit systemd crimes by passing in an entire environment file as a credential, and then loading it in the unit:
A number of modules in NixOS already do use LoadCredential
, on options that specify files with secrets. The advantage to doing it this way is that the permissions on the secrets paths do not need to be adjusted, as systemd will take care of presenting the credential to the service with the correct permissions. This is particularly advantageous when passing credentials into containers using --load-credential
, as those end up with root-only permissions. Where LoadCredential
is not used, we might have to add some extra scripts to copy system-level credentials into files readable by the target service, before service startup; this depends on how a particular NixOS module handles secrets, though.
As previously established, SVGs are nice. Being a vector format, they display well at varied sizes. Being a vector format, they also have to be rasterized by a renderer before display, which introduces an opportunity for rendering inconsistencies between different platforms.
\nOne possible source of such rendering inconsistencies is fonts. Just like with HTML, the basic use of a font requires that the font be available to the rendering program locally; if the font is not installed, the renderer may fall back to a different font. This is particularly undesirable if, for example, a piece of text is supposed to fit within a box: if the dimensions of the text change, it could possibly end up going over the box's borders. Therefore, if we want such SVGs to always look correct, we have to figure out how to work around this limitation.
\nFont glyphs (at least in a TrueType font) are defined as contours consisting of curves and lines. SVGs can also do paths that consist of curves and lines. It follows that text could be turned into SVG paths.
\nProperly rendering Unicode text can be difficult, so we could ask an existing font rendering library to typeset our text properly, give it to us in a vector format, and then turn that entire bit of text into paths. This is how several tools do it, including Inkscape, which can turn text into paths via the Object ▸ Object to Path command.
\nThe disadvantage of converting the entire text object into paths in this way is that each glyph rendered will be a separate path. Even when a character occurs within the text multiple times, each occurrence will be stored as a separate path in the SVG file. How much of a problem this is depends on how much text there is, so it can get annoying if there is a lot of it. In theory, it should be possible to list individual glyphs in <defs>
of an SVG image, and then <use>
them repeatedly, but the difficulty with that is the need to hook into a different point of the text rendering path.
There are other downsides to this approach: when rendered as a document, text in an SVG is selectable, but text-rendered-as-paths is not. Inkscape inserts the original text into an aria-label
, so at least accessibility tools can still read it.
Conversion of text to SVG paths is also lossy. On displays without a high pixel density (we have to assume not everyone is using fancy new high density displays, at least not all the time), font rendering can involve application of hints, that the font file provides for aligning the glyphs to the pixel grid. The font rendering system can also do things like subpixel rendering. As rendering SVG paths usually does not involve such steps, text-as-text and text-as-paths can end up rendered differently:
\n\nNevertheless, this approach is expedient, and works fairy well for, say, text that is part of a logo-like design. But, perhaps we can find a different solution...
\nCSS-as-used-in-HTML supports the @font-face
at-rule, which can be used to tell the browser where to download a font. The font can then be used as if it was installed locally—the browser will download it when it needs to, and use it to render text in the document. A rather minimal example looks something like this:
@font-face
is also available with CSS-as-used-in-SVG, but there is an obstacle to using it when showing SVGs in the browser: when used as images, SVGs cannot download external resources.
There are several ways to use SVGs with HTML documents, like <img>
elements, <object>
elements, inlining, or use as CSS images. The uses fall into two broad categories: one where the SVG is treated like an image, and one where the SVG is treated like an XML document. When an SVG is included via say, an <object>
element, it is treated like an XML document, and so it can do stuff like download external resources or run embedded JavaScript. On the other hand, when an SVG is included via an <img>
element, or used for background via CSS, it is treated like an image, and restricted from accessing external resources, running scripts, or providing interactivity—it acts like a raster image would. As such, use of external resources in @font-face
limits how we can use our SVGs.
As an aside, these limitations mean that viewing an untrusted SVG included via an <img>
tag—such as on a website that allows user-uploaded media—will not result in immediate disaster. When it is viewed this way, scripts inside the SVG will not run at all. However, SVGs viewed in a tab by themselves do run as XML documents. With the user-uploaded media example, an attacker could thus link to the SVG file directly in order to get someone to run malicious scripts in the context of the domain where the SVG was uploaded. Furthermore, the browser may expose an Open Image in New Tab option in the right click menu for an SVG in <img>
, which provides a way for the user to wander into running the SVG's scripts.
The CSS url()
function can accept data URLs. As a data URL does not require accessing an external resource, we can use it to include a font in a way that will work even when the SVG is included via an <img>
element. As a bonus, we end up with an SVG that is self-contained.
In order to put our font file in a data:
URL, we have to encode it with base64, which introduces 33% of overhead when it comes to file size. Most modern browsers support WOFF2—a compressed font format designed specifically for transferring fonts to browsers—so we can offset the space requirement somewhat. It would look something like this:
The problem with embedding a whole font in our SVG file is that we are embedding the whole font. The font file may include glyphs which are not going to be used in rendering the image at all. If the text we are including only ever makes use of a subset of all the available glyphs in the font, we could drop all the unused glyphs, to save on space. This technique is commonly used with PDFs (which can also embed fonts), but we can also bring it to SVGs.
\nThe fontTools package contains a subset
module, which can be used for producing subsets of fonts. Properly producing a font subset can be tricky, as we have to make sure we do not omit any of the data required to render the desired text (which can include more than just the obvious glyphs), but fortunately the subsetting tool's default settings work well here. fontTools can also output a compressed WOFF2 file, although this requires fontTools with the woff
extra. It works like this:
Extracting the text from the SVG image can be the tricky part. It is easy enough to specify "beep boop" on the command line, if that's all our image contains. pyftsubset
can also read the text from a plain text file, but we have to actually extract the text from the SVG first. As a quick hack, we could just copy and paste the text into a file, and feed that to pyftsubset
. More properly, the solution would involve walking the XML tree, finding the text elements, copying their contents, and dumping that out. This process has been automated, with tools like svgoptim, which I have not, however, successfully tested (and the author of which has a blog post much like this one, but better). In the end, the result is something like this:
For short headline bits of text, or text incorporated into designs like logos, turning it into SVG paths is generally an expedient way to get decent results. For things like multiple longer diagram labels, or stretches of body text, embedding a subsetted WOFF2 file offers a portable SVG minimized filesize requirements, which is usable in <img>
elements. There is also the option of not using any of these techniques, if the design can accept the possibility of substitute fonts being used, and that possibility is not objectionable.
The Fediverse has been around for a while. Exactly how long depends on what one counts as the Fediverse, but ActivityPub—the protocol now used by perhaps the best-known Fediverse software, Mastodon—is over five years old.
\nThe Fediverse has historically mainly involved non-commercial entities. Indeed, for many of its users, its non-commercial nature is a major part of the appeal. Recently, though, corporations including the likes of Facebook have started expressing more interest in ActivityPub. Twitter's recently accelerated deterioration under Elon Musk is likely to blame for this, as the Fediverse (or rather, Mastodon) has been more widely covered as an alternative to the ailing social network. If Twitter does collapse—or turn entirely into just another Internet pit of fascism—then a vacuum will emerge. Moving into that vacuum is of interest to the other big corporate players, and the possibility that ActivityPub will be relevant for doing so explains the attention that it is now getting.
\nParts of the Fediverse have expressed concern about the possible entry of corporate interests into the space. These parts posit that introduction of corporate, profit-driven entities is generally not desirable in spaces that are managed by community, and for the community. A concept that gets brought up in these discussions, and the thing that people fear will happen if corporate encroachment is not resisted, is embrace, extend, extinguish, or, with more penchant for the dramatic, embrace, extend, exterminate (EEE for short). But, what is it, and how likely is it to happen?
\nIn the ancient days of 1990s, before the inception of Google or Facebook, tech's prototypical giant evil corporation was Microsoft. Like today, Windows was dominant on the desktop, and unlike today, desktop was how most end-users did computers.
\nMicrosoft of that era engaged in many practices aimed at locking users into its products. An example is Microsoft's approach to Java. At the time, Java was a hyped new technology, as it promised a platform which could be used for distributing software (applets) over the Internet, while enabling developers to target the Java Virtual Machine, instead of the underlying operating system. Microsoft, indeed, embraced Java—there was even a Microsoft Java Virtual Machine.
\nMicrosoft also had concerns. Code written for the Java VM was not tied to Windows, and the Java VM could run on multiple operating systems. This meant that users were now less locked into Windows. To address this, Microsoft extended their Java VM with an API that allowed calling into the Windows API from Java. The standard Java platform contained abstractions over the operating system's API, which meant that code written for the standard Java platform was portable. The Microsoft extensions worked only with Windows, so anyone who wrote Java code that used the extensions was writing code that, despite being Java, would only work on Microsoft's operating systems. Sun Microsystems—creators of Java and holders of Java trademarks and copyrights—sued Microsoft over this, and eventually reached a settlement. Ultimately, Microsoft discontinued their own version of the Java VM.
\nIn the 1990s, the World Wide Web was an emergent technology that everyone—including Microsoft—wanted to get in on. Bill Gates, who at that point still took an active role in setting the direction of Microsoft, publicly declared the intention to "embrace and extend" the related standards. Microsoft's actions in the area, however, were a source of even more legal issues for the corporation, this time from American antitrust authorities. Towards the end of the 1990s and in the early 2000s, Microsoft was subject to protracted legal proceedings, brought by the United States Department of Justice.
\nIt was during one phase of this United States v. Microsoft Corp trial that an Intel executive, Steven McGeady, was put on the witness stand. In his testimony, he recalled a 1995 meeting between Intel and Microsoft, on the two companies' involvement in the development of the Internet. According to McGeady, during that meeting, Paul Maritz—a Microsoft Executive—said that it was Microsoft's strategy to "embrace, extend, extinguish" Internet standards.
\nSoon, embrace, extend, extinguish became a popular, pithy descriptor of Microsoft's strategy, and a way to criticize it. Microsoft, indeed, embraced web standards, extended them with proprietary additions, and attempted to extinguish any competition. Evidence submitted during the antitrust trial included, for example, late 1990s memos, in which Bill Gates discussed the fact that browser-based viewers for Microsoft Office documents worked in non-Microsoft web browsers. Gates believed that this hurt the position of Windows on the market. "We have to stop putting any effort into this and make sure that Office documents very well depends on PROPRIETARY IE capabilities", he urged.
\nIt took until the mid-2000s for the dominance of Internet Explorer to wane. New browsers—like Firefox—offered a better user experience in many areas, which meant that many users preferred them to IE. Internet Explorer also continued with its own idiosyncrasies when it came to adhering to web standards. Web developers could either produce websites which worked in every other browser while being broken in IE, or do extra work to also make them work in IE. The fatigue with this situation, combined with later entry of Google into the browser market with Chrome, meant that Microsoft was no longer able to deploy its EEE ways to the extent that it could before.
\nBefore that, EEE did work, at least for a while. In 2023, Microsoft Windows still remains the most popular operating system for desktop and laptop computers. Microsoft's early efforts at locking their position in the market are likely a factor here, even if modern Microsoft's EEE efforts do not have the same overtness and intensity as they did during the 1990s.
\nAnd, indeed, Microsoft of today likes to present itself as a company more interested in embracing open standards without the old Microsoft's ulterior motives. It is still a corporation, and like all corporations it does not do things out of the kindness of its corporate heart. People have, however, grown more wise to the EEE ways over the intervening decades, and so blatant attempts at locking them in are more likely to be rejected, in favor of more open platforms. Lock-in requires more subtlety now.
\nIn 2004, Google announced that they would start offering an email service. The available storage for email would be 1 GB, which was an impressive amount at the time. Per-gigabyte cost of storage was in the single digits of US dollars at the time (in 2023, hard drive storage per-gigabyte cost is under 0.10 USD). Seeing gigabytes of stuff was not unfathomable to the people of 2004, but free email services tended to offer inboxes of maybe a dozen megabytes. On those services, it was expected old email would be regularly deleted to make room for new incoming email. Gmail, on the other hand, promised you would not have to delete your emails again.
\nGmail also offered a spiffy Web client for reading and sending email. By 2004, webmail had been a thing for a while, but Gmail did webmail in a way that more resembled a modern web app of later years. JavaScript-based interfaces, which could load new data without reloading the whole page, were still a fairly new trend. Gmail's web app was a cool new thing, compared to other free webmail offerings. Google also offered other perks, like free POP access (which is an older standard for accessing email from a desktop email client), at a time when other free email providers used a strategy of limiting free features to get people to subscribe to an expanded, paid offering.
\nImmediately after it was first announced, Gmail became the hot new thing. During the initial rollout, when Gmail was invite-only, people were actually willing to pay money for an invite (which is the type of hype some startups still try to recreate in 2023). Throughout the 19 years since Gmail's start, its free offering remained ahead of those of other free providers. For a lot of people in need of a free email account, Gmail has thus established itself as the default option. It may be the most popular email service overall, although the exact numbers are hard to figure out, since we cannot see all the email use out there.
\nThe dominance of Gmail does come with some downsides. As email spam remains a persistent problem, Gmail offers spam filtering. This means that they reject some of the emails that are sent to their servers, often according to criteria that are opaque to the senders. When your own small email server gets consistently rejected by another, small server, it is a small problem; when your server gets rejected by Gmail, it is a comparatively larger problem. Being rejected by Gmail's servers for unclear reasons, despite not originating any spam traffic is something that indeed happens to people trying to host their own email. Being denied the ability to talk any Gmail user means being denied the ability to talk to quite a lot of people. Gmail, thus, has an outsize influence over who can manage to effectively self-host email, and indirectly controls the market of other service providers.
\nThe Extensible Messaging and Presence Protocol (XMPP) is an instant messaging protocol which first emerged (under the name Jabber) in the late 1990s. Instant messaging (IM) software already existed at the time, even if modern smartphones did not. IM programs were generally used by someone at a desktop or laptop computer to chat, via text, in real time, with other people on their desktop or laptop computers. Presence is part of the protocol's name, because a user would have to know who else is also online at the same time, and thus available for chatting.
\nXMPP was an alternative to other, proprietary services. ICQ, Microsoft's MSN Messenger, and AOL's AIM were some of the proprietary IM services enjoyed mainstream popularity through the 2000s. XMPP was, however, the decentralized, open source alternative to the proprietary IM services that put their traffic through a corporation's servers. With XMPP, anyone could host a server, and XMPP servers could talk to each other using a standard protocol, allowing users on one server to send instant messages to users on other XMPP servers. XMPP could be also be used without federation, making it useful as basis for things like internal communication tools. It is easier to use an existing protocol, with available software, than to invent something from scratch.
\nIndeed, Google chose XMPP as the backing protocol for their Google Talk software. Google Talk first appeared in mid-2005, and although it used XMPP, initially Google Talk users could only chat with other Google Talk users. Google turned open federation on in early 2006, enabling users on other XMPP servers and Talk users to communicate with each other.
\nGoogle Talk incorporated a bunch of Google-specific extensions to XMPP (which is, after all, extensible). Standard XMPP clients could be used with Google Talk, but the added features would not always work. Some extensions, though, were eventually standardized: Jingle, for example, started as a Talk extension used for establishing voice calls, but eventually had a standard specification published, and is now part of mainline XMPP clients.
\nAs is often the case with Google, Talk was eventually discontinued. In the early 2010s, Google decided to move away from Talk and to Google Hangouts (which was not based XMPP). To that end, they stopped federating their XMPP servers, discontinued the various desktop and mobile apps, and removed Talk widgets from Google web apps. The servers themselves remained up, and reachable through third-party XMPP clients all the way up until 2022. By 2022, Google was actually moving away from Hangouts, and to other messaging apps.
\nOne can, of course, still use XMPP today. It remains an open standard. The XMPP Standards Foundation still maintains extension standards (called XEPs). There still is maintained server and client software, that runs on modern platforms. Like with email, the user counts are unclear, since there is not a central authority with an overview of all XMPP users, but there are at least some users out there.
\nMicrosoft's strategy in the 1990s and early 2000s was clear: if something is open, add proprietary bits to ensure vendor lock-in, and thus maintain ongoing monopoly and market domination. Was Google's strategy the same? Did Google seek to exterminate email? Did they purposefully kill XMPP?
\nOne interpretation is that over the 2000s, corporations moved away from embrace, extend, extinguish strategies. After all, EEE did get Microsoft into frequent legal trouble. People became aware of EEE, and rather than getting locked into a piece of software, would seek out alternative solutions that adhere to standards. Market domination is still possible under these conditions, but requires different approaches. Google, for example, didn't need to lock people into their email offering with proprietary extensions, because email addresses are already a form of lock-in by themselves—having to tell everyone you've switched your email address is annoying. Google's size means it can offer more for free as a loss-leader, and achieve domination that way.
\nAnother interpretation is that corporations have gotten sneakier. You cannot pull a Microsoft anymore—all the coverage of Microsoft's EEE practices in the past means that people know what that looks like. Instead, you have to appear to embrace open standards without ulterior motives, while slipping in subtle incompatibilities. Large corporations with large user bases can dictate how a standard goes—if they move in one direction, everyone has to follow, or be left behind. When Google came out with Jingle, everyone had to get on board; Google, on the other hand, did not have to get on board with anyone else's extensions if it did not want to. Such dominance grants control, which can be used to extinguish a standard, without having to go for blatant lock-in.
\nDoes it ultimately matter, though? One could argue that Google killed XMPP by endorsing it at first and then pulling out of the space, taking all the users with it. One could also argue that XMPP was an obscure chat protocol for a bunch nerds, designed for an older mode of communication, that saw several million users come in and then leave, putting it back where it started. One could argue whether Google intentionally killed XMPP in order to eliminate open competition, and push for its own IM platforms, or if Google simply boldly steered its bulk into the space, without caring who gets caught in the wake. But, does it matter? The results are what they are.
\nWhen rumors and reports of the possible involvement of Facebook (allegedly properly known as Meta) in the Fediverse began to circulate on the Fediverse, one of the concerns brought up was that the corporation's intentions were underhanded. The fear was that Facebook wanted to EEE the Fediverse.
\nIn the end, though, it does not really matter if a corporation like Meta enters the Fediverse with intentions that are actively malicious, or not.
\nOne might be tempted by the prospects that corporate involvement is going to lead to more work being done on the protocol and associated software, for the benefit of all. For those who care about there being more people on the Fediverse, welcoming a large corporation also sounds like a good way to bring in more users.
\nCorporations are, however, interested in mutually beneficial arrangements only because of the side of the arrangement that is beneficial for the corporation. When a corporation promises benefits to the community in exchange for the community letting it in, those benefits are only side effects that the corporation is willing to put forward as incentives. Profit is the primary goal of the corporation, and anything it does for the community is in pursuit of that.
\nImmediate, outright blocking of any attempt by Facebook to enter the Fediverse may seem like an excessive reaction; it is, however, an understandable one. Rejecting corporate entry into the space signals that, at least in that part of the Fediverse, corporate interests are not welcome, and that community is valued more than increased growth or mainstream acceptance. Principles of open federation that require unconditionally admitting all corporate actors may seem ideologically desirable, on some abstract level. On a slightly less abstract level, they potentially require admitting an actor that is not going to act in good faith, and will be harmful to the community—a thing that is, from an ideological perspective, undesirable.
\nPerhaps Meta would not be able to kill the Fediverse anyway. Large portions of the Fediverse, after all, were made by, and are populated by people who wanted to get away from corporate social media. The Fediverse is evidence that moving away from corporate social media is possible. Perhaps Facebook, when allowed to run free, would at most capture a portion of the Fediverse, find the lack of control limiting, find the opportunities for profit lacking, and leave by itself. Rejecting Meta outright from the start saves the Fediverse all that trouble, though.
\nWebsites about excluding Facebook/Meta from the Fediverse:
\n\nWhen doing diagrams of various sorts for blog posts, I like to use the SVG format. Vector images work well for diagrams, since diagrams involve simple shapes, lines, and text, and those are all things that vector graphics are good at. The SVG (Scalable Vector Graphics) format is—at least at the basic level—widely supported by modern browsers, making it a good choice.
\nInternally SVGs are XML documents, and, when viewed in a Web browser, there are ways in which they behave similarly to HTML documents. For one, SVGs support CSS—not in the same exact way as HTML, since SVG elements and HTML elements are not the same—but in ways that are often similar. One feature of CSS-as-used-in-HTML that CSS-as-used-in-SVG supports is prefers-color-scheme
media queries. This means that SVGs can adjust to dark and light schemes with CSS alone. This actually works—at least in Firefox and Chromium—even when the SVG is included in a document via an <img>
tag (which otherwise limits what SVGs can do). An example SVG image that makes use of this follows:
Diagrams tend to feature elements with consistent styling: boxes with the same kind of border, text labels with the same font, arrows with the same line thickness. In this situation—just like with HTML—it makes sense to give SVG elements semantic class names, or at least names that reflect reusable styles, and then write CSS rules based on those classes. A (non-diagram) example of this:
\n\nAuthoring SVGs entirely by hand is, however, a tricky proposition. There are tools like Mermaid which will, indeed, output SVGs that use classes and separate stylesheets for easier restyling. However, I like to author my SVGs with Inkscape, as it is a full vector image editor, and so it offers more flexibility than tools designed for authoring specific kinds of diagrams.
\nInkscape is a What You See Is What You Get editor. Because of this, it does not produce output where objects have consistent classes and where stylesheets are easy to apply. Indeed, it is is not really designed with editing the XML source of an SVG as part of the most common workflow.
\nNevertheless, Inkscape actually does feature an XML source editor. The XML editor panel is available under the Edit ▸ XML Editor… menu entry.
\n\nInkscape also has a newer CSS editor, which can be accessed via Objects ▸ Selectors and CSS…. This editor somewhat resembles the element inspector you might find in a web browser, and actually allows assigning classes to elements, as well as adding CSS rules based on class selectors.
\n\nIt is also possible to edit an SVG simultaneously in Inkscape and an external text editor. Inkscape will do mostly fine with rendering externally modified files (provided they're valid SVG), and will pass-through copy such modifications without rewriting them, unless a particular element is modified within Inkscape. It will not live-reload externally modified files—this is something the user has to remember to do themself.
\nBy default, Inkscape will not assign classes to new elements. It will, on the other hand, enthusiastically assign styles on element level. When I am drawing a diagram, I generally do not care about setting these element-level styles, other than temporarily, in order to preview how I want things to look in the end. Copying and pasting styles between elements is what I am trying to avoid, though.
\nI like to use Inkscape's alternate display modes. Through View ▸ Display Mode, Inkscape can be set to display all elements as outlines, or to display elements as normal, but add extra outlines to them. This is useful for laying things out, even if their current styles make it less obvious where things are, and where they end.
\n\nOnce an element is mostly where it needs to be, I use the XML editor to give it a class or an id. Just like with HTML elements, SVG elements can have both any number of classes, as well as a unique id. Inkscape does automatically assign everything ids, in a pattern like path1234
; the id can be edited later from the XML editor, or the Object Properties dialog. Note that Inkscape also has its own labels, stored under the inkscape:label
attribute—this is the name that objects show up under in the layer editor, but it is distinct from the id
attribute.
When elements have some classes assigned, I open the SVG file in a text editor, and start adding a light-theme stylesheet under a <stye>
element (directly under the <svg>
element). As with ordinary HTML, the default prefers-color-scheme
is assumed to be light
, so starting with a light theme by default, and then adding a dark one is an approach that makes sense.
At this point, I generally also open the SVG in a web browser. In theory, SVGs should render the same everywhere, but checking is still a good idea. An SVG viewed in a browser can also be easily refreshed, to show the most recent changes.
\nInkscape, on the other hand, can be forced to reload the SVG via File ▸ Revert. I try to save before moving out of Inkscape, and revert when moving back into it, so that I do not accidentally discard useful changes made in one program by saving over it in another (though text editors are better about handling changes made elsewhere).
\nOnce I have something in a dark-on-light scheme that looks okay, I start writing the light-on-dark version. I generally enclose the whole set of dark-themed styles behind a @media (prefers-color-scheme: dark)
query, though it is possible to intersperse such overrides at multiple points in the sheet. The web browser preview becomes useful at this point, as Inkscape will not read the styles behind the media query. As I normally use a light-on-dark desktop theme (and so my Firefox prefers a dark
color scheme), I get a preview of what is behind the media query. Firefox's Element Inspector has handy buttons for forcing prefers-color-scheme
to dark
or light
, which is useful for making sure that everything in the light theme still looks okay afterwards. The Element Inspector can actually also inspect SVG elements in the same way as HTML elements, including modifying their styles.
Ordinary Inkscape SVGs leave a bunch of Inkscape-specific stuff in the file, useful for editing, but less useful for display. Inkscape also tends to leave unused things in the file—for example, gradients which were applied to an object that was later deleted. This is fine as far as rendering in browser goes, but ideally, the SVG served over the web should not contain a bunch of stuff that will never be used. To that end, Inkscape has the option of saving files as "Plain SVG" (as opposed to "Inkscape SVG"), as well as the option to clean up unused stuff (under File ▸ Clean Up Document). I like to process Inkscape SVGs with SVGO, a separate external tool for optimizing SVGs, which also gets rid of redundant or unnecessary things. Inkscape does have a built-in optimizing functionality—when saving as "Optimized SVG", Scour is applied to the file—so the use of SVGO here is a preference.
\nSVGs with media queries in them do not seem very common in the wild. There do exist other ways of adapting images to dark and light color schemes. A common, idiomatic one under HTML is to use <picture>
with <source>
s conditional on a media query:
Although this approach requires two separate SVG files—which presumably duplicate geometry, while changing the style—a browser will normally not download any of the sources besides the selected one.
\nFancier solutions often involve using JavaScript to query the current prefers-color-scheme
, and then dynamically loading content based on that. The downside to this is that it requires the user agent to consent to running JavaScript, or fail to load the color scheme–specific content. If possible, it is generally better to use the above methods, which do not require running scripts.
When you run your own website—particularly one that includes a blog—you get the privilege of writing, on that website, about building the website. You even get to write about how much of a stereotype it is for a person who built their own personal website to post about how they've built their own personal website. Having now done the latter (if perfunctorily), I shall proceed to do the former.
\nI decided to switch the static website generator that I use from Hugo to Eleventy. I have been using Hugo for some time—I wrote a post about switching to it, back in February 2020—and it is an entirely fine static site generator. Hugo has been around for a while, seems to be fairly popular, is stable, performant, and generally does its job. However, it seemed like certain things that I found tricky to do with Hugo, and some things I potentially might want to do in the future might be easier with Eleventy, and so I decided to switch.
\nEleventy, also spelled 11ty is a static site generator written in JavaScript. Eleventy describes itself as a "simpler static site generator". In its overall design it is, indeed, simpler than some of the fancier static site generators out there. The simplicity does not mean that its functionality is limited, though. Rather, Eleventy offers a simple base which can be extended, and upon which projects of various complexity can be built.
\nBy itself, Eleventy supports maybe eight, nine, or ten template languages (depending on how you are counting). These include Liquid, Nunjucks, WebC (Eleventy's own thing), and EJS. In addition, Eleventy makes it possible to add custom languages. While custom languages will not be as well-integrated with Eleventy as the ones it ships with, the custom language facility is still useful. For example, while Eleventy does not support Sass natively, adding support can be accomplished by pulling in the Sass library (from NPM), and telling Eleventy to process any .sass
or .scss
files with it. This takes less than a dozen lines of JavaScript.
New template engines are not the only thing that can be added in Eleventy. For example, by default, Eleventy can load data from JSON files, and the fields from those files can then be used in templates. However, support for other formats can also be added, by passing in a function that takes the input file and outputs a JavaScript object, so support can be added for formats like YAML, or even KDL if feeling fanciful. Another example is the Markdown library which Eleventy uses: the library can, by itself, use plugins, and Eleventy has a way of using an instance of the library with arbitrary plugins added. Scripting can be used in a lot of places—templates themselves can be JavaScript files, for example.
\nThe project also brags about its performance. While Hugo is faster than Eleventy, Eleventy's own benchmarks indicate that it tends to be faster than a bunch of other popular JavaScript static site generators.
\nI decided to port the website over to Eleventy without making any other large changes in the process. The idea was to get to a working state first, so that I can then make further changes, but also can publish new posts, without having to maintain a fork of the old version at the same time.
\nBecause I like Nix too much, I also wanted to both have a way of building the website as a Nix derivation, and also to have a Nix shell capable of running a live preview server. With the previous site, I had direnv configured to load Hugo via a Nix flake, which meant that I could clone the repository, rundirenv allow
in it, and be ready to go.
While Eleventy is not in Nixpkgs itself, it is easy to install with Yarn, and mkYarnModules
can be used to produce the required node_modules
without overrides. Eleventy itself can be pointed at that node_modules
, by using the NODE_PATH
environment variable, which works for both putting a working Eleventy in PATH
, and building the website as a derivation.
The actual blog content was not particularly difficult to port. Both Hugo and Eleventy support Markdown, and that is what my articles are all written in. I also use custom shortcodes, but those were easy enough to rewrite for Eleventy. Hugo uses Go templates, while Eleventy, by default, pre-processes Markdown as Liquid templates. This means that the syntax for invoking shortcodes is slightly different, but in practice they are similar enough that a simple search-and-replace is largely sufficient.
\nThe non-blog portions of my website involved some more free-form rewriting. Eleventy is less rigid than Hugo when it comes to the directory structure of templates and layouts. This is handy for the less frequently edited portions of a website, as they can be kept more simple. Eleventy is also fairly easy to debug—there is plenty of opportunity to console.log()
the state (pretty-printed) at various points, even inside templates.
The downside of the simplicity is that some things are harder to accomplish. For example, under Hugo, a page can be a directory with an index.md
, and other, arbitrary files. Hugo will copy these other files as attachments, respecting the path settings in index.md
's front matter. Eleventy, on the other hand, does not really have the concept of a page and its adjacent files in the same way. Fortunately, the extensibility of Eleventy means that such functionality can be bolted on, and there, indeed, already are plugins (see Eleventy issue #1540 if interested).
Comparing static site generators is tricky, because because the category of static site generator encompasses a wide variety of tools. Some generators make it really easy to get going with a blog, and automatically handle things categories, tags, or web feeds; other generators come with less blog-specific functionality, but are more suited for a wide variety of different kinds of websites. Some generators produce traditional, static HTML sites, while others are meant for web apps.
\nPrior to Hugo, I used Lektor. Lektor is less rigid than Hugo. Instead of making assumptions about how input data should be structured, it requires explicit declaration of the structure. I left Lektor for Hugo, and Hugo's advantage was the fact that it comes with all the batteries included, and it mostly did everything I needed it to out of the box, where Lektor required plugins or patching. Then, I switched to Eleventy, which handles input data in a less structured way than Lektor, while also relying on extensibility rather than inclusion of batteries.
\nEleventy works well for me, for what I want to achieve, and so I considered the switch worth it. That does not make it a strictly best static site generator, and whether or not it will work for anyone else depends on their individual use case and needs. Even things like speed benchmarks are relative, since speed can matter less for smaller websites (such as my own). On the other hand, Eleventy is easy to play with, and relatively well-documented, so I can recommend looking into it, if it seems like it would fit your requirements.
\nI published a repository, which serves as an example of how an Eleventy website can be built as a Nix package, while also providing a direnv environment for working on it..
\nSince release 22.05 "Quokka", NixOS can be installed via a graphical installer, which makes for an installation experience closer to that of a traditional Linux distribution. NixOS is, however, in many ways not like a traditional Linux distribution.
\nThe manual way of installing NixOS—the main one available before the introduction of the Calamares-based graphical installer, and still available as an option now—generally goes like this: nixos-generate-config
creates some basic configuration files, which the user can then adjust to their needs. nixos-install
then builds the first system configuration based on those files, and sets up the machine so that it will boot into NixOS with that first system configuration.
Like Nixpkgs packages, NixOS system configurations are deterministic and reproducible. Indeed, on the low level, they are the same thing: paths that end up in the Nix store. When nixos-install
builds a new NixOS system configuration, it is no different from when, on an existing NixOS system, nixos-rebuild switch
builds a new system configuration to switch to. These commands do some extra work, however: nixos-install
gets the system bootable, and nixos-rebuild
performs the generation switch. As a consequence of this, if we understand all the extra stuff that nixos-install
does, we can install NixOS in some unconventional ways.
To understand how we can make a NixOS install bootable, it is useful to understand how NixOS actually boots.
\n\nThe very early stages of booting an x86 machine to NixOS work very much like most other Linux distros. After powering on, the firmware will first get us into the bootloader. How it finds the bootloader depends on whether we're booting in the classic PC BIOS way or the UEFI-GPT way. With BIOS, the firmware looks for the bootloader in the first sectors of the configured storage devices. With UEFI, the firmware consults its own non-volatile memory, which lists available paths that can be booted, or otherwise falls back to a spec-mandated default path.
\nWith NixOS, the bootloader that we get into will generally be either GRUB or systemd-boot. In either case, the bootloader has one or more boot entries, representing different NixOS system configurations (sometimes called generations). Each entry will point to a kernel, and an initial ramdisk (initrd) image, both of which are files within the boot partition—that is, not in the Nix store. The initrd contains the Stage 1 init script, as well as a number of executable binaries and kernel modules.
\nThe Stage 1 script's job is mostly to mount whatever filesystems are required for the system to boot. Because this can involve things like LVM or LUKS, the initrd (hopefully) contains all the modules that are required for handling the storage setup present. Also because of this, Stage 1 is when the user is prompted for any LUKS passphrases, if they are required. At its end, the Stage 1 script runs the Stage 2 script (via exec
).
The Stage 2 script is in the freshly mounted Nix store, under the path for the target system configuration (this directory is symlinked from /run/booted-system
on a running NixOS system). The bootloader entry from earlier also contained the store path to the Stage 2 script, and this is how the Stage 1 script knew what to run. The Stage 2 script contains the activation scripts.
The activation script portion of Stage 2 is built from the system.activationScripts
configuration option. Its job is to activate the system configuration. A NixOS system configuration contains in it things like files that should go in/etc
, configuration of user accounts, or sops-nix/agenix secrets. The system configuration, however, lives inside the Nix store, and we want our /etc
files symlinked from, or copied to the top-level /etc
directory, our users reflected in /etc/passwd
, and our secrets provisioned somewhere under /run
. The activation scripts make these things happen.
After the activation scripts are done, the Stage 2 script finally runs systemd (also via exec
). systemd is now running as PID 1, and takes over the rest of the booting. The rest of the boot happens as with any other systemd distro—units get started in the correct order, until systemd reaches the desired target.
If we consider the boot process, it turns out that actually installing NixOS is not really that involved. For example, we mostly do not need to provision anything in the root filesystem (/
) ahead of time, as this gets populated at boot time (a fact which some people use to run NixOS with tmpfs mounted on /
).
The path that we do need is the Nix store under /nix
(which can be a different filesystem than /
). Inside the store we also need to have a system configuration. Since copying a path with its entire closure is part of the core Nix functionality, copying the top level path for a system configuration into the store also copies all the other paths the configuration needs, so this task is relatively simple.
Outside of the store, we need to install the bootloader, and give it what it will need for booting Stage 1. Installing a bootloader is easy enough. Both GRUB and systemd-boot provide scripts for installing them, and these are wrapped in a distro-provided script. Provisioning the initial ramdisk is more complicated. The initrd is derived from the system configuration, but needs to be outside of it, inside the boot partition. To boot the system configuration, its entry needs to be added to the bootloader's config files, and its initrd needs to be placed somewhere where the bootloader can reach it.
\nAnother observation we can confirm here is that installing a new NixOS system, and switching an existing NixOS install to a new generation are actually mostly the same thing. The one thing that may differ is installing the bootloader: when setting up a new generation for an existing NixOS install, we can assume the bootloader is already there; when installing NixOS anew, we need to actually install the bootloader first.
\nLet's actually install NixOS, then. I am going to be installing to a host called sillyvm, which is a virtual machine (though it does not have to be). I am also going to use a second machine, called buildbox. buildbox is where the system configuration will be built, and while it is the VM's host, it could also be elsewhere.
\nLet us start by booting a live USB image on sillyvm. The reasonable thing to do here would be to boot a NixOS live image—either one of the generic ones that Hydra builds (these are the ones available on the NixOS website), or perhaps one we built ourselves, after customizing it with things like our public SSH key. So, let's instead use the Fedora Workstation 37 live image. It is relatively up to date, and has a bunch of tools useful for installing Linux.
\nAfter booting the live image and setting up sshd (which is already installed, and can be started by starting the sshd.service
unit via systemctl
), we can set up the storage. This works the same as with the NixOS live image, and so the instructions in the manual work, while GPT fdisk and fdisk from util-linux are also available. We'll want to boot this install via UEFI, so we'll set up a generous, 512 MiB EFI system partition, followed by a single Linux partition for NixOS, spanning the remaining storage volume. The EFI partition has to be formatted with FAT32, and we are going to make the NixOS root partition Btrfs. Just like with NixOS, we then mount the root partition under /mnt
, and the EFI partition under /mnt/boot
.
With a normal NixOS install, at this stage we might want to run nixos-generate-config
. The conventional approach spits out a template configuration file and a generated hardware-configuration.nix
to /etc/nixos
. The template file is handy for getting started, but not strictly necessary, and we are going to be writing the configuration files on buildbox anyway. What would be handy is the hardware configuration file, since this is hardware-specific (hardware here includes the partitions present). For this use case, nixos-generate-config
has the --show-hardware-config
switch, which outputs the hardware configuration to standard output.
nixos-generate-config
is in the nixos-install-tools
package… but we do not have Nix on the booted Fedora system, so we can't easily nix shell
into that package. We also can't copy it into the system from buildbox, since Nix is not available on sillyvm.
Okay, but if we nix copy
to a bare local path (as opposed to a file:
path), Nix will create a whole new store. What if we just copied that over?
Turns out that this works, for the most part.
\nComing back to buildbox with the hardware-configuration.nix
file we generated on sillyvm, we can now write the rest of the configuration. Channels represent mutable state, so if we opt to use flakes instead, we can make things a bit easier for ourselves.
There are many ways people organize their flakeified NixOS configurations (there is a list of configuration repositories on the NixOS wiki that features various examples), but we'll start with something straightforward. The flake.nix
can look something like this:
We can them write a minimal configuration:
\n\nBy default, nixos-install
prompts for a password for the root user on the newly installed system. If we end up skipping that step, we can end up booting into a system where we have no way to grab root at all. To work around this, we add our user to the group wheel
, which, by default, has sudo
permissions. We also provide a password hash (made with mkpasswd
) and an ssh key that can be used to log into the account. Putting the verbatim password hash here is not the most secure thing (the hash ends up world-readable in the Nix store), and if we were using something like agenix or sops-nix, we could use that to provision the password instead.
Let's look at the generated hardware configuration file we pulled from sillyvm:
\n\nWhile the file starts with an admonition to not edit it, that does not really apply to us. Re-running nixos-generate-config
on a live NixOS system can overwrite /etc/nixos/hardware-configuration.nix
, but we are on a different machine entirely right now, and will not be using or even touching /etc/nixos
at all.
There is also a weird thing that happened with nixpkgs.hostPlatform
. This part is populated by nixos-generate-config
executing nix-instantiate
. As Nix is not properly setup in our live environment (we literally just rsynced a store to it), nix-instantiate
spits out an error message to standard error, which nixos-generate-config
fails to ignore. Arguably, this is a bug, though it only comes up in exotic circumstances, such as our current ones.
Anyway, let's make some changes:
\n\nWe have added some mount options (fun fact: Btrfs mounts with discard=async
under Linux 6.2 by default), but the filesystem settings look fine otherwise. More complicated setups—for example, ones involving LUKS—might need further tweaking here.
More importantly, we've configured the bootloader. boot.loader.systemd-boot.enable
tells NixOS that we want the bootloader to be systemd-boot. systemd-boot only works under UEFI systems, but it's a less complicated option than GRUB. Of importance is also boot.loader.efi.canTouchEfiVariables
, which does what it sounds like—when set to true
, NixOS is allowed to add an entry for the bootloader to the machine's efivars.
Okay, now we can build the system configuration:
\n\nThis may look like a bit of arcane incantation, but we can break it down part by part.
\n.#
tells Nix which flake we want—the flake in the current directory.nixosConfigurations.sillyvm
is the output we declared in flake.nix
.config
is the same config
we use inside configuration files, which is to say it is the final effective configuration with all the options defined; in the same way, we can, for example, do nix eval .#nixosConfigurations.sillyvm.config.networking.hostName
to get the final, effective value for networking.hostName
.system.build.toplevel
is an option inside the config. The definition of an option can depend on the definitions of other options, and system.build.toplevel
is simply a derivation that happens to depend on the definitions of essentially all the other system configuration options. The derivation's output is a system configuration, and it is what we are building.nix build
will have given us a ./result
with the system configuration, but it's on the wrong machine. Fortunately, we can just repeat our silly rsync trick again:
This covers the part about having the system configuration in the Nix store on the target system, for the most part. Now we have to install the bootloader and populate it with the initial ramdisk, and the corresponding bootloader entry. The infrastructure for doing both of those things is actually already in our freshly built system configuration.
\nWe want to invoke the system configuration's script for installing the bootloader… but that script is in the Nix store under /mnt/nix
. If the script refers to an absolute path under /nix/store
, that will point to our live environment's store, not the target system's store. This means we need to chroot ourselves to /mnt
before invoking the install script.
Fortunately, the nixos-install-tools
package we rsync'd to the live environment earlier also comes with the nixos-enter
script, which is essentially chroot
into a NixOS system, along with some handy setup steps, like setting the locale paths or bind-mounting the host system's /dev
and /sys
(handy if we want to modify those EFI variables).nixos-enter
will refuse to enter a system that is not NixOS, but a NixOS system is simply one where /etc/NIXOS
is present, so we can get our nascent install recognized as NixOS easily.
We had to go digging in our new system configuration for Bash, because otherwise nixos-enter
would have trouble finding it, but there we are… in the NixOS install of sillyvm, kind of.
Now we need to do two things: set our new system configuration as the current system generation, and then run the activation script to install and configure the bootloader.
\nUnder NixOS, the system generations are tracked via the /nix/var/nix/profiles/system
profile. This profile does not exist in our store at this point, but we can simply use nix-env
to create it:
Okay, now for the activation script. The NIXOS_INSTALL_BOOTLOADER
environment variable can be set to make the activation script install the bootloader (predictably enough). nixos-enter
actually set /run/current-system
to point to the our system configuration, so we don't have to keep pasting the long path, and can just do this:
Cool. Now we can Ctrl+D out of the chroot, and reboot the live environment (or shut it down and switch the boot device in the VM configuration). Then, if everything went well, we should be able to boot into the NixOS system.
\n\nIs this a practical way to install NixOS? No, not really.
\nIf we wanted to use a downloaded live image, then one of the NixOS live images would have been the best choice for installing NixOS. If that is not available—such as when using a cloud server from a provider who does not offer the ability to boot arbitrary live images, and provides a selection that does not include NixOS—another option would be booting another Linux live image, and installing Nix into it. A number of distributions already have Nix in their repositories, and if that is not available, there are always the Nix install scripts.
\nHaving actual, properly installed Nix in the live environment makes some things easier: instead of rsyncing a Nix store to /mnt
, we could do nix copy --to ssh-ng://sillyvm?remote-store=/mnt
. nixos-install
has a --system
option, which can be pointed at the freshly copied system configuration, meaning we can still build it elsewhere first. The installation process would be similar internally, but the actual tools account for some edge cases, and so are far less likely to break in weird ways, compared to our unconventional methods.
Nevertheless, understanding how the install process works allows us to pull off some unconventional install methods that are actually practical.
\nAs the release of Hogwarts Legacy approached, certain details about the game's plot became public. This has led people to share those details with those who may not necessarily wish to know those details ahead of playing the game; that is to say it led people to post spoilers.
\nInterestingly enough, this isn't the first time that spoiling a Harry Potter thing has become an Internet meme. Another time was over 17 years earlier, in 2005, at the point when the book Harry Potter and the Half-Blood Prince was first released.
\nWhile these two points in time share symmetry, the context around them is quite different, and it is those differences that highlight the arc that the world took over that span of time. 17 years ago, the earliest elements of the modern Internet were just getting their start, and today we can see where they eventually arrived.
\nThe sixth Harry Potter book, Harry Potter and the Half-Blood Prince, was scheduled to release on 16th of July, 2005. By the time the series got to number six, it was already widely popular, and its author—J.K. Rowling—has already made ridiculous amounts of money off it. There were movies, video games, and assorted other stuff that comes with a popular franchise. The book release was highly anticipated. A store in Canada accidentally sold a couple of copies to some fans prior to the official release, and the buyers were subsequently prohibited by court order from even reading their copies. It was one of those releases that cause lines around the block.
\nThe details of the plot of the book became generally known at about release time. There was enough Internet in 2005 for that knowledge to spread around to those who sought it. There was also enough Internet to link unsuspecting fans of Harry Potter to said spoilers, in the manner of a shock site (something that was already a thing by the mid-2000s).
\nIn 2005, Facebook was still restricted to university students, and Twitter did not exist yet. Social media, in fact, was not even a term in common use. 2005 did have ways of shooting digital video, and ways of publishing that video on the Internet—YouTube launched in February of 2005, although earlier and jankier ways were available prior to that. This is how we can still watch a blurry, low-resolution, and low-framerate video of someone rolling in a car past a bunch of Harry Potter fans and yelling out "Snape kills Dumbledore".
\nWhat was also around in 2005 is 4chan—it has been around since 2003, and so predates Facebook, Twitter, and YouTube. 4chan did take notice of the The Half-Blood Prince spoilers. Two years later, Harry Potter and the Deathly Hallows leaked despite the publisher's likewise extensive efforts to prevent that. 4chan users took it upon themselves to spoil the book to fans queuing up in physical spaces, in imitation of the driver from 2005. This is why if, today, you look for videos of Harry Potter being spoiled, you are more likely to find videos related to The Deathly Hallows, including some featuring white British lads in suits and afro wigs yelling spoilers over megaphones, a weird artifact of contemporary Internet culture.
\nHogwarts Legacy is the first major video game set in the Harry Potter universe to release in years. As one of those high-budget video games long in development, broad in scope, and high in marketing spending, it has been eagerly anticipated by those fans of the series that still remain fans.
\nBy 2023, J.K. Rowling has revealed herself to be an odious transphobe, and became a prominent voice in the so-called debate on whether transgender people should have rights—on the no side. While Rowling had no involvement in writing for the game, she did create the franchise, and so receives royalty payments from the various properties under it. Trans people, and others with decent opinions on trans people, have pointed out that purchasing the game gives money to a notorious bigot, and have asked people to refrain from doing so.
\nOther controversies included the fact that Troy Leavitt, the game's lead designer, used to run a YouTube channel with anti-feminist and anti–social justice content. The game's content also attracted some criticism: the plot focuses on goblins which, within the Harry Potter universe, are hook-nosed greedy bankers—an antisemitic trope in the real world.
\nAll of this has led some of the aforementioned people with decent opinions to spoil the game for unsuspecting fans who have not yet played through it. A common copypasta for this is thus:
\n\n\nYour teacher, Professor Eleazar Fig, dies at the end of Hogwarts Legacy. This happens in all possible endings and can't be changed. Oh and Rookwood is the one who cursed Anne while the goblins were framed
\n
Of course, people these days generally don't line up at a store to purchase video games on physical media, so the spoiling has been chiefly been taking place online. On the other hand, scrolling through social media timelines is a far more common practice today, so those present a ready place for the spoilers.
\nThe two points in time offer an interesting look into the arc that the Internet took over a span of over 17 years. In 2005, pocket computers were an esoteric gadget, and the infrastructure to comfortably operate them everywhere was not yet there. Social media was nascent, and spending a lot of time talking to people over the Internet was the domain of weird nerds, rather than a normal part of everyday life for a large portion of the population.
\nWhat existed was memes, and what existed was spaces on the Internet inhabited by the aforementioned weird nerds. 4chan is, perhaps, the most well-known of them, as it remained notable and notorious over the following decade and beyond. Looking at the artifacts of that era's Internet, we can glimpse a particular culture, which is also exemplified by the 2005 and 2007 spoiling memes.
\n2000's shenanigans were, notably, not particularly political. Fans of the Harry Potter franchise were not categorized as any particular political camp at the time. Rather, the efforts to ruin their day are motivated by a desire to cause mayhem for the sake of mayhem. People yelling spoilers while wearing vaguely racist costumes claim to do so for the lulz, because bullying people is amusing.
\nIf you do not find bullying people amusing, you are, of course, excluded from the in-group. If you think that the conduct of the group is perhaps a bit too libertine, then you are a killjoy who deserves to be the group's victim. This dynamic has existed long before the Internet; it probably also existed in Internet spaces prior to the mid-2000s. It was the mid-2000s when there were the first glimpses of how that dynamic would play into the increasingly central place the Internet and social media played in people's lives.
\nThe 2023 conflict over Hogwarts Legacy is, by contrast, quite political. The opposition to the opposition, coming from the right-wing, is the familiar kind of contrarianism that seeks to dismiss and oppose any sort of concern that the more progressive elements bring up—even if that concern is that trans people should not be subjected to genocide.
\nOne could say the 2023 spoilers are not coming from the opposite side of the political spectrum, relative to the mid-2000s spoilers, because the mid-2000s spoilers were not coming from a political camp at all. However, it feels like the mid-2000s spoilers were coming from the modern right, because the modern right is essentially what the milieu that brought us the 2000s spoilers eventually grew into.
\nSince, over the span of 2005 to 2023, the Internet has shifted from being a separate thing to being an integral part of the real world, we should examine the broader context of how things changed.
\nThe antisemitic caricatures used for the goblins in the Harry Potter universe were Rowling's invention, long before they found their way to the latest video game. With the rise of Rowling as a prominent public bigot, people have started pointing out that her Harry Potter books have had bigoted elements in them from the start. Whether it is the lazy use of heavy-handed racial stereotypes for minority characters, or seemingly more malicious inclusion of bigoted tropes (like the goblins), it leaves a sour taste for many readers in 2023.
\nIt would be a mistake to assume that the views we, in 2023, consider progressive or left-wing have emerged whole cloth between 2005 and now. In a lot of cases, what progress entailed was broadening of the portion of the population that hold these views to be true. The opinion that it is fine to treat queer people with disdain is less common in 2023 than it was in 2005, but even in 2005 and before, we had people who believed that it was wrong.
\nThe Internet has a lot to do with this state of things. The Internet has provided a vector for exposing people to, and convincing people of views that may otherwise be considered radical. Radicalization does not only work in one direction, however. The culture which spawned Deathly Hallows spoiling events is what eventually evolved into one of the components of the modern Internet political right. One need look no further than the fact that the 2023 4chan is commonly considered one of the hives of that broad group.
\nThere are two ways this can be looked at.
\nOn one hand, we started out with some weird nerds who, freed from some of the constraints and limitations that life otherwise imposed on them, found spaces on the Internet where they could indulge in whatever they wished, with no regard for the consequences for themselves or others. As the Internet increasingly merged with the so-called real world, these consequences also increasingly became real-world themselves, which led to more pronounced opposition. The erstwhile weird nerds then became reactionaries, joining the broader reactionary currents. Trolling people for the lulz became owning the libs.
\nOn the other hand, we started with Internet spaces inhabited by weird nerds, with misogyny, racism, and other forms of bigotry that alienate people who are not the right kind of specific weird nerd. The broadening reach of the Internet, combined with general social progress has, however, made more inclusive spaces available to a broader range of weird nerds, and other people. As even people who are capable of staying in toxic spaces often find staying in non-toxic spaces preferable, the average baseline level of toxicity has decreased. Those who do actually prefer their spaces toxic were forced to consolidate, and so became a more coherent reactionary force, but overall the Internet is a better place to be in. If you believe that you should not be subject to genocide, then the Internet can let you reach plenty of people who believe likewise.
\nBoth of these perspectives are fundamentally true, of course. It is common to lament the world which our interconnectedness has brought us. Perhaps it is important to also acknowledge the ways in which our world is now better, if only because it shows that things can get better, which means that there is a point to it all. Knowing what is wrong, and how things got to be wrong is important, but so is believing that we can win.
\nSo, you know, fuck the wizard game.
\n"},{"id":"https://dee.underscore.world/blog/export-subst-reproducibility/","url":"https://dee.underscore.world/blog/export-subst-reproducibility/","title":"Git export substitutions and reproducibility","date_published":"2022-06-10T19:34:42.000Z","date_modified":"2022-06-10T19:34:42.000Z","content_html":"There is an issue that I have encountered some months ago (and mentioned on the Fediverse back then), but I am still occasionally reminded of it when troubleshooting weird behavior with Nix, so it might be interesting to take a closer look at it. In short: when using Git's export substitutions, a tarball exported by Git from the repository at a particular revision may not always be the exact same, depending on external factors.
\nTo understand how this quirk affects Nix, we have to both understand what Nix's fixed-output derivations are, as well as how some rather obscure Git features interact with Git internals.
\nIf you have had passing contact with Nix and Nixpkgs, you may have heard people refer to packages from Nixpkgs as derivations. A derivation is, broadly, a thing that describes a build action that Nix can take.
\nDerivations take as inputs whatever is needed to carry out the build action, contain a script that specifies how to do the build, and register whatever that script produced as outputs. Suppose a simple C program: the inputs to its derivation would include things like the source code of the program, a C compiler, Make, perhaps some library that the program needs; the build script would call make
and make install
; and the output produced would be the executable binary that make install
copied out.
Inputs to our derivation are outputs of other derivations. Those derivations can, of course, have their own inputs, which are other derivations, and so on—this is how we build our dependency graph.
\n\nThis is also where Nix's functional nature comes in: a derivation whose input derivations are unchanged, and whose build script is unchanged is assumed to always produce the same output—like a pure function. This is what allows for a Nix binary cache: instead of building the derivation ourselves, we can download the output of the same derivation built on some other machine, because that output is assumed to be the same.
\nIn our previous example, one of the inputs to the simple C program derivation was its source code. This presents a problem: source code is not really the output of a build process that takes other inputs. Instead, source code is an input from the outside world.
\nTo address this, Nix has a special type of a derivation: a fixed-output derivation (often abbreviated as FOD). Like with ordinary derivations, fixed-output ones have inputs, a build script, and produce an output, but in addition they also contain the expected cryptographic hash value of the produced output.
\nAfter Nix carries out the build action specified by a fixed-output derivation's build script, it hashes the produced output, and checks it against the pre-recorded expected hash. If the hashes do not match, the build is considered to have failed. Unlike with ordinary derivations, a fixed-output derivation is considered unchanged if its expected hash remains unchanged—its script and inputs can change, as long as it produces the same exact thing as before.
\nWhile builds carried out from ordinary derivations do not have network access, fixed-output derivation builds do. In practice, fixed-output derivations are thus often used to fetch source code from the Internet. Because we have to specify the hash of the source code to be fetched, we can be reasonably certain that we are fetching the same source code every time we rebuild the derivation.
\nGit has a handy command for exporting the tree at a given commit to a single archive file (tar or ZIP): git archive
. This is useful for situations such as distributing a source code release: simply git archive
the repository at the relevant tag, and publish the resulting tarball.
Not having the Git repository around can pose some problems, though. For example, some build processes expect to be able to discover the current Git commit hash, because they embed it in the version information of the binary they produce. To address this, Git provides means of doing export substitutions. Using .gitattributes
, one can specify a list files that should have placeholder substitution performed on them when the repository is exported to archive. These placeholders can be for things like the current commit hash (including in its abbreviated version), the output of git describe
, or metadata like the commit author and date.
In many places in Git's UI, it uses abbreviated commit hashes. While a full Git commit hash is 40 hexadecimal digits long (as SHA-1 hashes generally are), commits can usually be unambiguously referred to with some smaller amount of digits.
\nThe bigger a repository—or, more precisely, the more objects a repository contains—the larger the probability that two object hashes will share the same prefix of some length. A long time ago (before 2016), Git used to default to 7 digits. As time went on and some repositories grew in size, it turned out that 7 digits, or even 8, 9, and larger amounts were not enough to unambiguously refer to objects in those repositories. Because of this, a heuristic was added to Git to estimate how long an abbreviated hash needs to be, based on the estimate of the object count of the repository. This is not an exact determination of the minimum unambiguous length of a hash, but rather a relatively fast guess. It is used by default in various outputs of Git, though a fixed length can be set via configuration.
\nfetchFromGitHub
Github repository pages offer an option of downloading the current tree as an archive file.
\n\nThis functionality internally uses git export
, which means that the downloaded ZIP has the relevant placeholders substituted with actual values. The archive downloads are also available under a predictable URL, which is convenient for scripting.
Nixpkgs contains a fetchFromGitHub
function for—predictably enough—creating fixed-output derivations that fetch source code from Github. As an optimization, when possible, fetchFromGitHub
will opt for downloading an archive tarball (Github does provide both tar.gz
and zip
archives). This is desirable, because we usually do not need a Git repository clone, and the tarball is a compressed (thus smaller) archive that can be quickly and easily downloaded over HTTPS.
Consider a repository that uses Git export substitutions to place the abbreviated hash somewhere in the exported archive file. If we use this exported file in a fixed-output derivation, it will work as long as the length of the abbreviated hash does not change. If it does change, the cryptographic hash of the archive will be different, and our fixed-output derivation's build will start failing.
\nThe length Git picks for the abbreviated hash can change over time—if people keep making new commits in the repository, the amount of objects Git keeps track of will increase, and the heuristic will pick more digits for the abbreviated hash. Under this assumption, our fetchFromGitHub
derivation can become invalid at some point after we write it, even if we are still fetching the same revision.
In practice, however, this is more complicated. Practical tests show that repeatedly downloading a repository archive from the same URL, within a short span of time, can result in getting versions that include both shorter and longer substituted hashes, seemingly at random.
\nWe can speculate on why that is. Consider that the hash length estimation is based on the number of all objects in the Git repository, which does not necessarily include only the objects within the current commit tree (which is to say objects associated with the current and previous commits). Such a Git repository can include other things, such as orphaned commits which have not been garbage collected yet, or branches for things like pending pull requests. Github could, conceivably, have several servers which hold copies of a given repository, with each containing the entire main
branch, but some of these servers could have differing subsets of other objects.
If our request can potentially go to any of those servers for load-balancing purposes, then we could end up with different tarballs based on how many objects the given server holds. Yet another reason would be some form of caching, where one cache server holds a tarball generated at a time when a shorter hash sufficed, while another has to regenerate the tarball from the current repository state. The details are opaque to us since we are (presumably) not Github, but we can certainly come up with plausible scenarios.
\nThe obvious solution is to not use abbreviated hashes in export substitutions. If the commit hash is to be embedded somewhere in the built artifacts, using the full hash ensures a far smaller chance of a collision in the future either way (if the full SHA-1 hash collides, then we are really in trouble).
\nOutside of that, fetchFromGitHub
can be forced to download via Git, rather than via an exported archive. This can complicate the build process, but a Git checkout should be entirely reproducible (Git commits are, after all, referenced by cryptographic hashes themselves).
Generally, when building from a tagged release, embedding Git revision hashes may not be necessary. The tag will exist in the repository and point to the given revision, and in general should not be moved anyway. Packages in Nixpkgs are most commonly tagged releases, rather than arbitrary commits from the trunk branch, and those commonly embed version numbers, rather than the precise commit hash.
\nexport-subst
featureexport-subst
Nix-related issueHTTP compression has been part of the standard since the late 1990s. The idea behind it is simple: to save bandwidth, compress the responses that a server sends. The two most common compression formats understood by browsers today are gzip and DEFLATE. Both of them employ the same compression algorithm, but vary in how they do checksumming and headers. Both of them have also been in the standard since the late 1990s.
\nBrotli is a lossless compression algorithm developed at Google, starting in the year 2013. Initially intended for use in the Web Open Font Format, its specification was published as RFC 7932 in 2016, and at about the same it started being available in mainline Web browsers as a generic HTTP compression format.
\nAs Brotli tends to achieve better compression ratios than gzip, and is widely supported by browsers in use today, it may be a good idea for today's Web servers to support it, in addition to the classic gzip and DEFLATE encodings. NixOS's Nginx can do Brotli, but it requires enabling an extra module, as Brotli support is not built into the server normally.
\nNginx from Nixpkgs can be rebuilt with the Brotli module by using override
:
nginxMainline
is Nginx from the mainline branch. Mainline is the Nginx branch which sees more active development, while the stable branch is closer to what in other places would be called a Long Term Support version—it sees mostly bug fixes, and no new feature development. In both cases, there are tagged releases, and both branches are considered suitable for use in production. nginx
in Nixpkgs points to nginxStable
, but it is fairly easy to switch between the two.
When overriding modules
, we have to remember to include the original modules—nginxMainline.modules
—and append brotli
to them. Unlike with NixOS configs, there is no merging of lists, so if we were to set modules
to just the brotli
module, we would get rid of the defaults.
Chances are, this modified Nginx will not be in cache.nixos.org, and will need to be built locally. An exception is nginxStable
with Brotli; this tends to be in cache, because a test incidentally builds it. The Discourse module in Nixpkgs enables Brotli with Nginx, and so when Discourse is tested, Nginx with Brotli has to be built, and subsequently ends up in cache. Fortunately, compiling Nginx is not a very resource intensive operation either way.
In NixOS configurations, an alternative to manually overriding modules
is available: services.nginx.additionalModules
. This setting internally does what we previously did: add our select modules to the default list of modules.
Adding the Brotli will, however, not add the relevant Brotli settings. The module's README contains an example configuration, so we can crib that instead.
\n\nThis sets up both serving of statically compressed assets with brotli_static on
(more on this later), and dynamic compression of requests with brotli on
. The compression level can be tweaked to trade speed and CPU usage for density. We generally do not want dynamic compression to be too intense, because the server spending more time to achieve higher compression means more time spent waiting by the clients. There is also a reasonable list of MIME types to compress. We do not want to compress things like already-compressed images, but on the other hand, mainstays of the Web like HTML and CSS do compress pretty well with Brotli.
This configuration enables Brotli on the whole server, but we could also enable Brotli on vhost or location level—just add brotli on;
to the extraConfig
there instead.
Dynamically compressing responses generated by a dynamic service is useful, but sometimes we also serve stuff that is entirely static. Since, in that case, we have the static files ahead of time, we can compress them ahead of time as well. As a bonus, in this situation we can also use higher compression levels—when building our website ahead of time, we do not have the same time constraints as an active Web server.
\nWhen brotli_static
is set to on
and Nginx is asked for a file, it will look look for a file with the same name as the requested one, but with .br
suffixed, and serve that compressed file directly in response (if the browser can accept Brotli compression). This means that in order to serve precompressed files, we simply need to put a bunch of files with .br
extensions next to our original files.
Modern fancy webapp build systems often have something that can be inserted into the build pipeline to generate such precompressed files. When dealing with packages in Nixpkgs, however, we might want to avoid patching the build scripts in each package to get it to emit .br
files, and instead add them to the finished thing with a more generic solution. To this end, we can write a derivation that uses a simple Bash script and the brotli
binary.
Let's follow this from inside out, starting with buildPhase
. First, we call find
to locate files of the types that we want to compress, and pass their list to xargs
, so that we can do parallel execution with -P
(which cannot be done with find -exec
).
The variable $NIX_BUILD_CORES
is supplied by Nix, and is meant to be the number of concurrent jobs that should be executed during a build. It corresponds to the nix.conf
setting cores
(nix.buildCores
in NixOS config), is meant to be the number you would pass to make -j
, and can vary from build machine to build machine (hopefully your builds don't break reproducibility when parallelized). Nix actually has a second setting—max-jobs
(nix.maxJobs
in NixOS config)—which controls the number of build jobs (builds of individual derivations) the particular machine will run at the same time. Each build job can potentially run cores
processes, so you could have max-jobs
×cores
processes running at the same time.
Our find
call is supplied with a list of arguments from findQuery
, which tells it which files to locate. What we are doing here is turning a list of extensions (like, say, [ "html" "css" "js" ]
) into a series of -iname
(case-insensitive name match) arguments separated by -o
("or"), which would turn our example into -iname '*.html' -o -iname '*.css' -o -iname 'js'
. We enclose this all in parentheses (escaped, so they don't get eaten by bash), because otherwise find
would interpret the query as (in pseudocode) (of type file and extension "html") or extension "css"
instead of of type file and (extension "html" or extension "css")
.
Our derivation is two nested functions. This is so that we can get the inputs of the outer function supplied by callPackage
, giving us the inner function that we can call directly, using it something like this:
In our previous brotlify.nix
, we simply added some .br
files to an existing webapp, and copied the whole thing to a new output. This is convenient, but sub-optimal when it comes to composability.
Consider, for example, that we may wish to provide .gz
files in addition to .br
files. Nginx can serve precompressed .gz
files, using a module which is built into the Nginxes from Nixpkgs by default. While gzip compression is usually less resource intensive than Brotli compression, there is the Zopfli project (Google apparently likes to name their compression projects after bread), which tends to produce .gz
files with better compression ratios than those achieved by GNU's gzip, at the cost of longer compression times.
We could create something like brotlify.nix
that calls zopfli
on all the files instead of brotli
, and then chain them, like brotlify { src = (zopflify { src = element-web; }); }
. We can reason that the functions commute—regardless of whether we apply the Brotlification or the Zopflification first, we still get the same result, as each function simply adds more files, leaves existing files unchanged, and will not try to compress each other's outputs. The problem is that Nix cannot reach that conclusion, and so brotlify { src = (zopflify { src = element-web; }); }
and zopflify { src = (brotlify { src = element-web; }); }
are not the same derivation. Suppose we added a couple more filter functions like this, and the amount of possible permutations becomes a problem.
A different solution is to make brotlify
and zopflify
output just their corresponding compressed files, and then overlay their outputs on top of the original webapp. In addition to nicer composability, we also receive the benefit of being able to apply the functions concurrently, while with the previous example, we would have had to wait for the first function to produce an output before applying the second function. Let's modify brotlify.nix
for use with this pattern.
The only change is essentially that we now only copy *.br
files to $out
, preserving the same directory structure (which is where install -D
comes in handy). We can conceptualize the corresponding zopflify
version as being essentially the same, except calling zopfli
instead of brotli
and exporting *.gz
files instead of *.br
files.
To compose all of this together, we use the buildEnv
function in Nixpkgs. buildEnv
combines several directory trees together using symlinks, which is handy for all sorts of things, like "installing" several end-user packages into one environment. It can also be used to layer our compressed overlays over the base package. With the default configuration, Nginx will follow symlinks, so a directory tree full of symlinks is fine to use as root
.
The nixos-rebuild
script, when asked to switch
to this configuration, will build both the Brotlified and Zopflified versions, possibly concurrently.
We can verify that the server is indeed serving precompressed responses by issuing a request with curl: curl --raw https://element.example.net/olm_legacy.js -H 'accept-encoding: br'
, and possibly comparing it against the precompressed version on disk. If you need to find the store path of the composed tree on a running system, you can do systemctl cat nginx
to find out the location of the Nginx config file, and then inside the config file, look for the root
directive that was emitted under the relevant vhost.
The above derivations could use some further improvements. For one, they indiscriminately compress everything, including very small files. Compressing very small files can result in compressed files that are actually larger than the original, which is why both the Brotli module and the built-in gzip module by default do not apply compression to small responses; the default the threshold is 20 bytes for both. Still, it would be prudent to examine how much each file was actually compressed, and prune those which do not show sufficient improvement over uncompressed sizes.
\nAnother thing to note is that combining compression and encryption opens the possibility of compression oracle attacks. These attacks rely on the fact that plain text with repeating strings will compress to a shorter length than text where those strings do not repeat. Consider a webpage that displays both a secret token and some user input. If both the user input and the token are the same, the page will compress to a shorter length than if they are different. An attacker who can repeatedly try different user inputs can, therefore, check the length of the response to see if their guess was correct, even if they cannot actually decrypt the response. This is the basic principle behind the BREACH attack.
\nNevertheless, compression of static, non-secret assets is generally safe.
\nIn January 2024, I rewrote my precompression scripts, which were based on a slightly expanded versions listed in this article, into a more reusable package, largely based on Nushell. It is called nix-compressify, and is in a hopefully more easily reusable shape.
\nIf you just want to install some fonts through Home Manager, you can enable the fonts.fontconfig.enable
option, and then add some font packages to home.packages
.
This article explores doing some weirder things with font packages and Home Manager.
\nRecently, I was reading some source code on my computer, and suddenly it occurred to me to stop reading the source code and start messing with my computer's font setup instead.
\nMy preferred font for reading source code is Fantasque Sans Mono. In version 1.8.0, Fantasque Sans Mono received a number of programming ligatures, which is a problem for me, because I do not like programming ligatures. Some people find them useful in making code easier to read, but I am not one of those people.
\n\nPreviously, I dealt with this problem by simply using an older version of Fantasque. I would drop it in my ~/.local/share/fonts
, where it would be picked up by Fontconfig
This is, of course, not very reproducible—if I wanted to use these fonts on another computer, I would have to (probably manually) copy the files into that computer's ~/.local/share/fonts
. Avoiding these sorts of things is one of the reasons I use Home Manager—I can keep a set of configuration files in one central Git repository, and not have to worry about keeping track of a bunch of files, while remembering the where, what, and how of them.
Fantasque Sans Mono can be found in Nixpkgs (under fantasque-sans-mono
), and thanks to the diligence of Nixpkgs contributors and maintainers, it is at the latest version: 1.8.0. This means that I can easily install the font by simply adding it to home.packages
in my Home Manager configuration, but if I do that the font will come with ligatures, which I do not want. I thus set out in search of more complex solutions, and perhaps some yaks to shave along the way.
Within an OpenType font, glyph substitutions are organized into categories called features. The idea is that software doing typesetting can decide which features are desired, and only perform glyph substitutions listed under those features.
\nFantasque lists its programming ligatures under the calt
feature, also known by its friendly name Contextual Alternates. If software can be convinced to ignore the Contextual Alternates feature, it should not apply the ligature substitutions when displaying text using the font. This is, in fact, how the example image was created in Inkscape: SVG supports styling text with CSS, and CSS supports font-feature-settings
to control font features, which means we can do font-feature-setting: "calt" off
to tell the rendering engine to skip the Contextual Alternate substitutions.
Fontconfig can be used to set the default font features for a given font, and the Arch Wiki provides a helpful example of how to do this. In order to apply this solution via Home Manager, we need to instruct it to deploy the configuration file to where Fontconfig will expect it (i.e. ~/.config/fontconfig/conf.d/
).
Problem solved, right? Not quite. As the Arch Wiki article points out, not all software respects this setting. Software using Pango (so GTK apps) in general will follow the setting and disable the ligatures, but other software—such as Firefox, or programs using Qt—will fail to do so.
\nA way to ensure our software will not render ligatures is to give it a font file without the ligatures in it. One option is to use an older version of Fantasque, as previously; another is to strip the ligatures out of the latest version. Fortunately, I am not the only one who prefers fonts without programming ligatures, and so someone has already provided the latest version, but without ligatures (I would like to note my appreciation for the "sans ligatures" name here).
\nThe Sans Ligatures version of the font should be relatively easy to package, since it is very similar to the upstream Fantasque. This means we can take inspiration from fantasque-sans-mono
as it is packaged in Nixpkgs. Unfortunately, since the derivation uses fetchzip
we cannot simply use fantasque-sans-mono.overrideAttrs
to change its url
, sha256
, and name
. What we can do is take more than just inspiration, grab the entire file, and modify it for our use. If you do something like this, and also publish your NixOS or Home Manager configurations on the Internet, keep in mind the Nixpkgs license (MIT).
Normally, when using fetchzip
from Nixpkgs, its postFetch
extracts the ZIP we asked it to download, preserving the tree. This is useful for getting the source tree of something that we are building, but it is not what we want in our case. What we do instead is extract all the .otf
files from the archive and put them under share/fonts/opentype
, which is the standard location OpenType font files are expected to be. We also need to populate the sha256
hash of the output; an easy way to figure that out is to set sha256 = lib.fakeHash;
, and try to build the derivation, which will cause Nix to error out and report what hash it got.
What we can do now is simply callPackage
on the file we wrote (well, the file we modified).
I use exa as my ls
replacement. exa has a feature which allows it to display icons next to listed files. For this feature to work, exa has to be run inside a terminal emulator using a font that supports those icons; they are codepoints in the Private Use Area, and thus not standardized in Unicode, which means they are not covered by most fonts.
I had previously been using the Nerd Fonts version of Fantasque Sans Mono for this purpose. Nerd Fonts is a project that takes existing fonts and embeds a whole bunch of icons from various icon sets in them. They distribute a number of pre-patched fonts, and, in fact, you can get those pre-patched fonts via Nixpkgs: pkgs.nerdfonts.override { fonts = [ "FantasqueSansMono" ]; };
will get you Fantasque Sans Mono with icons.
Of course, because the Fantasque patched by Nerd Fonts is the latest Fantasque, it will have ligatures. Fortunately, Nerd Fonts also provides a patcher script which can be used to embed all the various Nerd Fonts icon sets into any arbitrary OpenType font. This means we could have a derivation that patches a font—like the Fantasque Sans Ligatures from above—with the Nerd Fonts icons.
\nOne problem, though: in order to run the patcher, we also need a bunch of files from the Nerd Fonts repository, but the repository also contains a bunch of stuff we do not need, and is huge. The project commits all their pre-patched fonts, and there is a lot of patched fonts: over 3,000 files, totaling over 4 GB. This is a problem in two ways: one, it means downloading the whole repo takes a lot network traffic—over 2 GB if compressed; two, hashing the downloaded repo is a long and resource-intensive process for Nix.
\nOne thing we can do is take inspiration from having already messed with fetchzip
before, and override the unzipping script to exclude the big directories from the downloaded snapshot of the repository. This does not solve the downloading problem, but it solves the Nix hashing problem.
This derivation is a bit more involved. Our src
is the ZIP file containing the repository source snapshot that Github offers as a download for the given release. We extract this, omitting the fonts files that we do not need (this is what the -x
argument of unzip
is for).
The derivation takes Python and python3Packages.fontforge
as a inputs. We can find out we need the latter by reading the readme for the Nerd Fonts project, and we obviously need to have Python available if we want to run a Python script.
The actual build script goes through all the fonts in the Fantasque Sans Ligatures package, and calls the font-patcher
script on them, asking the script to output patched fonts to this derivation's output directory. The --complete
argument asks the patcher to add all the icons it knows; we could replace that with some subset of icons, instead. We also skip drawing progress bars, because such things do not play well with line-based Nix build logs.
At this point, the having to download a Zip file that is gigabytes in size still remains a problem. Fortunately, modern Git is actually pretty decent at not pulling more data than is needed.
\nTraditionally, cloning with Git downloads the whole tree for the HEAD commit, or the selected branch or tag, as well as the whole history. The latter can be truncated at some point with --depth
, including at the commit itself, as is the case with --depth 1
. This is a shallow clone. Such clones still involve pulling the whole tree, and with Nerd Fonts the problem is not just that the history is deep and contains a lot of data, but that one commit is already a lot of data.
git clone
also takes another useful argument: --filter
. Using --filter=blob:none
, we can tell Git that we do not want to pull the blobs—that is, all the files that end up in a checked-out directory tree. In this situation, Git will lazily pull the blobs from the remote as needed, such as when we tell it to check out a commit, and it notices that it does not already have the blobs needed for that checkout. This is a partial clone.
The third useful feature is sparse checkout. Git allows us to specify filters for paths, that get applied when performing a checkout. This involves essentially telling Git that when you are checking out a branch, you are interested only in a specific list of paths, and it should not check out any other paths, even if they are in the commit. We can ask Git something like "Please switch to this branch, but only give me src/
and README.md
, and nothing else".
We can combine these three features to minimize the data we need to download to get a working patcher. The history can be skipped with --depth 1
, as we will not need it. The blobs can be skipped at first with --filter=blob:none
, and by telling Git to not initially check out anything at all. Then, we can set a filter to only check out the file we will need, omitting the directories with thousands of fonts in them. The result of this is Git only pulling the small files we need, and not bothering to pull the huge directory. This helps us keep both the size of the repo down, as well as reduce the amount of network traffic.
More good news is that we do not actually need to write a fixed output derivation with a script that does this whole Git dance. In current Nixpkgs master, fetchGit
and various fetchers that use it (like fetchFromGitHub
) now support a sparseCheckout
argument, which makes the fetcher internally do something very similar to the manual example here. The author of the recently merged PR which adds it even mentions Nerd Fonts!
The sparseCheckout
syntax is essentially the same as the one used in .gitignore
, except reversed: anything that matches will be checked out, and anything that does not match will be ignored. This is how we can add the src/
directory, but exclude its unpatched-fonts/
subdirectory (we do not need the unpatched fonts, we bring our own).
With some minor modifications, the Nerd Fonts patching derivation could be made even more fancy, allowing for patching of arbitrary fonts.
\nIt is thus, after meandering around various topics, that I have arrived at working fonts—at least some of them, as I did not discuss the boring ones.
\nThe conclusion here is that Nix is enjoyable if you are the kind of person who finds appeal in the idea of thinking "I should mess with my font setup" and then spending several hours diving into several different topics.
\nAfter publishing this article, I found out that the Nerd Fonts patcher is, in fact, already packaged in Nixpkgs. It is available under nerd-font-patcher
(note that this is "font" not "fonts"). I now feel silly.
Note: Since this article was written, there have been changes to how Home Manager works with flakes. Additionally, while flakes are still marked as an experimental feature, the stable version of Nix has also since advanced. Newer versions of Nix and Home Manager have also changed where profiles are stored by default (there may now be a ~/.local/state/nix
).
While a lot of the content of this article still broadly applies, you may wish to consult the Home Manager manual (specifically the chapter on flake use), or the Nix Reference Manual, for more up to date information.
\nIf you are a Nix user, you may have heard of Home Manager as the recommended way to manage your user environment. You may have also heard of flakes, the upcoming new way of managing dependencies and packages.
\nYou can combine the two, and use Home Manager with flakes. Frequently, this is done by including a Home Manager configuration in the (also flakified) system configuration. Home Manager can, however, be also used on operating systems other than NixOS, in which case the NixOS system configuration is not there. Fortunately, Home Manager can also use flakes entirely on user level. Let's assume, then, that we have an existing Home Manager installation, not currently based on flakes, and want to switch it over to flakes.
\nWith your usual Nix installation, each user can manage their environment with the nix-env
tool. When the user installs or uninstalls things with nix-env
, a new generation of the user's environment is created with these changes, providing a simple way of managing packages that resembles the classic package managers found on other operating systems. The nix-env
approach does have some problems, however, broadly relating to it being imperative in a system focused on being declarative. To address these problems Home Manager was created. With Home Manager, the user can write a configuration for their environment, similar to a NixOS system configuration. This Home Manager configuration specifies the packages and home directory configuration of a single user.
Flakes, on the other hand, are an upcoming feature of Nix itself. Flakes are intended to introduce a new way of declaring dependencies in Nix packages, improving reproducibility and, from a practical standpoint, proving a better alternative to the current approach of using channels and ad-hoc pinned dependencies. If you have ever used Niv, you can think of flakes as that, but better integrated into the Nix tool itself.
\nFlakes are a feature of the upcoming 2.4 version of Nix. The standard Nix, even in the unstable Nixpkgs channels, is Nix 2.3, but there is also pkgs.nixUnstable
, which points to 2.4 builds.
Both nix
and nixUnstable
come with a binary called nix
, and so collide. If we want to experiment with flaky Nix without overriding stable Nix, we can make a wrapper (courtesy of the NixOS Wiki). An easy way to do this is to use our existing home.nix
, the Home Manager configuration file:
Note that even with Nix 2.4, we need to have the flakes experimental feature explicitly enabled. This can be done by adding experimental-features = nix-command flakes
to our nix.conf
(generally found in ~/.config/nix
), but if we do this, regular Nix 2.3 will complain about not knowing what experimental-features
is. As an alternative, we apply these settings on the command line in the wrapper.
After switching to our new configuration we will have a nixFlakes
in our PATH
, and so will be able to use it from the command line.
flake.nix
fileLet us assume that our prior Home Manager configuration was kept in a Git repository somewhere in our home directory, with ~/.config/nixpkgs/home.nix
symlinked to a relevant file within the repository.
To create a basic flake.nix
, we can use execute nixFlakes flake init
within our configuration repo. This command can be used to create flakes out of templates, and if we don't explicitly ask for a specific template, it uses the default one, which looks something like this:
As we can see, a flake file is simply an attrset. The two important attributes in it are inputs
(absent from template), and outputs
. inputs
defines the dependencies of our flake, and outputs
defines the things our flake offers, that other flakes and tools can consume.
The template does not have explicit inputs
; nixpkgs
is resolved using our Nix flake registry (a locally stored map from flake names to flake URLs, that Nix populates from a global registry on the Internet). The template does have two outputs: packages
, which defines one named package, and defaultPackage
, which marks the one package as the flake's default.
Flakes can define various outputs, which are then used by various tools—defaultPackage
, for instance, is what Nix would look at if we asked it to install this flake into our profile, without explicitly asking for any specific package from inside the flake. Home Manager, on the other hand, uses the homeConfigurations
output.
homeConfigurations
should be an attrset, mapping names to Home Manager configurations. The names are either usernames, or in the format of "username@hostname".
For inputs to our flake, we will want to include Home Manager. This does mean that our flake is technically independent of our current Home Manager installation. In fact, we could bootstrap Home Manager this way without having Home Manager installed in the first place! Having Home Manager already installed is also okay—our prior Home Manager generations will not be reset.
\nAssuming our username is someuser
and our hostname somecomputer
, we can come up with a basic flake like this:
As we can see, "someuser@somecomputer"
is mapped to a call to homeManager.lib.homeManagerConfiguration
. We are calling a function exported by Home Manager's flake that will create a Home Manager configuration, out of the attrset that we pass to it.
configuration
should be the most familiar bit of said attrset. This is essentially the same function as the one we would normally define in our home.nix
. In fact, instead of defining our configuration inline, in the flake, we could do configuration = import ./home.nix;
, which is the practical way to move our old configuration to flakes. The one caveat is that some of the home
settings that might have previously been in our home.nix
file are now in the flake—home.stateVersion
, home.username
, and home.homeDirectory
now live adjacent to, instead of in configuration
.
We can build our home-manager profile with just our flakey Nix. This can be done by building the activationPackage
attribute of a particular configuration. In this specific case, this means invoking nixFlakes build '.#homeConfigurations."someuser@somecomputer".activationPackage'
. After the build completes and we have our result folder, switching to the newly built configuration is simple: invoke result/activate
from the command line.
We should now be in our new profile. It is, of course, possible something has gone terribly wrong and our new profile is broken in some way. Fortunately, we are using Nix, so we can perform a rollback. Invoking home-manager generations
should give us a list of generations, and paths to them. We can simply take the next-to-last path, append activate
it to it, and run that, which should reactivate our previous generation, rolling us back. If home-manager
is absent from our PATH
, we can also go to /nix/var/nix/profiles/per-user/someuser
, where we can find a number of home-manager
symbolic links. Again, if we follow the next-to-last link, we can use the activate
script within to restore that generation.
home-manager
tool with flakesWe could just repeat the nix build
command whenever we want to switch to a new Home Manager generation, but that is a bit awkward. Fortunately, the home-manager
tool, besides working with the usual ~/.config/nixpkgs/home.nix
, can also work with ~/.config/nixpkgs/flake.nix
.
Unfortunately, in this situation, we cannot simply create a symlink from ~/.config/nixpkgs/flake.nix
to wherever our configuration repository is, as we would experience problems due to restricted mode; Nix will refuse to poke around the symlink target unless --impure
is passed. One interesting—if seemingly hacky—thing we could do, however, is create another flake.
Yes, we can use local filesystem paths as flake inputs. Of note is the fact that we could use network URLs here as well, and let Nix handle pulling our configuration from elsewhere, if we happen to have it on an easily-reachable server.
\nWe have also declared nixpkgs
as a separate input, and pinned homeManagerConfig
's nixpkgs
input to our nixpkgs
input. This is one way to achieve a workflow similar to that we would have previously used with channels: our nixpkgs
input will stay pinned where it is until it is updated. This can be useful if, say, we are changing something in our home-manager configuration and want to rebuild without necessarily downloading a lot of fresh updates.
We can update individual inputs by using --update-input
: nix flake lock --update-input nixpkgs ~/.config/nixpkgs
. home-manager switch
and home-manager build
should work with the configuration specified in ~/.config/nixpkgs/flake.nix
, although we might have to make unstable Nix our main nix
binary, rather than have it behind nixFlakes
.
If we do not add a flake.nix
to our ~/.config/nixpkgs
, we can instead explicitly point home-manager
at a flake: home-manager switch --flake path:/home/someuser/stuff/home-manager-config
.
One of the reasons to get into flakes is for their ability to manage inputs other than mainline Nixpkgs. With stable Nix, the options here are usually either manually managing fetch*
functions that pull in some repository, or adding Niv, and letting it manage pinning the added packages. With flakes, however, this is handled natively by Nix itself.
There are two types of repositories we can encounter in the wild: ones which have their own flake.nix
file, and ones which do not. Fortunately, Nix can handle even the non-flake inputs—we simply have to indicate that the input is a non-flake:
The main thing to point out here is extraSpecialArgs
—this is how we pass extra arguments to the configuration
function, outside of the pkgs
which Home Manager will supply by itself. Inside configuration
itself, we import nixGL
just like we would if we had used Niv or called fetchTarball
or fetchFromGitHub
ourselves. The details of how to import a given non-flake input, and what to do with it, will vary, but generally non-flake instructions should be adaptable to use with flakes.
Flakes, on the other hand, are more structured. Conventional flakes will generally provide packages
, defaultPackage
, and overlay
as outputs. As such, we will have two options for installing packages: either by grabbing the package from the two packages outputs directly, or by applying the flake's overlay onto our imported Nixpkgs.
The former case is pretty simple:
\n\nDoing the same with an overlay is slightly more complicated:
\n\nWhile we could use nixpkgs.overlay
inside the Home Manager configuration
function, we can also apply the overlay in the flake, by defining the pkgs
attribute (in fact, we can do both at the same time). The only caveat is that we will also need to supply the system
when importing nixpkgs
in the flake.
The deploy-rs tool being under pkgs.deploy-rs.deploy-rs
is a consequence of the deploy-rs overlay being structured like that, and not a specific quirk of flake overlay use.
Hopefully these examples give a broad overview of how Nix flakes can be used in practice, but to augment them, here is some extra stuff to read:
\n