Operating System

[bvisness]

Modern desktop operating systems are terrifyingly bad and bloated:

• Windows is enormous, has fragmented UI (getting worse with each new release), doesn't respect the user (forcing updates and other anti-user practices), and has now started putting ads in random places. Microsoft has started requiring all users to have Microsoft accounts in order to use the operating system at all.
• Linux is still a joke, with a massively fragmented ecosystem and poor hardware support. (See: the endless memes of "Linux on the desktop")
• Even the legendarily great macOS is getting worse, with higher memory costs, everyday operations being juddery and slow even on modern machines, and swaths of nonsensical UI departures designed for iPads.

We think an operating system should:

• Be small and light on resources.
• Always prioritize the user, not the programmer or a corporation.
  • The user gets to decide when the computer starts up and shuts down. The user gets to decide when updates install. The user gets to decide what is running in the background and using up their hardware resources. The computer exists to serve the user, not the other way around.
• Work out of the box on all popular consumer hardware.
  • Sound should just work. Wi-Fi should just work. Monitors should just work. Graphics cards should just work.
• Never require the user to open a command line.
  • Normal people on Windows and macOS have been able to use their computers for decades without touching a command line. There is simply no conceivable reason that a normal user should have to interact with a modern computer in this way. The command line is a relic of the past and should only be interacted with by programmers who need it in order to interact with other systems.
• Make it a joy to write graphical applications.
  • The web has succeeded largely because it is easy to get stuff you care about on screen. This should be an achievable goal for a modern operating system without requiring the entire web stack.

There are also other problems with current operating systems that may be worth addressing:

• The application and permission models of today's operating systems are a poor fit for a world in which people download and run untrusted software from the internet. Browsers and mobile phones provide compelling models for how this might be done differently.

Making a new operating system is a very large and ambitious goal. But, it is a goal worth pursuing.

Handmade projects

• Essence, by nakst (website, HMN project)

Other notable projects

• SerenityOS (website)

[dortamiguel]

Is this going the be made? I will like to work on this, seems interesting

[kernel-dev]

There are issues with some of these aspirations that I would like to address:

• (1.) "Be small and light on resources"

  • (1.a) While it is true that it would appear that modern operating systems take a tremendous toll on hardware and the limits of it's capacity, I do not think, that in most cases, it is the OS's fault itself. There are a plethora of dekstop applications built using Electron to make it more accessable to those who do not want to tackle low-level programming. When Electron apps are designed poorly, they are a huge hog on memory resources, and this holds true for applications such as Discord. Whilst, if Electron applications are designed properly, they are much less intensive on memory - take for example VSCode, it is not a memory hog, but it is still an Electron application.

  • (1.b) Another example might be GPU or CPU usage. These usages, IME, rarely occur because of the operating system's internal tasks (okay, I will mention that Windows's Anti-Malware executables are prone to show high usages, meanwhile that may not be the case, as even Task Manager may not report usages accurately for each process/task).

  • (1.c) You can indeed create a high-scale OS with the size of only a few gigabytes, and not use up the CPU or GPU as heavily. Linux is a decent example of this, although I have other issues with Linux that are not tied back into this necessarily.

• (2.) "The user gets to decide what is running in the background and using up their hardware resources. The computer exists to serve the user, not the other way around."

  • (2.a) While this may sound like a good idea at first glance, I do think the primary duty of an operating system is for it to be securely accessible to the user, without it encountering many issues during it's runtime/usage. You could argue that Linux departures from this, but Linux is inteded for those who wish to tackle more challenging and advanced concepts of IT. In Linux, the security of your system is as good as the user makes it.

  • (2.b) The previous point I had mentioned is also a dilemma of sorts. You are left with the choice of providing no integrated implementation of advanced security features, or integrating it within the OS with all fresh installations. You could optionally allow the user to disable the default security features set in-place, and allow for the user to develop their own, but this would, I imagine, be a massive challenge to accomplish smoothly.

  • (2.c) However, you do mention this issue later in the message by stating: "The application and permission models of today's operating systems are a poor fit for a world in which people download and run untrusted software from the internet. Browsers and mobile phones provide compelling models for how this might be done differently"; which is understandable, but I do think my previous points still hold value.

• (3.) "Work out of the box on all popular consumer hardware."

  • (3.a) I do not understand why the operating system shouldn't support non-consumer level hardware? You mention that X, Y, Z, etc. should "just work," but this may go directly against your own initial wishes. I think the initial idea is sane - where only consumer-level hardware is met with support, but I do not think the appeal of "X just working" is accurate enough to what this is meant to represent.

• (4.) "Make it a joy to write graphical applications."

  • (4.a) This is what no operating system out there (so far) has been able to conceive in a realistic manner. It is an utter pain to provide such an abstract array of utilities and libraries to make writing graphical applications so easy and mundane. It is merely impossible without having some drawbacks.

[bvisness]

You can indeed create a high-scale OS with the size of only a few gigabytes

Essence (linked in the original post) takes up only a few megabytes. This is one of the reasons I say a Handmade operating system should be small and light on resources. Obviously programs can bloat things too, but my computer happily takes many gigabytes of memory when it's doing nothing these days.

Re: the user having control: an important part of giving the user control is restricting the power of programs. What we think of as "security features" are, broadly, just tools allowing the user to see and restrict what their programs are capable of.

Re: hardware, I don't know where you got this impression that it should only support consumer hardware, whatever that means. I just mean that people should be able to install it on their computers, and it should work. Plain and simple.

And finally I don't know why you say it's impossible to provide a good app graphical app experience, when people happily used win32 for years, and they happily use the browser today!

[kernel-dev]

but my computer happily takes many gigabytes of memory when it's doing nothing these days.

It’s not really doing “nothing” though, it’s running various services which support the graphical interface, networking, audio, etc. - what we think of as “nothing” is likely that none of our applications that we use on a daily basis are running adjacently to these. If you didn’t have the OS run processes such as drivers, and even other core parts of the kernel, you’d be unable to use the OS in such an interactive state. Also, as I said, the reported resources used may not be accurate, since it’s quite hard to get a fairly good estimate even. That also isn’t to mention caching, which does tend to take up great amounts of memory, but is relatively useful to the OS.

I don't know where you got this impression that it should only support consumer hardware, whatever that means. I just mean that people should be able to install it on their computers, and it should work. Plain and simple.

Well, it says that in your original post: “Work out of the box on all popular consumer hardware.”
And I don’t think it’s realistic to support ALL hardware out there, excluding popular consumer-level hardware.

[kernel-dev]

I also want to say that, if it weren't for 64-bit, we would still be stuck on 512MB of RAM as our maximum.

[kernel-dev]

You should also consider the fact that some OSs, such as macOS, will indeed be snappier and a more enjoyable experience for the user, but this is because it's proprietary to the hardware they chose to support. If you want to make an OS that supports all hardware, you're going to be disappointed very quickly with its efficiency.

[mmozeiko]

Before, almost every processor was single-threaded, OSs only had a TUI/CLI, and even the OSs that had a GUI were abhorrently slow, as is the expectation of weaker hardware at the time.

Don't know what OS'es are you talking about, but, for example, in beginning of 2000s you could use Windows 2000 with 256MB or something RAM just fine. It was responsive and fast, very stable and not much different (in sense what tasks you used it for, like web browsing, running IDEs for development, or gaming) from any newer Windows version (except uglier and laggier UI nowadays). Almost every process was multi threaded and was able to use however many threads it wants.

[bvisness]

You clearly cannot be bothered to watch the Essence demo I keep mentioning, so I will give you the link: https://media.handmade-seattle.com/essence/

In the demo you can see a broad array of modern UI and modern applications, and they are clearly quite responsive. The kernel is using less than 200MB of memory, as seen when he demos the task manager. The demo is running on a real laptop that nakst owns.

It is also noticeably faster in common operations than my 2019 Intel MacBook Pro, which you claim is magically better because Apple's products are proprietary.

Everything you are saying is trivially refuted not only by a simple glance at the history of modern operating systems (e.g. older versions of Windows) but in fact by existing projects being worked on today by people in this community.

[kernel-dev]

I don't quite understand what you're saying about the demo of Essence, since I have seen it and actually ran it in VBox myself. It takes up 50%+ of the CPU (or so it reports) just to render my cursor on top of another window lol.

I'll go ahead and install it on bare metal to see if the same results are yielded.

[guitarvydas]

If we're going to reinvent the wheel, then I suggest that the concept of "operating system" should be reinvented.

My notes about Operating Systems, in screencast form: https://beta-share.descript.com/view/gdfwt4MfKjF

divide-and-conquer - developers vs. end-users
divide-and-conquer - 2 main technical aspects of operating systems
bloatware
processes
(disclaimer: I'm toying with Descript beta to produce tech video. This is my first attempt and might be unduly klunky)
(disclaimer: I'm still mulling on what to do for the jam. There is a strong chance that I won't be able to decide before the jam ends.)
(ps. I would be happy to discuss this stuff with anyone who is interested...)

[bvisness]

As I recall, you posted some similarly "rethink everything" comments as your submission for last year's jam, and no one had any idea what you were talking about: https://handmade.network/forums/jam/t/8115-jam_submission_-_deprecating_programming_languages_and_operating_systems

This discussion forum will work best with written conversation, so while the screencast is cool and all, I would encourage you to at least try and summarize your main points in a way that people can read.

[bvisness]

Also, I watched the screencast and don't really understand your point. It seems to me like a hodgepodge of mostly-unrelated ideas. I'm not sure how one would even continue the conversation.

What is your actual vision for an operating system? It's nice to rethink things, but what might you actually propose an overall operating system design to look like? There's not much here to discuss right now.

[guitarvydas]

(Thanks for taking the time to watch/read and comment. I have tried to dial in some of what I learned from comments here and in the past, in my submission today).

[Androthi]

should run with every hardware out of the box is easier said than done. linux has hundreds (if not thousands) of people trying to make linux compatible with all hardware for the past decade with limited success. the reality is, without the help of the hardware developers, this task is an impossibility, especially for a small team working on a new OS that nobody is using.

i have an idea for a 'modern' os that i'll just spitball here.
it's a DOS like operating system that is an operating system server.
what this would theoretically be is a boot loader, filesystem and network server, call it a DONS. you can install "operating systems" images on it. and the boot loader will give you a list of OSes (or self-contained images) you have installed and you pick the one to start.
the DONS will require polling for file system and network access. it will give all the system memory to the loaded image (except for whatever it needs for itself). the DONS will theoretically have shared device drivers that any of the images can use directly once they are started.

these images can be DONS fs image files or fully installed on internal/external HD/flash/SSD, etc partitions.
these image-operating systems will have close to ring-0 access to your machine, and they would poll the DONS for file system and network access. they would otherwise have full control over hardware drivers/devices and memory, but they would have access only to a virtual file system given to them by the DONS.

how would this look? imagine turning on your computer, the DONS will boot up, scan all the plugged in drives for any DONS (or non-DONS) images and populate a bootlist with all the available images. these images can be almost anything. so you could have as an example:

1. GTA V
2. Call of Duty
3. Ubuntu
4. Windows 11
5. DONS-Electron

of course, this would require those developers to support DONS. if they did, their games would have near bare metal access and run with much better performance than on an OS.

for the ubuntu and windows 11, DONS would behave like a standard boot loader, just handing off the system to those OSes. they will typically be installed on their own partitions as usual.

[guitarvydas]

I wonder if Justine Tunney's APE has relevance to your idea...

[Androthi]

i don't quiet know what sorcery justine is using, but i'm intrigued. i'll definitely be looking closer into the blog and relevant posts.