About Containers and Vms

On incus/lxd is true there containers can only be Linux..

Not sure about the one app thing but that’s the general design of those ad well I suppose.

linux containers, be it a lxd container, or a containerd/dockerd one, only run on linux hosts.

windows containers, only run on windows hosts.

when you run a linux container on a windows host, you're actually running a linux container inside of a linux vm on top of a windows host.

containers share the host operating system's kernel. it is impossible for a linux container (which is just a linux process) to execute and share the windows kernel. the reverse is true, a windows container (which is just a process) cannot execute and share the linux kernel

the article is correct, linux containers can only execute on a linux host

Not only that, containers predate Linux implementations, I was using HP-UX Vaults in 1999.

I think they linuxcontainers.org people would disagree. Like the table is trying to communicate, in contrast to eg Docker, this is not about application containerization.

That's literally the opposite of what this documentation is explaining. System containers exist. You can run the entire userspace of an OS (including systemd) in a container.

And a system container makes an OS (or OS userland) believe that that it has the kernel to itself.

I'll have to remember that one!

According to their Github page, they _are_ linuxcontainers (in a way), and Incus is Apache licensed:

Incus, which is named after the Cumulonimbus incus or anvil cloud started as a community fork of Canonical's LXD following Canonical's takeover of the LXD project from the Linux Containers community.

The project was then adopted by the Linux Containers community, taking back the spot left empty by LXD's departure.

Incus is a true open source community project, free of any CLA and remains released under the Apache 2.0 license. It's maintained by the same team of developers that first created LXD.

LXD users wishing to migrate to Incus can easily do so through a migration tool called lxd-to-incus.

https://github.com/lxc/incus

Linux Containers, or LXC, came before Docker and OCI standardization.

As the others have mentioned, Incus is the community fork led by former members of the LXD team.

Are (self hosting) people putting multiple services like Django app, Postgres, Redis etc into a single container/lightweight VM instead of using Docker Compose with single-purpose containers?

incus is the truly open source version of lxc/lxd. It is stable and incredible. I manage dozens of machines and want for nothing, and most importantly, pay nothing for that luxury.

I think you could also add Xen to that list. IIRC, the old Xen PV mode was purely paravirtualized without using any hardware extensions.

In my experience TCG (or any method that doesn't require root / admin power) is pretty slow. But I'd be happy to be wrong about that, for an odd project I have

The article is pretty useless at explaining the difference, I agree. It makes claims about Docker that aren't true (e.g. single container) while making inadequate reference to the OS features likely involved in making "system containers" what they are (SECCOMP, capabilities, network namespaces, nftables).

As an engineer this page has a real "trust me bro" feel to it. Maybe fine as a marketing and product positioning thing, but not interesting for HN.

It mostly isn't. Almost all Linux container escapes only require the ability to make system calls to the shared kernel from processes inside the container. The system container doesn't really restrict this ability. It also increases surface area to compromise the container before attacking the host system, since there's now a bunch of extra software running inside the container.

If privilege isolation is a priority but you want to use containers, gVisor and Firecracker are way ahead of anything else. The Linux kernel API has proved to be very hard to secure, and not for lack of trying.

In the context of Incus, they are the same.

Incus and LXC internally use umoci to manipulate the OCI tarball to conform to how LXC runs containers.

See: - https://umo.ci/ - https://github.com/lxc/lxc/blob/lxc-4.0.2/templates/lxc-oci....

It's not really.

Any shared resource between containers or the kernel itself is an attack surface.

Both options have a very wide attack surface - the kernel api.

Nothing really beats virtualization in security, the surface shrinks to pretty much just the virtualization bits in the kernel and some user space bits in the VMM.

Complexity is generally the enemy of security, because securing a system requires understanding it. If you can build a more understandable, less moving parts, more observable, more easily manageable etc system with system containers, it's a security argument.

It generally is more secure just because the system container virtualization system is "more complete", so it's harder to get out from under it.

My understanding with Incus(the OP link) it's the same virtualization system, so there is no real difference, security wise between the two.

The question then becomes can they get out from under the virtualization and can they get access to other machines, containers, etc.

Docker's virtualization system has been very weak security wise. So a system container would be more secure than docker's virtualization system.

Seems mostly off topic to the article. I think system containers should be implemented in user space. They are not about security theatre but about getting a sandboxed environment which feels like a real/virtual machine but is lighter weight. Very useful e.g. when I want to emulate a whole cluster of Linux machines. And for those needs security is nice but not key.

It is application containers which maybe should be replaced by better kernel security, not system containers.