I am working on a project that will be like a whole environment
bootstrapped from the smallest possible beginning.
I thought a hex-to-binary converter like hex0 was too much like
machine code, and I wanted everything to be source code, so I
chose to instead start with a super simple interpreted language
that is described in english and the user is supposed to write
an interpreter for it themselves in any way they want. This
way I connect the human to the computer using the language of
the human which I thought felt better and easier to understand.
This super simple interpreted language then builds a super
simple RISC-V assembler. Here I came up with a nice technique
where the functions in the assembler are called through
pointers, which makes them replacable while the asesmbler is
running. So as soon as the assembler has compiled a better
version of one of its own functions, that function is then
replaced with the better version while the asesmbler is running
and it continues to compile the rest of the code which can then
use the new features. So it evolves without having to restart.
I still have a small fee available from NLNet for the implementation of a RISC-V back-end for a compiler (more a transpiler) from a small stack language to assembly. I have already started the effort, but at this moment not the time to work on it. I am available for support. Back-ends to x86, x86_64 and arm64 are available and working. This is part of the MES-replacement project [1]. Let me know if you are interested (see email at the bottom of the website mentioned in my profile here or on GitHub.)
We bootstrapped our 100% deterministic and quorum-signed LLVM/musl container/native distribution this way from day 1 thanks to the incredible work of this team. Bootstrappable builds unlocked stagex, which unlocks remotely attestable boots that can walk the trust chain from a live server boot all the way back to hex0.
> The hex0 program provides a way to turn a string of hexadecimal text into a binary with those bytes.
I'll use this to ask for an explanation about the fundamental idea: How is hex text better than a binary blob? Both need detailed knowledge to understand; both need a tool to display (reading ASCII or binary), either of which could be compromised in a "trusting trust" sense.
It seems to me that the actually important aspects of the initial "seed" are its size (larger is harder to verify) and that the language used (whether it is Lisp, binary machine code or whatever) has rigorously defined semantics.
(Orthogonal to that, you'd want to store everything, seed and the rest, on a medium that cannot hide anything from you, and use that as the source. But that applies to all approaches.)
Would love to see someone try to automate the bootstrap chain from a working C89 compiler to Rust.
At this point I think current LLMs are able help these incredible feats of bootstrapping as they can grind out the impossibly long built times over multiple days/weeks.
I am very optimistic for deterministic builds in general.
I am working on this![0] But this is actually a herculean task, I've been chipping at it for weeks and even trying to cheat somewhat by translating mrustc from C++ to C89. If anyone has ideas/cheaper ways to run it I would like to know.
In the meantime, months ago I achieved bootstrap of TCC (then GCC 4.6) via the same path as the live-bootstrap project but replacing the Scheme part with a C compiler in Ben Lynn's Haskell dialect.[1] This was a project I wish I had time to do years ago[2] but never had the time or motivation for, and the type of things agents can do well now (they couldn't write good Haskell a year ago).
> Would love to see someone try to automate the bootstrap chain from a working C89 compiler to Rust.
We did that in stagex over a year ago actually and several major orgs are using it in production. Also no dependency on libgcc. LLVM native. We presented it at Open Source Summit and Linux Security Summit this year.
I am working on a project that will be like a whole environment bootstrapped from the smallest possible beginning.
I thought a hex-to-binary converter like hex0 was too much like machine code, and I wanted everything to be source code, so I chose to instead start with a super simple interpreted language that is described in english and the user is supposed to write an interpreter for it themselves in any way they want. This way I connect the human to the computer using the language of the human which I thought felt better and easier to understand.
This super simple interpreted language then builds a super simple RISC-V assembler. Here I came up with a nice technique where the functions in the assembler are called through pointers, which makes them replacable while the asesmbler is running. So as soon as the assembler has compiled a better version of one of its own functions, that function is then replaced with the better version while the asesmbler is running and it continues to compile the rest of the code which can then use the new features. So it evolves without having to restart.
I still have a small fee available from NLNet for the implementation of a RISC-V back-end for a compiler (more a transpiler) from a small stack language to assembly. I have already started the effort, but at this moment not the time to work on it. I am available for support. Back-ends to x86, x86_64 and arm64 are available and working. This is part of the MES-replacement project [1]. Let me know if you are interested (see email at the bottom of the website mentioned in my profile here or on GitHub.)
[1] https://github.com/FransFaase/MES-replacement
We bootstrapped our 100% deterministic and quorum-signed LLVM/musl container/native distribution this way from day 1 thanks to the incredible work of this team. Bootstrappable builds unlocked stagex, which unlocks remotely attestable boots that can walk the trust chain from a live server boot all the way back to hex0.
https://stagex.tools
https://codeberg.org/stagex
https://distrust.co/blog/enclaveos.html
https://caution.co/
> The hex0 program provides a way to turn a string of hexadecimal text into a binary with those bytes.
I'll use this to ask for an explanation about the fundamental idea: How is hex text better than a binary blob? Both need detailed knowledge to understand; both need a tool to display (reading ASCII or binary), either of which could be compromised in a "trusting trust" sense.
It seems to me that the actually important aspects of the initial "seed" are its size (larger is harder to verify) and that the language used (whether it is Lisp, binary machine code or whatever) has rigorously defined semantics.
(Orthogonal to that, you'd want to store everything, seed and the rest, on a medium that cannot hide anything from you, and use that as the source. But that applies to all approaches.)
Would love to see someone try to automate the bootstrap chain from a working C89 compiler to Rust.
At this point I think current LLMs are able help these incredible feats of bootstrapping as they can grind out the impossibly long built times over multiple days/weeks.
I am very optimistic for deterministic builds in general.
I am working on this![0] But this is actually a herculean task, I've been chipping at it for weeks and even trying to cheat somewhat by translating mrustc from C++ to C89. If anyone has ideas/cheaper ways to run it I would like to know.
In the meantime, months ago I achieved bootstrap of TCC (then GCC 4.6) via the same path as the live-bootstrap project but replacing the Scheme part with a C compiler in Ben Lynn's Haskell dialect.[1] This was a project I wish I had time to do years ago[2] but never had the time or motivation for, and the type of things agents can do well now (they couldn't write good Haskell a year ago).
[0] https://github.com/siraben/mrustc/tree/siraben/cmrustc
[1] https://github.com/siraben/blynn-bootstrap
[2] https://logs.guix.gnu.org/bootstrappable/2021-05-26.log#1438...
> Would love to see someone try to automate the bootstrap chain from a working C89 compiler to Rust.
We did that in stagex over a year ago actually and several major orgs are using it in production. Also no dependency on libgcc. LLVM native. We presented it at Open Source Summit and Linux Security Summit this year.
Clone https://codeberg.org/stagex/stagex and run "make core-rust".
Will build "FROM scratch" in containers from hex0 to recent rustc.
Are chain-of-trust attacks real? Is it necessary to make so much effort to avoid what supposedly never happened?
I think the security aspects are overblown... but being able to edit the code of any part of your system is very useful.