Document build dependencies

[spencerking]

I've found that a clean install of Fedora 30 is pretty lacking when it comes to building the project. I'm using this issue to document all of the packages I need to install as I work towards a build. Going forward it may make sense to create a nix or guix configuration, or perhaps split the custom components into their own repositories with clear documentation.

Packages:
llvm-devel
pcre-devel
libevent-devel
patch
texinfo
zlib-devel

[drhuffman12]

Maybe wrap the build in some docker/compose files?

[ffwff]

Yeah I don't use Docker.

Maybe I'll document it in depth later but for now please just view Crystal and GCC's manual for required build dependencies.

[spencerking]

In response to #15:

I never found out what package contains x86_64-elf-as, which is needed to build the kernel.

When trying to build binutils I get the following:

make[1]: *** [Makefile:849: all] Error 2
make[1]: Leaving directory '/home/spencerking/Documents/lilith/userspace/toolchain/binutils'
make: *** [Makefile:13: build_binutils] Error 2

[ffwff]

If you're just building the kernel, you'll only need x86_64-elf binutils, GCC and Crystal; userspace/toolchain/binutils is for userspace applications.

Building patched Crystal fails despite various attempts, probably due to different (and constantly updating) LLVM and Crystal versions. (Edit: I didn't make clean after the initial build failed, screwing up future builds.)
@ffwff Could you tell us which version of LLVM and Crystal you're using to build this? I can't sleep I think I'm gonna cry
Edit: Compiler building works with LLVM 9.0.0 (installed with linuxbrew) + Crystal 0.32.1 (From crystal github releases)

[spencerking]

Building the patched Crystal consistently works for me.

The bigger issue for me is it's really not clear what packages are required for the entirety of the build process (kernel, userspace, the whole deal). This is why I had originally mentioned creating a nix shell. Everyone would be using the same versions of the same packages, making this a non-issue.

I would definitely appreciate that, but I'm sadly not well acquainted at nix to approach that :(
I'll at least try to document the process thoroughly after I get it to work :)

[spencerking]

@Scorbie I would be happy to do it once I can actually build the project.

Okay, I got all the specified requirements ready. I can build the kernel (which doesn't run but that's a seperate issue AFAIK; origin/master is shaky, captain!)

@ffwff Present for your 1000th commit, the text below is yours. Congrats.

The requirements

LLVM 8.0 (recommended by Crystal team at the time of writing)
or 9.0 (worked for me)

• Get it with your package manager or LinuxBrew or nix or compile it.

Crystal
(Note. currently it has to be installed as /usr/bin/crystal)

1. You may get it with your package manager; see https://crystal-lang.org/install/ for details.
2. There are linux binary releases at https://github.com/crystal-lang/crystal/releases

• v0.32.1 works with LLVM 9.0
• I'm not sure which version goes with 8.0, so you need to experiment.
• There may be more hidden dependencies (see first post) that I installed in the process of trying with LinuxBrew (which failed)

Cross-Binutils (amazingly the easiest step for me)

Edit(2020.11): Welp... I was careless and misread the instructions and this tripped me.
Updating the instructions to be clearer (and hopefully more foolproof).

• I don't think there's prebuilt packages available, even for NixOS. Only Arch seems to have it. Source
• Download & Compile Binutils yourself! Recommended instructions here.
• You probably should prepare two copies of the same source; one for each arch.
  (i.e. one for x86_64-elf and the other for i686-elf)
  • configure --prefix ... --target x86_64-elf && make && make install in the first copy
  • configure --prefix ... --target i686-elf && make && make install in the other one
  • Even make distclean doesn't allow you to reuse the same source directory.
    (There are build caches not cleaned) I had to learn this the hard way. I hope you don't.
• Also, when running configure, you have to
  • Either not set --prefix if you have root (root is needed for make install)
    and don't mind the tools getting in your /usr/local/bin
  • Or set it, then Add PATH="<prefix-path>:$PATH" in your sh startup script.
    • The tools are used during make and make launches sh for executing commands.
      That sh has to see the tools in $PATH, regardless of which shell you use.
    • According to POSIX the startup script is specified in the $ENV variable;
      So this is guaranteed to work on all POSIX-compliant systems unless I am mistaken:

      export ENV=<sh-startup-script-path> in your ~/.profile if it's not set
      PATH="<prefix-path>:$PATH" in your <sh-startup-script>
      logout and login to make sure changes are applied.

    • Although on my Ubuntu 20.04, adding the paths to my $PATH in ~/.profile did work for some reason I don't understand.

Building

Of course git clone https://github.com/ffwff/lilith and cd lilith first.

Building the kernel

Actually I'm just reiterating the pretty well documented readme, it just doesn't work It may not build when something screwed up in the process so good luck.

1. Build the toolchain: make toolchain/crystal/.build/crystal
   • Note: if the build fails make sure to at least cd toolchain/crystal && make clean && rm -rf ~/.cache/crystal so that the patched Crystal is not littered with your past failed build artifacts.
2. Build the kernel: make build/kernel

Building the userspace (optional)

Make sure to install bison and flex beforehand, then see my post below for instructions.

Running

• make run - Qemu hangs with no disk found but it runs, at least.
• make disk.img => make run_img - Fails with error, will raise an issue if needed.

I'll try with docker if there's much request (I haven't used docker much so it's quite time consuming)

[spencerking]

@Scorbie I had to edit the Makefile to hardcode the paths to the binutils binaries on my system. Possibly a fish issue, but I don't know yet.

I have gotten the kernel to build, but when I try to make run I get a black screen in qemu and an error in my terminal stating that no disk was found.

Edit: Nevermind, I see you ran into the same issue.

[ffwff]

I'm busy at the moment, but I usually use this command for debugging:

make rungdb_img RELEASE=1

You can also pass NO_CPUEX=1 to trigger a breakpoint on CPU exceptions.