What Should the Experience of Building Rebol Be Like?


#1

When R3-Alpha was open-sourced, the make directory contained one GNU makefile, and Carl gave these instructions:

About the Makefile:

You might notice that the makefile is a bit old-fashioned. That’s because I still support REBOL on some very old systems that don’t offer newer makefile features. So, please keep that in mind and stick with this simple format.

Also, the makefile is built by REBOL. Typing “make make” will rebuild it, and you can also select a different platform target the same way by providing the REBOL platform identifiers (the last two parts of the version string).

Build Instructions:

Parts of REBOL are built by REBOL. So, to build it, you’ll need to download a running binary into the local make directory. Call it r3-make.

The build happens in the make directory. It will create an obj sub-dir for storing the object files. I prefer this over mixing the source and object files into the same directory.

The biggest step is to do the “make prep” which will use REBOL to configure and build a number of important C header files.

If you are building for a new platform that has no existing REBOL, you can use “make prep” to build all the files you need on an existing platform and copy them to the new platform.

After the prep, just run “make” and the rest of the system will build.

One might ask: if the makefile is built by Rebol, why would it be committed in the repository? What platform is this makefile for? What happens if you just say make?

The answer is that the makefile was generated for 32-bit Linux (0.4.4), so it was produced by running the command r3-make %../src/tools/make-make.r OS_ID=0.4.4. However, it contains a trick: it has a target itself called “make” such that once generated, running make make OS_ID=0.X.Y will overwrite the makefile itself with an updated version for that OS.

We found this annoying, because the committed makefile would appear dirty in version control if you are using another system. And also, one could streamline this process a bit for the average person if instead of them having to go look at the %systems.r table and find their platform, if the r3-make could inspect its own system/version and assume you wanted to build that same OS_ID.

And so %make/makefile.boot was born. It was a committed file which saved you all the trouble of typing ./r3-make ../src/tools/make-make.r followed by make r3. Instead you could now type make -f makefile.boot.

If it sounds stupid, it kind of is. But it wasn’t rocket science to write, and you could also think of it as a kind of an interactive README file…if you didn’t have something you needed, it could tell you. And given that the build process was already dependent on you having GNU make, it could also catch you on that “meta-step” of needing make in the first place.

But we live in a different world now. You no longer need GNU make to build Ren-C. You can still target it, but you can also generate files for Microsoft’s NMAKE, or a Visual Studio project file. And you can even forego the idea of a makefile altogether–instead having the Rebol executable drive the build process using CALL on the C compiler directly. So it’s time to scrap %makefile.boot and let a Rebol script be the entry point to the process.

I’m going to be talking about platforms, building, extensions in some more technical posts. But this is more a philosophy post, or user-experience question. Maybe any points people want to raise about building a system where it’s been done right?

One thing we definitely should make easier is bootstrapping to new platforms. The generated products of a Rebol “prep” process need to be in one central place to be grabbed and moved over. Bootstrapping in this way should be part of the test process, e.g. one of the Travis instances that does a Win32 build on Linux could do the Win32 prep using the native Linux executable. And then make sure it can just move the one folder with prep products into a completely fresh git install and still build with the cross-compiler…


#2

Some thoughts:

What are the challenges of getting it included in package managers? While it’s desirable to download a binary and that’s all the installation you need, package managers—particularly ones as slickly marketed as Homebrew—create the impression of an even easier installation process with two tangible features: consistency and immediacy of installation (don’t need to monkey around getting the binary into /usr/local/bin and chmod); and is actually a build—not a prefab binary.

curl -O <path-to-latest-binary>
chmod +x <latest-binary>
mv <latest-binary> /usr/local/bin/

Vs.

brew install ren-c

What are the downsides of using the Rebol->Call control? Red’s approach to cross-compiling is appealing (if less nuanced than the platforms breakdown in %systems.r): red -t Windows hello.red


#3

I must miss something, because although I was able to build original Carl’s Rebol quickly on Mac and Windows, I’m not able to do the same with the Ren-C repository. I wonder how you use it, because I don’t consider it to be friendly at all.


#4

Hm… probably found the main issue… one must create manually the objs directory. Now it compiles something.


#5

Also other directories under objs must be created manually and there is many of them inside.


#6

I never said it currently was. Far from it: I explicitly said that Shixin had kind of gotten partway through a fairly large project to overhaul the build system such that it didn’t depend on GNU make or CMake. He chose not to write the code to be compatible with R3-Alpha, and instead made it use a version of Ren-C that he had snapshotted.

(I’d have preferred it work in R3-Alpha, still, as we’d historically worked pretty hard to do that. But it was somewhat validating that he considered it too annoying to have to go back in time and code for R3-Alpha…that the features and increased reliability of Ren-C were worth it.)

Unfortunately, other tasks called him away before even updating the README–or leaving any instructions. I don’t especially enjoy working on build systems, and ideally I would have his participation rather than being left to sift the code and reimagine it on my own. So I have been waiting for that to happen, and it hasn’t, and now you are asking questions.

But given that I have told you this is the situation, the ideal thing would be to come and ask in chat–say what you are doing–and ask how to go about it. Bumping around by yourself is inadvisable, and a good way to waste your time.

In any case, me asking this question is specifically about how we can make the experience better and what expectations people have (realistically) or good features from building other systems. Indicating I think it needs a top down review and tidying from its current state, or even its historical state.


#7

I don’t say that you said, that it is friendly… it is my opinon after trying to build the Ren-C in comparison to original R3 building… After manually making all the folder tree in the objs directory, I have builded r3.exe, but unfortunatelly it halts with error: Bad composed integer assignment for byte-ordering macro.

Btw… I was going thru the commit history and it is very interesting how quickly (even durring the time when Carl was still active) was the original Carl’s approach messed up in the Saphirion version, which was merged with Atronix later
.


#8

Hey, wow. Well, you’re right. Guess I wasted all that time. R3-Alpha was better all along!

So why don’t you go develop on that branch? You can add features to LOOP.

Meanwhile, we’ll be wasting time with our “complicated” Rebol-powered C parser (a useful tool in itself, and good test code) we can do little things like edit our native specs right in the C comments, and use names to access parameters…still by index. And wasting time solving problems like all those issues that I brought up in the slides.

And of course there’s the ability to use an embedded TCC compiler to build natives from strings, if the performance doesn’t satisfy.

Well, those don’t matter. Because Carl had it all sussed out. So go knock yourself out. Because you’re not going to get any help from me following the path you have followed.


#9

Take it easy… I was just asking how you build Ren-C on Windows… because I don’t have success with it. And if there is question in the title of this topic: What Should the Experience of Building Rebol Be Like? I just answer that it should be friendly and successful.

And instead of some constructive info I got reply that I will not get any help, because I was trying also the original R3 repository. Which is also not too friendly btw.


#10

These build steps were working for me as of July (and I think since then as well) using MinGW. Not sure why you have to create obj directories, but you can see what function it’s calling to create them and reason why that function isn’t being called correctly.

First, modify a config file %myconfig.r. Mine is simply:

REBOL []

os-id: 0.3.1
rebol-tool: %r3-make.exe
toolset: [
    gcc %mingw32-gcc
    ld %mingw32-gcc; linking is done via calling gcc, not ld
    strip %strip
]
with-ffi: no

Then from my console, I run:

r3-make.exe ../src/tools/make-make.r CONFIG=configs/myconfig.r TARGET=execution

I manually delete r3.exe and the objs/ folder before building. Or you can run TARGET=clean (not done automatically)


#11

Thank you… this was helpful… I was able to make a build using this method.