-- Leo's gemini proxy
-- Connecting to nanako.mooo.com:1965...
-- Connected
-- Sending request
-- Meta line: 20 text/gemini; charset=utf-8
I love Crystal. I'm still enthusiastic about Crystal. But, I've had some random musings over the past year and a half of doing intense stuff with it (namely midi123 and Benben). Things that make me go "damn, I wish Crystal had this" or "I really wish Crystal didn't do this". That's what this is about.
This really is just a "wishlist", too. All of these have workarounds. But they're all things I feel they would enrich the language and Crystal experience.
For server-side programs, and for many concurrency needs, Crystal's concept of Fibers and their M:N scheduling works fine. But there's two places where this falls short:
Actual multi-threading. Yes, you can achieve multi-threading with Fibers, but it's clunky, and too strictly tied to CSP. Specifying the number of true threads to use (the N part of M:N) shouldn't be done solely through an environment variable.
The Fiber scheduler doesn't always perform as well as the OS scheduling true threads.
Channels are too academically strict.
No way to instruct how threads/fibers are handled at exit.
As to the first issue, I really don't like the whole CRYSTAL_WORKERS mechanism, as I've stated in previous Gemlog posts. What I would *instead* want is for schedulers to be first-class and controllable. In Common Lisp, we can achieve this with the lparallel library and its concept of a "kernel". It spawns a requested number of threads (and not through an environment variable, which should never be used for parameters), then schedules tasks over those threads. In all my calls to the library, can either use the current default kernel (lparallel:*kernel*), or I can create my own and use them in a way similar to a thread pool.
Of course, I don't always want to schedule M number of tasks over N number of threads. Sometimes I just want a damn thread. For example, Benben's UI, player, and overall manager each run on their own threads in both the Common Lisp and Crystal versions (though see the next paragraph for how this was done in Crystal). These are very, very separate processes that don't need to care about what the other processes are doing. They just send messages.
Also of issue is the absolutely **UGLY** kludge I had to implement to ensure that the correct number of true threads were spawned when Benben starts. It essentially overrides CRYSTAL_WORKERS to ensure a good minimum of threads are started. Because if not enough true worker threads are started, the program will either deadlock at certain places, or perform horribly.
And while CSP is absolutely amazing, there are some times where I don't want my messages sent through channels to be delivered in deterministic way. Sometimes I actually *want* that "fire and forget" functionality of a more general (and optionally unlimited sized) message queue. I also had to over-compensate with channel capacities to make sure those don't deadlock, because Crystal has no unlimited capacity channels (I almost implemented my own for Benben, and other programs I've written in the past).
And that's where it crosses over into the second and third issues. Between the scheduler and the lack of unlimited-sized channels, Benben's performance is hampered. If I render a very large number of songs to WAV (an embarrassingly parallel operation, which is why it's truly parallel in Benben), using my desktop (20 logical cores), and I watch the CPU graphs in htop... I see them regularly stall out and drop below 95% usage. In fact they often drop down to 5% for a few moments in groups and stay there for a half to a full second. Somewhere, the scheduler and/or channels halt the program flow when they don't need to. When I recently reimplemented the exact same design in Common Lisp using true threads and a better scheduler (lparallel's kernels), the CPUs stayed at 95%+ the entire time. Well, as long as I had >20 songs to render. That's the performance I would expect from something embarrassingly parallel.
And finally, one thing I find myself missing from SBCL specifically is how it can handle threads at shutdown. Depending on how I call SB-EXT:EXIT, the runtime will either immediately kill off threads, run exit hooks, wait for a timeout and then kill threads... etc.
So, wishlist item 1: Better control over threads. Give me something akin to a kernel in lparallel where I can setup and control my own scheduler, and true access to threads.
Yeah I know, "goto considered harmful" and such... but that's simply not true. Misuse of goto is harmful, but goto itself is not.
While porting Doom to Crystal (still not finished due to elusive bugs), I noticed a lot of cases in the original C# source (and C sources) I was working from where goto was used. These uses were excellent examples of where goto can actually be good, too, such as breaking out of nested loops to go to an end condition. It was actually *much more readable* than what I had to do to work around the lack of goto. A named break statement might also be nice here, but hold up just a second.
CPU emulators are whack. Even ones for virtual CPUs are whack. Here's a couple of examples - do a ctrl-f for "goto" on these, see how many hits you get:
YunoSynth actually started out as a port of libgme, not of the emulators in VGMPlay. That port was abandoned because of the sheer undertaking it would be to remove the goto statements from that code, and the bloat it would cause.
Of course, goto can be severely abused. So why not take a middle-ground? Common Lisp has a goto (GO), but it's always scoped (TAGBODY) and the way the scoping works is clearly defined.
Anyway, wishlist item 2: A goto statement.
Having a very nice set of intrinsics for SIMD stuff in the standard library would be AMAZING. Going back to SBCL yet again, its recent SB-SIMD package is downright sexy and amazing. SIMD can be a huge speedup for audio stuff in particular, and a huge number of other fields as well. I've been tempted to just do my own personal "good enough" SIMD library for Crystal in the past, but I haven't quite wrapped my head around the inline assembly syntax in Crystal. Chalk that up to being used to Common Lisp, I guess.
So wishlist item 3: SIMD intrinsics in the standard library.
Imagine this: you're working on a large program. You compile it and see no compilation errors. But when you run it, the program doesn't work right (or crashes). The culprit? You made a simple typo when setting a variable.
Stuff like this isn't fiction. It has happened to me multiple times, and in all of those cases, it was a huge pain to debug. This example code is very short, so imagine that it's actually inside some much more complex code... but it illustrates my point:
variable1 : Int32 = 0 variable2 : Int32 = 1 # ..some stuff with the code happens... variabel2 = 69 # Whoops, you just declared a new variable # ...mode code, but variable2 now has an incorrect value
Yes, it's nice to be able to declare variables without types, but I *really* wish I could disable that in the code I'm working on (maybe a declaration at the top of the source file?), or opt in to a true "var" keyword. This type of error cost me multiple days of headaches in both CrDoom and Benben, and the language just lets me do it. Thank the gods I don't write mission-critical or safety-critical code.
Wishlist item 4: opt-in "var" keyword, or opt-out type inference on a per-file basis.
I have a confession: I hate LLVM. Yes, it produces very nice and blazing fast code, but good LORD can it be slow and memory hungry. This is one reason I continue to prefer GCC when compiling C/C++ programs. Crystal's compiler is generally Fast Enough™ in my opinion, except in the stage where the LLVM stuff happens. That part is agonizingly slow. There's been some work in Crystal 1.11.0 with optimization levels that helps, but truth be told, I'd rather just have multiple backends to chose from. Especially since I've also run into a situation in the past where I couldn't compile the compiler because it dropped support for the LLVM I had at my disposal in my operating system.
So the final wishlist item: Multiple backends, especially a GCC backend.
--------- Page served by Aya https://nanako.mooo.com/fossil/aya/ Aya is under the GNU Affero GPLv3 license
-- Response ended
-- Page fetched on Sun May 19 23:21:22 2024