-- Leo's gemini proxy
-- Connecting to gemini.tuxmachines.org:1965...
-- Connected
-- Sending request
-- Meta line: 20 text/gemini;lang=en-GB
Tux Machines
Posted by Roy Schestowitz on Jun 26, 2023
> This article describes some principles I’ve found useful for designing good Python library APIs, including structure, naming, error handling, type annotations, and more. It’s a written version of a talk I gave in June 2023 at the Christchurch Python meetup.
> It’s been nine whole years since PEP 484 landed and brought us types from on high. This has made a lot of people very angry and been widely regarded as a bad move1. Since then, people on the internet have been clamoring to find out: does this mean we can now compile Python to native code for more speed? It’s a totally reasonable question. It was one of my first questions when I first started working on Python compilers. So can we do it?
> No. But also, kind of, yes. I’ll explain. I’ll explain in the context of “ahead-of-time” compiling within or adjacent to CPython, the predominant implementation of the Python language. Just-in-time (JIT) compilers are a different beast, and are described more below. None of the information in this post is novel; I hope only to clarify a bunch of existing academic and industry knowledge.
> The core thesis is: types are very broad hints and they are sometimes lies.
> Instead of having to limit sanity checks to the boundaries of the program, we could re-use those as function contracts using the assert keyword. Indeed, setting PYTHONOPTIMIZE removes all assert, making the check useful in dev, and free in production.
> Unfortunately, the community doesn't know about the feature, and use assert for things that should never be removed, so using the flag would likely introduce bugs into your program.
> Calling teams “Scrum Teams” implies that effectiveness is defined by Scrum. Spotify shifted to “Squad”, at least partly, as a way to be clearer that teams should not limit themselves to Scrum patterns and practices. Outside of Spotify itself, I’m wary of suggesting “Squad” as an alternative to “Scrum Team”, mainly because of a general tendency to blindly copy Spotify. “Crew” is an alternative from the Cynefin community but it’s fairly niche.
> Take any ID or value in your system. How many computers does it touch? How much time does it spend in HTTP packets? How big and how long does it spend in RAM? How many times is it copied in CPU cache? How is it moved or copied on the program stack? How is it represented in the GPU?
> When designing systems, it’s easy to get caught up in the minutia of each decision that has to be made along the way. Technical systems may raise concerns about choosing the appropriate logging framework or determining the most effective concurrency model. Human systems may instead be concerned with how many meetings to hold, with whom, and how often.
> While individual decisions may be important to the outcome of the project, the ability to adapt and change those decisions in the future is likely to be significantly more critical. Despite careful thought and planning, achieving 100% certainty in the correctness of every decision is highly unlikely. Even if this level of certainty were possible, the Pareto principle suggests that 80% of the overall outcome will be driven by only 20% of these decisions; therefore, investing excessive time to ensure the absolute correctness of every decision would necessarily result in a considerable amount of wasted resources.
> This post may seem a bit harsh, but I’m tired of SQL shaming that has somehow become a thing in the industry. I have a right to disagree, don’t I?
> Every year or so, a new general-purpose database engine comes out. And that’s great! It can bring new valuable approaches, architectures, and tools (plus, building database engines is fun).
> Often this new database engine comes with a new query language. And that’s probably good, too. Or maybe it’s not.
> The only difference is the spelling of the name of the thing we’re calling (which is a function template in C++ and a macro in Rust) - otherwise, identical.
> Nevertheless, there is a surprisingly vast gulf of difference between the two languages in how they handle formatting, at basically every level beneath the user-facing syntax. I thought the differences were pretty interesting and worth going over.
> Earlier this year, Wanhong Huang, Tomoharu Ugawa, and myself published some new experiments on interpreter performance. We experimented with a Genetic Algorithm to squeeze a little more performance out of bytecode interpreters. Since I spent much of my research time looking for ways to improve interpreter performance, I was quite intrigued by the basic question behind Wanhong’s experiments: which is the best order of bytecode handlers in the interpreter loop?
-- Response ended
-- Page fetched on Sat Jun 1 07:00:06 2024