Anfang

Themen

Artikel

• » Project and package organisation in Scala (and Java) (7. April 2025, en)
  Scala applications can become a mess if you don’t pay attention. Over time, I’ve settled into a few patterns that help keep your project layout clean and maintainable. The foundation of this post is based on Global organisation, local chaos » package structure is global organisation, what you do inside a class or object depends on the problem at hand....
• » Modelling domain and application errors in Scala (23. November 2024, en)
  Software projects can get out of hand quickly. There are various reasons for it and I picked one reason that can slow engineers down considerably. Often it goes unnoticed: Error Modelling Perhaps it starts even earlier: the lack of thinking about error and exceptional conditions. Engineers gloss over this topic all too often in the first stages of a new...
• » Managing your Python project with a Makefile (16. Juni 2024, en)
  Python projects are nowadays quite straight-forward. If you use VSCode or PyCharm, the complete setup and handling of the project is taken care of. Still, I tend to keep a Makefile around, because I grew up using the terminal and somehow cannot let go Sooner or later you want to integrate parts of your deployment into a CI/CD pipeline CI/CD...
• » Streamline Airflow's SparkSubmitOperator (11. Februar 2024, en)
  In Airflow deployments that use (Py)Spark to crunch data, you might encounter the SparkSubmitOperator as operator of choice. It is a wrapper around spark-submit. By using this operator you can specify a target script that will be executed by spark-submit. In most cases this is a Python script executing PySpark code (see full code here): # dags/customer_ingestion.py from airflow.models import...
• » Don't use Poetry, if you don't have to. (10. Dezember 2023, en)
  In my brain the title translates to »basically never use Python Poetry«. But why? Python Poetry is a neat tool. It fixes the majority of dependency issues, adds some bells and whistles at the side and seems to be “hip”. But my gut says: I’m not sure. Huh? »Why?« I might ask, but the thing with gut feelings is that...
• » An alternative to sqlcmd and more: bulk insert csv data into Microsoft SQL Server (22. Oktober 2023, en)
  The Microsoft SQL Server Command Line Tool, aka sqlcmd, is .. working, in … a way, somehow, but especially the csv bulk import, not as I expected.
• » Secure yourself against Python dependency confusion (6. August 2023, en)
  Dependency confusion is a tricky business in Python land, especially if you are an organisation that maintains a private Python package repository. There are not many options. The obvious first: by specifying --extra-index-url, pip will contact the extra index and the official index on pypi.org, check the versions and install the »highest« one. I did not look up what happens...
• » Either-like result type in Typescript (30. Juli 2023, en)
  Hmmm, Monads and the delicious Either type. Smells functional, smells Haskell, smells Scala, smells complicated. But it isn’t!!!. Lesson number one: Stay away from functional terminology. It just messes with you. The Either type allows you to deal with valid and invalid values in a unified way. An Either is like a garage with two doors where you can park...
• » Azure pipeline container jobs with non-root Docker containers (10. Juli 2023, en)
  Once in a while you may need to run a container job in an Azure pipeline using a custom container. At the same time you need to pass the vulnerability scan that flags containers running as root.
• » Fennel lang memory aid (3. April 2023, en)
  I don’t use Fennel a lot, so when I want to use it, I usually need a quick refresher. Here I’m collecting stuff I always forget.
• » How to tame your Python codebase (3. Februar 2023, en)
  Sooner or later you will reach the point where you realise: You look at your code base and realise: I’ve created a mess, or at least, contributed to it. You change one thing here and ten other things start failing over there. Welcome to the Big Ball of Mud.
• » Why you should ditch YAML and use Jsonnet instead (14. Januar 2023, en)
  I don’t know exactly when the realisation came, but looking at the YAML format and how it is used: it must have been one of the devil’s masterpieces. Similar to flies attracted to a huge pile of cow excrement, developers, me included, are pushed by an invisible force towards this format. It could have been so easy. I mean, there...
• » Build your own custom protocol handler for MacOS (7. Januar 2023, en)
  I keep my personal wiki/knowledge base in Markdown. Usually it is enough to edit the files with a text editor. I also like to have my wiki as static pages served locally. There are plenty of solutions out there, such as Hugo or MkDocs. However, the transition from editor to webpage to editor is not seamless and I decided to...
• » Retrieve pass store secrets in Python, Bash and Perl scripts (17. Dezember 2022, en)
  One of the major advantages of Pass: The Standard Unix Password Manager is its interoperability. You can basically call it from anywhere. Basically also means that there might be a catch: I use the ncurses version of pinentry on the console. If you call pass from a script it needs to be able to start pinentry and show the dialog...
• » Mocking nested calls of a Spark instance (or dataframe) (26. August 2022, en)
  Python gives you nice tools to test your code. Most prominent are pytest and unittest.mock. There you can find all info you need to write python tests and mocks. Still, sometimes you might wonder: how should I test this Spark function? Here you might find an answer! First get acquainted with unittest.mock: unittest.mock — mock object library unittest.mock — getting...
• » Paste the unpastable: how to paste in RDP sessions that don't allow pasting (24. Juli 2022, en)
  NOTE: This solution works, but has its quirks. Enjoy with care! Some companies, notably banks, put security on top priority. There is nothing wrong about it. I even support it. However, if taken too far, security becomes an unnecessary burden. Setting the stage You are connected via RDP to a VM, aka secure environment. Copy and paste is disabled. The...
• » Tree-shakeable enums in Typescript-based ES modules (14. März 2022, en)
  NOTE: If you are in a Typescript-only codebase, you probably can just use union types instead of enums. Typescript has enums, but this feature (currently) has its shortcomings. What? Shortcomings? Not in my code! Actually, it is more about compatibility. Typescript transpiles to JavaScript which (under usual conditions) will end up in your app and finally as bundled JS in...
• » YAML tricks (27. Januar 2022, en)
  YAML is a tricky format and can lead to surprises.
• » Code Quality Essentials (26. Dezember 2021, en)
  My most important and basic recommendations on code quality, which hopefully result in a high quality code base.
• » Python IO and Serialisation Snippets (8. August 2021, en)
  numpy, pathlib and combined json encoders Being able to serialize arbitrary objects to json can be very handy, for exchanging data, but also for logging or debugging. However, for unknown object types, you’ll get a TypeError: Object of type ... is not JSON serializable when trying to serialize them. This error indicates that the json.JSONEncoder needs some help to figure...
• » Publish your Swift app via homebrew (16. Mai 2021, en)
  This is a recording of my own odyssey developing my first Swift command-line (CLI) app to change the default browser on macOS.
• » An alternative to exceptions in Java: validations, part 2 (17. März 2021, en)
  Validations are a concept to handle errors in a direct way. In essence, this means that errors can be treated as first class citizens and returned by a function. Suddenly they change from being a byproduct to being »into your face«. Is this good? Sometimes yes and sometimes no. It might get mazy.
• » Never again: broken code snippets in your blog posts (4. März 2021, en)
  You probably know the feeling when you discover that your code snippets you embed into your blog are not doing what they should do or, even worse, not even compiling. I mean, you did your best, carefully checking everything in the IDE and copying it over. Perhaps it happens here or in a later stage where you change some little...
• » Make your app fast again: caching function calls in Python (8. September 2020, en)
  It’s the usual pattern: you start with a new webapp project, the app is responsive and fast. But after a while it gets slow and messy. There are many reasons why it can happen. Set aside the obvious reasons such as »technical debt«, messy code and lack of concurrency/parallelism, performance bottlenecks can arise from slow functions that are called frequently...
• » Python logging in Azure: adding application-wide custom dimensions (2. Juni 2020, en)
  Integrating Azure App Insights into Python for logging purposes can be quite tricky, in praticular if you want to add custom properties to the log entries in Azure Log Monitor. This post gives a short outline on this topic.
• » An alternative to exceptions in Java: validations, part 1 (5. Dezember 2019, en)
  Exceptions in Java are inconsistent due to their special handling. They are like a separate flow of information, which not only claims extra resources of a developers mind, but also comes with a lot of boilerplate code and high likelihood of new bugs. Instead of exceptions, errors can be handled with »validations«. The underlying concept of a validation offers an...
• » The complete picture: Lagom and Play in action (Java) (12. Oktober 2019, en)
  Lagom and Play fall into the category of reactive microservices. Lagom serves as basis for microservices and with Play you can build gateways that tie these services together. What puzzles me, though, is that Play and Lagom are handled in isolation when searching for information or going through the tutorials.