LLMs on a Homelab Without a GPU? Here's What I Found

I’ve recently been experimenting more with the power of Large Language Models (LLMs) and how they can supercharge my productivity. This especially has helped with the problem of a blank page wondering where to start writing – I can dump my unsorted thoughts into a chatbot and get a skeleton of a document or implementation in a few minutes. It’s by no means perfect what comes out, but it’s (usually) a very good start. I’ve been playing with both ChatGPT, and a series of local models running on my laptop with Ollama, Fabric, and OpenWebUI.

I wanted to be able to continue to use some of these self-hosted models while I’m out on the go, so I wondered - could I host these on my homelab server? My main homelab server runs an AMD Ryzen 5600G processor which includes an integrated GPU. Can this be used to improve the performance of a local model when I do not have a dedicated GPU available?

That doesn’t seem possible right now though, at least not in an efficient way. This post is a point-in-time snapshot of what I found when looking in to this.

» Continue reading


Installing Python Libraries from Source: pip, Poetry, and Flat Layout Pitfalls

There may be times when you want to install a Python library from source — maybe to test a specific commit, use a pre-release version, or debug a branch. Normally, this is relatively straightforward:

pip install git+https://github.com/aws-ia/taskcat@73de28457c66635a4517393c86484c18f44fef6f

But sometimes, things aren’t that easy… we are getting an error when trying to do this:

  error: Multiple top-level packages discovered in a flat-layout: ['e2e', 'assets', 'taskcat', 'installer', 'taskcat_plugin_testhook'].

  To avoid accidental inclusion of unwanted files or directories,
  setuptools will not proceed with this build.

  If you are trying to create a single distribution with multiple packages
  on purpose, you should not rely on automatic discovery.
  Instead, consider the following options:

  1. set up custom discovery (`find` directive with `include` or `exclude`)
  2. use a `src-layout`
  3. explicitly set `py_modules` or `packages` with a list of names

  To find more information, look for "package discovery" on setuptools docs.

» Continue reading


Modernising the Experiments Project - Containerisation and Code Coverage Improvements

Wow, it’s been over 4 years since the last post in this series. At that point I had hit all the key features that I thought I needed to learn for an upcoming project, and then moved on to other things. Now I’m coming up to some work where I may need this sample application for looking into aspects of OpenTelemetry - so it is time for some updates.

The microprofile-experiments project was really out of date. This PR brings the project up to the latest versions of most dependencies, but also includes some notable improvements.

The PR looks bigger than the changes really are however, mostly due to a combination of the javax to jakarta namespace change and the code formatter setup altering the expected import order.

» Continue reading


Sharing from Mail.app to Obsidian

Over the last few months I’ve been trying to get more organised and take a proper look back at finding a productivity and note taking system that works for me. I’ve been through many different iterations over the years - sometimes it lasts for a while and other times it doesn’t. At the moment I’m trying to centralise on using Obsidian as the tool for all my notes.

There are many emails that I receive that I want to create notes based off of - this might be planning next steps for a project based on information I have been provided, or for more informational messages what my summary/highlights/takeaways are. The point of truth for the message will remain to be Mail, so as well as the contents I want a link back to the original message.

Unfortunately there is no share sheet or shortcuts functionality for Mail on that we can use to extract out mail message contents, so this is a MacOS only solution as it needs to use AppleScript. Even though this can only be created on MacOS, the Mail links which are embedded in the notes will also work on iOS/iPadOS.

Ultimately what I wanted to create is a new note in an “Inbox” directory in my Obsidian Vault with the title being something like <datetime> - subject line and the note containing something like the following:

> [!quote]
> Subject: Mail subject
> Sender: Mail sender
> Date received: Mail date received
> [Open in Mail.app](Mail deeplink)
> 
> ---
> 
> Mail body

» Continue reading


Publishing a Jekyll Site to GitHub Pages using GitHub actions

Ever since GitHub actions was announced I’ve a had a low priority item on my todo list to move my Jekyll blog build process to this over the native GitHub Pages setup. The key benefit to this has been that it allows you to use more modern versions of Jekyll as well as any custom plugins that you choose - instead of being limited to just the plugins and versions that GitHub supports.

Recently this has been becoming higher priority as I was having issues keeping an updated devcontainer that still worked with this old version of Jekyll without needing to use an image that was not optimised for Apple Silicon. This was probably a solvable issue, but changing the build process was too.

This used to be quite a custom process, requiring writing your own workflow using a number of third party actions. I was very pleased to see that it is now an off the shelf solution - so it took me less than 10 minutes to be switched over and on all the latest versions of Jekyll and plugins. I’ve not added anything new yet, but can already see the Paginate V2 plugin supports generating pages for individual tags & categories, something that I was doing manually with a rake script. At this point I am looking to reduce the amount of custom code in my blog and go a lot more stock - I’m even planning on taking another look at minima now that I’m on a recent version of Jekyll.