How slow is “out of the box” when you’re running a 27-billion-parameter model on a brand-new, dual-GPU workstation? For the first several days I spent getting llama.cpp and vLLM running on a pair of Radeon AI PRO R9700s, the answer was a painful 19 tokens per second. This is the story of that journey, all the way up to 185.
Text embeddings are typically aggregated before use in classification. Does the method in which the embeddings are pooled affect the ease of classifying the text?
If you read the description of a World Bank project, would you be able to guess how much of it was spent on climate adaptation? BERT might be able to.
IATI is an incomplete, but up-to-date source of data on international assistance flows. Here’s how it can be used with the OECD DAC CRS to get a better global picture of where money is flowing now.
How can one combine the advantages of modern single-page applications with traditional search-engine optimization (SEO)? Here’s the lessons we learned while implementing SEO on the IATI Datastore Search.
If you’ve ever had to write software that imports or exports IATI data, you may have noticed that the order of the XML elements is considered a critical part of the schema validation. Why was this change made historically, and why does it persist today?
Hello and welcome to the first post on my blog. I’m intending this blog to be a place to host some of my previous and current writings, documenting some interesting technical challenges I’ve faced as well as the solutions I’ve designed to overcome them.