I made a hostel search site: https://hostelpunk.com/.

The short version is that I kept ending up in clean, central, 9.1-star hostels where nobody talked to each other. That is fine if you came to sleep. It is not fine if you came to meet people. A generic 8.9 cannot tell those two places apart, because everything good on a hostel platform compresses into a tight band between roughly 8.5 and 9.6, and the thing I actually want to know about — whether the common room is alive — is buried in subscores and review text.

So the main HostelPunk score is not a rating in the normal sense. It is one specific question:

PerfAtmPct = perfect-atmosphere reviews / atmosphere reviews

If a hostel has PerfAtmPct 93, that means 93% of reviewers gave the atmosphere the top-of-scale rating in the source review data. It is not cleanliness, not luxury, not a weighted soup of staff and breakfast and WiFi. It is a deliberately narrow social-vibe statistic, and it is cruel to places that are pleasant but not socially exceptional. That is the point.

A top-box rate behaves differently from an average. An average is forgiving; everyone giving 8/10 looks healthy. A top-box rate asks “did this place actually work for people, or was it just fine?” For solo-traveler hostels, “fine” is the failure mode I’m trying to detect.

Sample size, or: 100 from 7 reviews is not 100

The obvious bug: a 100% score from 7 reviewers should not casually outrank a 93% from 900. HostelPunk handles this in two places. The displayed score is the raw PerfAtmPct, because that’s the number a human can interpret. A confidence tier sits next to it. Ranking uses a Wilson-style lower-bound adjustment so small-n hostels can appear without pretending to be deep signal.

Current confidence tiers:

• certain: 300+ reviews
• confident: 100+
• mid: 50+
• low: 20+
• blind: below 20

Absence-of-warning claims (e.g. “no bed bug reports”) require at least 100 analyzed reviews. Below that, “we don’t know” is the honest read, not “clean.”

Bed bugs are a different axis

The most useful thing the site does is refuse to compress everything into one score.

A hostel can be social and have recent bed bug reports. Averaging those two facts into a 7.9 is worse than useless — they drive different decisions. Some people will never book a place with any positive bed bug evidence. Others will tolerate one historical report against hundreds of clean recent reviews. The job is to show both, with dates and severity, and let the user pick.

As of today the bed bug detector has positive evidence for 1,075 properties: 4,839 total reports, 658 in the last twelve months, 81 in the last thirty days. The product copy says “reports” and “evidence,” not infestation. Bites can be misidentified, reviews lag conditions, hostels remediate. UI tiers care about recency and repetition — one old report is not the same object as multiple reports this month.

Two concrete Amsterdam examples:

• Onefam Amstel — PerfAtmPct 93/100, 916 reviews, certain confidence, #1 in Amsterdam, #1 in the Netherlands. Dorms from $79, privates from $253. Top languages English 59%, Spanish 8%, Turkish 7%. Clean example, no warnings.
• Princess Hostel Leidse Square — PerfAtmPct 2/100 from 499 reviews, 59 bed bug reports, 18 in the last twelve months, latest 2026-05-02. High tier. The extreme version of the page doing its job.

Suspicious reviews as a distribution problem

The other detector is weirder, and it’s where the project drifts from travel site into applied statistics hobby.

Explicit paid-review complaints are rare. Sometimes a reviewer writes that staff offered a free drink for a positive review; that’s easy to catch but doesn’t move the ranking much. The interesting signal is aggregate: too many short top-rated reviews compared with what peer properties produce. The detector estimates excess short top-atmosphere reviews against a leave-one-out baseline. That estimate is not a per-review verdict. It’s a property-level shape anomaly large enough that I don’t want the raw atmosphere score to be the only score.

When the warning threshold is met, HostelPunk also computes an adjusted atmosphere score with the likely distorted reviews excluded, and shows both. Today, 53 current properties cross that line with an adjusted score below the raw score. Average drop: 3.9 points. Largest drop: 16.4 points.

Examples:

• Mama Hostel & Rooftop Pool, Hanoi — raw 70.4, adjusted 54.0, estimated 90 incentivized.
• ClinkNOORD, Amsterdam — 3,015 reviews, PerfAtmPct 53/100, also 7 bed bug reports and a suspicious-5-star warning with about 104 possibly bought. The mixed-signal case: well-known place, large sample, weak vibe, two separate warning kinds — the page can show all of that at once instead of melting it into a single number that picks a side.

The wording in this section of the UI is intentionally less fun than the rest of the brand. An aggregate anomaly is not a claim that review #12345 is fake, and the copy has to keep that clear.

The unglamorous half: source data is weird

The funniest bugs are not in the ranking code. They’re in source data.

Two real Hostelworld quirks the scraper has to handle:

• Review notes from the legacy review endpoint sometimes come back exactly 512 characters long, cut mid-word. The public Hostelworld site shows the same cutoff, so the truncation is at the source, not in our parser.
• The city availability endpoint can return room prices like 999999.99. If you request a different currency, the sentinel is numerically converted and comes back as something like 1170756.99 USD — still bogus, now wearing a currency label.

So a lot of the engineering is not “fetch JSON, insert rows.” It’s deciding which fields are trustworthy enough to display, what counts as sentinel garbage, and which narrower endpoint should override a broader one. None of that ends up in product copy, but it’s why prices on the page don’t lie.

Stack

The boring-in-a-good-way part:

• Go for the user-facing app (hostelpunk)
• PostgreSQL + PostGIS
• Server-rendered HTML, a little HTMX
• No npm
• Python scrapers with uv
• Review analysis runs as a separate worker pool against models served locally and via OpenRouter

HostelPunk is the public surface of a bigger pile called SocialSpots, which has Hostelworld, Booking.com, Google Maps, iOverlander, OSM, and Workaway data behind it. The order-of-magnitude shape: roughly a million-ish Hostelworld reviews, tens of millions of Booking reviews, and tens of millions of review-analysis rows feeding the detectors. A traveler does not need to see any of that. They need to see that a hostel’s 93 came from 916 reviews, that there are no current warnings, and that the language mix probably fits them.

Still rough

The UI isn’t done. Some detectors are conservative, some are still mining signals. Rare-class precision is hard. Source platforms change APIs. A property can improve or decay faster than the crawler notices. The right posture is not “the model knows the truth” — it’s “this is a much better prior than reading 300 random reviews at 1am the night before a flight.”

It already works well enough for me. If I’m choosing between two hostels in Amsterdam or Hanoi or Hoi An, I’d rather start from atmosphere hit-rate, confidence, warning evidence, suspicious-review adjustment, language mix, and source links than from a generic 8.9.

https://hostelpunk.com/

В инструменте Curves горизонтальная ось — это яркость: чёрный слева, белый справа. Вертикальная ось показывает, сколько пикселей приходится на каждый уровень яркости. Но форма графика сама по себе мало что даёт — нужно понимать, какие области кадра её формируют и стоит ли в этих областях что-то сохранять.

В этом конкретном примере фотография явно тёмная, гистограмма сильно сдвинута влево. Справа тоже есть какие-то данные, но само по себе наличие ярких пикселей ничего не говорит о том, принадлежат ли они объекту съёмки или чему-то совершенно постороннему — отражению, сценическому прожектору, бьющему в объектив, гирлянде светодиодов на заднем плане.

Первая гистограмма

На первом скриншоте большая часть изображения лежит в тёмной части тонального диапазона, и хотя справа на гистограмме видны несколько ярких пикселей, к модели они не имеют отношения.

Этот кластер данных в светах — сценическое освещение, попавшее в объектив. Да, на гистограмме справа что-то есть, но к модели это не относится. Сама модель сидит слишком далеко слева — изображение недоэкспонировано ещё до какой-либо коррекции.

Обрезаем вводящие в заблуждение света

На следующем скриншоте яркие источники света обрезаны из кадра.

Без источников света гистограмму читать проще: ярких пикселей стало ещё меньше, и сразу видно, что почти все тона лежат в тёмной и средней частях диапазона. Модель просто не дотягивает до достаточно ярких значений, чтобы нормально отображаться на экране.

Нескольких сценических светодиодов хватает, чтобы на гистограмме появились света, а объект при этом остаётся тёмным.

Инструмент

На скриншотах в этой статье используется Snapseed — бесплатное приложение для редактирования фотографий, доступное на Android и iOS. Однако та же логика работает в большинстве фоторедакторов — почти все включают либо инструмент Curves, либо более простую Levels-коррекцию, которая делает то же самое. Интерфейсы и названия отличаются, но принцип читать гистограмму в контексте реального изображения остаётся одинаковым независимо от того, какой инструмент вы используете.

Сдвиг белой точки

Сама коррекция простая: белая точка в Curves сдвигается влево, примерно туда, где гистограмма начинает резко подниматься.

Светлые тона объекта перераспределяются по более широкой части доступного диапазона, и изображение раскрывается очень быстро — новых деталей не появляется, просто те тона, что уже были, теперь эффективнее используют выходной диапазон.

После применения коррекции и повторного открытия Curves гистограмма выглядит заметно иначе.

Тона теперь занимают гораздо большую часть гистограммы. До правого края она не доходит — и не нужно: запас по светам это нормально, загонять их в клиппинг ради красивого графика смысла нет.

Изоляция самых ярких пикселей

Один практический способ проверить, что на самом деле содержат самые яркие части изображения — это сдвинуть чёрную точку в Curves до упора вправо. Это давит почти всё в чёрный и оставляет видимыми только самые яркие пиксели.

Ожидаемо: осталось сценическое освещение, светодиоды и мелкие яркие элементы, к модели отношения не имеющие. На самой модели в этом диапазоне яркости пикселей почти нет — пара изолированных точек в синем канале, не больше.

Самые яркие значения в исходном кадре — это осветительная установка и артефакты сцены, а не кожа или лицо.

Почему это важно на экране телефона

На телефоне фотография существует внутри интерфейса — белый текст, иконки, кнопки, другие фотографии. Все эти элементы используют тот же экран и тот же диапазон яркости.

Если объект сжат в тёмную половину диапазона без причины, фотография будет выглядеть заметно темнее интерфейса и соседних фотографий в ленте. Можно получить изображение, в котором технически ничего не обрезано, но которое выглядит неоправданно тусклым там, где его реально будут смотреть.

Каждое изображение до чисто белых светов доводить не нужно. Но тона объекта съёмки не должны сидеть внизу шкалы только потому, что пара ярких пятен занимает правый край гистограммы.

Та же логика внутри Instagram

Для Instagram стоит смотреть на изображение в том окружении, где его увидят.

Первый скриншот — экран Instagram как есть: фотография, UI, текст, иконки, лента, плюс наложенная гистограмма Curves.

Следующий скриншот изолирует только яркие пиксели, сдвигая чёрную точку вправо до тех пор, пока всё ниже выбранного порога не станет чёрным.

Обратите внимание, что осталось видимым. На этом пороге большая часть интерфейса Instagram по-прежнему на месте — текст, иконки, фотографии других пользователей. Элементы платформы занимают те значения яркости, до которых наша фотография едва дотягивается.

На следующем скриншоте белая точка сдвинута на ту же позицию.

Модель теперь экспонирована так, что корректно читается внутри интерфейса Instagram. Элементы UI слегка переэкспонированы, но они и не часть нашего изображения — внутри самой фотографии ничего не потеряно.

Слайдер пришлось сдвинуть почти до середины экрана. Это большая коррекция, и она показывает, сколько динамического диапазона в исходной обработке оставалось пустым — объект был заметно темнее, чем нужно, и сам по себе, и относительно окружения в ленте.

Важно, расположены ли тона объекта съёмки достаточно высоко в диапазоне, чтобы нормально читаться там, где фотографию реально будут смотреть. Если нет — белую точку обычно можно сдвинуть гораздо дальше, чем кажется, пока клиппинг не затрагивает то, что в кадре важно.

In the Curves tool the horizontal axis represents brightness — black on the left, white on the right — and the vertical axis shows how many pixels fall at each brightness level. That much is straightforward, but the part that actually matters is understanding which regions of the frame are responsible for the shape you see in front of you, and whether those regions contain anything worth preserving.

In this particular example the photo is clearly dark and the histogram is stacked heavily toward the left side. There is some data on the right as well, but the presence of bright pixels alone does not tell you whether those pixels belong to the subject or to something else entirely — a reflection, a stage light aimed into the lens, a string of LEDs in the background.

The first histogram

In the first screenshot most of the image lives in the darker part of the tonal range, and while there are a few bright pixels visible on the right side of the histogram, they do not belong to the model.

That small cluster of highlight data comes primarily from the stage lighting shining into the lens. The histogram does contain information on the right side, but it is not information that carries any meaningful detail about the subject — the model herself is sitting too far to the left, which is why the image reads as underexposed before any correction has been applied.

Cropping out the misleading highlights

In the next screenshot the bright lights have been cropped out of the frame.

Once the lights are removed from the visible area, the histogram becomes considerably easier to interpret: there are even fewer bright pixels now, and it shows much more clearly where the meaningful tonal information of the image actually sits. Nearly all of it belongs to the darker and middle parts of the range, and the model is simply not reaching far enough into the brighter values to render well on screen.

This is precisely why reading the full histogram without paying attention to what produces its shape can be misleading — a handful of stage LEDs are enough to suggest that the image already contains strong highlights, while the actual subject remains too dark.

About the tool

The screenshots in this article show Snapseed, a free photo editing app available on Android and iOS. However, the same logic applies to nearly all photo editing apps — most of them include either a Curves tool or a simpler Levels adjustment that accomplishes the same thing. The specific interface and names may vary, but the fundamental principle of reading the histogram in context with the actual image remains the same regardless of which tool you use.

Moving the white point

The correction itself is straightforward: the white point in Curves is moved to the left, roughly to the position where the histogram begins rising sharply.

This remaps the brighter tones of the subject across a wider portion of the available range, and the image opens up very quickly — not because new detail has appeared, but because the tones that were already present are now distributed more effectively across the output values.

After applying the adjustment and reopening Curves, the histogram looks noticeably different.

The tonal information now covers much more of the histogram. It still does not extend all the way to the extreme right, and there is no particular reason it should — some headroom is normal, and driving important highlights into clipping just to fill the graph to the edge accomplishes nothing useful.

Isolating the brightest pixels

One practical way to verify what the brightest parts of the image actually contain is to move the black point all the way to the right in Curves. This crushes nearly everything to black and leaves only the very brightest pixels visible.

The result is not surprising: what remains visible is stage lighting, LEDs, and other small bright elements that have very little to do with the model. On the subject herself there are hardly any pixels in that extreme brightness range, aside from a few isolated blue-channel specks scattered here and there.

This confirms the earlier reading — the brightest values in the original frame were not carrying important skin or facial detail, they belonged almost entirely to the lighting setup and the surrounding artifacts in the scene.

Why this matters on a phone screen

When a photo is displayed on a phone it does not appear in isolation — it sits inside an interface full of white text, icons, buttons, and other photos, all of which use the same screen and the same available brightness range.

If the subject is compressed into the darker half of the tonal range without good reason, the photo will appear noticeably darker than the surrounding interface elements and often darker than other photos nearby in the feed. This is not a hypothetical concern: it is entirely possible to produce an image that is technically not clipped anywhere and still looks unnecessarily dim in the context where it will actually be viewed.

This does not mean every image needs to be pushed to pure white highlights. It means that the useful tones of the subject should not be left sitting too far down the scale simply because a few irrelevant bright spots happen to occupy the far-right edge of the histogram.

The same logic inside Instagram

For photos intended for Instagram it is worth looking at the image not as an isolated file but in the environment where it will actually be displayed.

The first screenshot in this sequence shows the Instagram screen as it normally appears — the photo, the UI, text, icons, and the surrounding feed content all visible together, with a Curves histogram overlay.

The next screenshot isolates only the brighter pixels by moving the black point to the right until everything below a chosen threshold turns black.

What matters here is not just which parts of the screen disappear, but which parts remain. At that threshold most of the Instagram UI is still visible — the text, the icons, other users’ photos. In other words, the platform elements and the surrounding content are occupying brightness values that the photo in question is barely reaching.

In the next screenshot, instead of using the black point to inspect the threshold, the white point is moved to that same position.

The model is now exposed at a level that reads correctly inside the Instagram interface. The UI elements and some of the surrounding content appear slightly overexposed, but that is expected because they are not part of our image — within the photo itself no important information is lost.

This also serves as a practical measure of how much of the available dynamic range had been left unused. The slider had to be moved almost to the middle of the screen, which is a very large correction and which shows that the original edit was leaving the subject considerably darker than necessary, both in absolute terms and relative to the environment where the photo would be viewed.

The practical question is therefore not whether the file technically contains highlight data somewhere in the frame. The question is whether the tones that fall on the subject are placed high enough in the range to display clearly in the context where the image will actually be seen. If they are not, then the white point can usually be moved much further than one might expect, as long as clipping is avoided in the parts of the image that carry meaningful detail.

You can open a user page like listenomics.unmb.pw/user/unmanbearpig/charts and get charts for listens over time, artist listens, track listens, unique tracks, new artists, and a few ratio/age metrics.

Behind the scenes it stores scrobbles in SQLite and fetches updates in background workers, with incremental fetches instead of re-downloading everything.

UI is shit now, will improve

Things that lastfm doesn’t have

• How many unique artists / tracks you listened within a month / week It shows if you’re listening to the same song over and over, or a lot of songs each one time, or somewhere in between.
• How many new artists / tracks you find in each month / week
• Track age: Do you mainly listen to things you found years ago or stuff you just found today?

Here is the line that fixed it:

context.map.touchZoomRotate['_tapDragZoom']['_enabled'] = false;

That’s it, folks!

The solution? Let’s ask ChatGPT!

can you write a minimalist firefox extension that disables hotkeys that some websites override?

Which got me some manifest.json and content.js. Having never written a browser extension that helped a lot, now I have something to start with. I needed to ask things like how to test that extension, etc. but that was easy enough.

It didn’t work though. The code I got from the LLM was:

// content.js
document.addEventListener("keydown", function(event) {
  // List of hotkeys you want to disable
  var blockedKeys = ["F5", "Ctrl+R", "Ctrl+U", "Ctrl+Shift+I"];
  
  // Check if the pressed key combination is in the blockedKeys list
  if (blockedKeys.includes(event.key) || blockedKeys.includes(getKeyCombination(event))) {
    event.preventDefault();
    event.stopPropagation();
  }
});

// Function to get the key combination
function getKeyCombination(event) {
  var keyCombination = "";
  if (event.ctrlKey) keyCombination += "Ctrl+";
  if (event.altKey) keyCombination += "Alt+";
  if (event.shiftKey) keyCombination += "Shift+";
  keyCombination += event.key;
  return keyCombination;
}

It seems like preventDefault and stopPropagation don’t achieve what I intended. Now I just have to figure out what to replace it with.

Shouldn’t be hard, let’s find some code that does it. A quick search on addons.mozilla.org provided me with a bunch of extensions. Upon expecting the source code of the first one, I found:

    event.stopImmediatePropagation();

And it worked! It was easier than I expected.

Next, I needed to add some configuration options so that I wouldn’t have to rebuild the extension every time I wanted to add more websites and shortcuts. I didn’t want to disable all overrides on all websites, as many of them are actually useful. Creating a proper UI seemed like a lot of work, so I opted for the simplest solution: a JSON config!

Chatting with ChatGPT provided me with prototypes for options.html and options.js. However, they didn’t work. After some debugging, I discovered that you can’t have callbacks when blocking event propagation, which actually made the extension better, in my opinion. Instead of checking the configuration on every key event, we can load it on page load and save it to a window variable. Then, we can simply check the window variable. This method is likely faster, as we don’t need to use the browser.sync API very often.

That might look terrible, but it works! And I don’t need anything else, do you? Having the config just as a text file, it’s easy to copy and paste all of it, or parts of it, it’s easy to share it with other people or copy to your other browser, easy to back it up, etc. I actually love that solution! Why reinvent the wheel?

The rest of the process involved cleaning up some debug statements, drawing an icon in 5 minutes, asking ChatGPT to come up with a name, and submitting it to addons.mozilla.org.

We’ll see if Mozilla accepts it. I’ll update this blog post when I hear back from them.

The entire process took about 3 hours, including submitting to addons.mozilla.org, drawing the icon, debugging, etc. Actually, most of it was the boring stuff, like figuring out how to make an icon and what else I needed to submit.

Surprisingly, it was easy overall. I didn’t expect it to take only 3 hours, considering I had almost zero knowledge of how browser extensions are made!

I’m not sure if anyone’s going to use it, but hey, why not spend an extra hour and publish it?

Link to repository: github.com/unmanbearpig/unmenu

Download here

A long time ago, I was using Alfred for launching apps on my Mac. Then Apple introduced Spotlight, and I switched to that because it’s a one less dependency. And then I switched to Gentoo and used dmenu. Now, when I’m using both Gentoo and mac, I’ve noticed how bad Spotlight is compared to dmenu - a dead simple fast app that does the job well enough.

dmenu is one of the suckless.org apps. Their philosophy is minimalism to the point of having configs in “.h” files, so that you need to recompile it every time you change the config. I don’t use a lot of their apps but have respect for their dedication to simplicity. unmenu is not as simple, for the most part because it’s a pain to compile and install compared to something like dmenu in linux.

So, what I found wrong with Spotlight:

• Imperfect fuzzy matching, like “ws” not matching anything even when “Wireshark” and “WhatsApp” are available.
• It needs to index stuff.
• Has web search and too many other features that are hard to disable. At least I couldn’t.
• It searches who knows what, often finding random files by even when I disabled searching everywhere but the “/Applications” directory.
• Not scriptable, can’t launch anything except .app bundles.
• Not consistent. The same string might be completed to one app today and to another app tomorrow.
• Not debuggable. There is no Terminal.app in my Spotlight for some reason. How do I debug this? I don’t know.

I could probably fix some of those issues, but overall, it seems like a wrong tool for the job. I don’t want most of the features it offers, and the ones that I do want don’t work the way I want them to. Luckily, José Luis Pereira wrote dmenu-mac, which is almost exactly what I wanted. But with a few issues, like this bug that switches to another desktop randomly when hitting the hotkey, or some issues with completion. So, I forked it to fix everything that bothers me and add some features.

Changes from dmenu-mac

• Fixed a longstanding issue.
• Switched to Accessibility API for handling hotkeys because it seems to work well.
• Implemented what I believe to be a superior fuzzy matching algorithm, leveraging lotabout/fuzzy-matcher.
• Introduced a configuration file located at ~/.config/unmenu/config.toml, enabling users to customize search directories, filter out applications, and integrate scripts and aliases.

Examples of scripts and aliases

Generate a QR code

Create a file in one of the directories listed in unmenu’s config with the following content. It’ll generate and show a QR code from the copied text. Don’t forget to chmod +x this file

#!/bin/sh

set -e

FILENAME="/tmp/$(date).png"

pbpaste | qrencode -o "$FILENAME"
open "$FILENAME"

Alias / run CLI command from unmenu

To allow to run scrcpy from unmenu create a file scrcpy with the following content:

#!/bin/sh

scrcpy

Open video in mpv video player

#!/bin/sh

mpv `pbpaste`

Implementation

Style

Intentionally quick and dirty. I never finish my projects because of my desire to improve one thing or another before writing a blog post or even README and publishing it somewhere. So this time I focused on making it work the way I want and not worrying about how exactly I’ve done it and if the code looks good. The code is terrible, there are still some debug statements, commented out code, most git commit messages are 1 line if not 1 word or 1 character. But who cares! I’ve been using it for half a year without an issue. There is not much code so I’ll figure it out next time I need to change something, which would probably happen not sooner than in a year. Works well enough for me already, so there is nothing to change.

Hotkey handling

I suspected that deprecated APIs like getEventMonitorTarget that were used originally were the cause of the bugs, so I switched to the API that requires additional permissions but seems to work without an issue.

Fuzzy matching

I’ve chosen to use a Rust library (https://github.com/lotabout/fuzzy-matcher) for fuzzy matching, mostly because Rust is more familiar, and I don’t need to use Xcode for that part of the project. Works well enough for now. I needed to make an interop via C API, and had to debug a few issues with my memory management, but that’s okay. I probably learned something along the way, but I did that part half a year ago so don’t really remember. I should really write the blog post just after I implement something interesting instead of waiting for half a year.

LLM Conclusion

Unmenu, my fork of dmenu-mac, aims to provide a more efficient and reliable application launcher for macOS users who prefer a keyboard-centric workflow. By addressing the limitations and bugs found in dmenu-mac and Spotlight, Unmenu offers a superior fuzzy matching algorithm, hotkey handling, and a customizable configuration file. This project not only improves upon the original dmenu-mac but also enhances the overall user experience by making app launching faster and more intuitive.

I remember that years and years ago I was able to increase the poll rate of this exact mouse to 500 hz (default rate is 125 hz) in Windows. Theoretically it should reduce the mouse latency and improve its smoothness and accuracy when moving very fast.

I’ve tried looking for software for linux that would do that, but couldn’t find anything that works. So I’ve decided to dig into it a bit more.

Measure

We can’t optimize anything unless we measure it.

In Linux each input device is a file in /dev/input/, for example my mouse is at /dev/input/by-id/usb-Logitech_USB-PS_2_Optical_Mouse-event-mouse.

We can just read the data from it like so:

> sudo cat /dev/input/by-id/usb-Logitech_USB-PS_2_Optical_Mouse-event-mouse | xxd

00000000: 4e1a a362 0000 0000 5ea5 0400 0000 0000  N..b....^.......
00000010: 0200 0100 ffff ffff 4e1a a362 0000 0000  ........N..b....
00000020: 5ea5 0400 0000 0000 0000 0000 0000 0000  ^...............
00000030: 4e1a a362 0000 0000 d8bc 0400 0000 0000  N..b............
00000040: 0200 0100 ffff ffff 4e1a a362 0000 0000  ........N..b....
00000050: d8bc 0400 0000 0000 0000 0000 0000 0000  ................
00000060: 4e1a a362 0000 0000 33cc 0400 0000 0000  N..b....3.......
00000070: 0200 0000 0100 0000 4e1a a362 0000 0000  ........N..b....
00000080: 33cc 0400 0000 0000 0200 0100 ffff ffff  3...............
00000090: 4e1a a362 0000 0000 33cc 0400 0000 0000  N..b....3.......
000000a0: 0000 0000 0000 0000 4e1a a362 0000 0000  ........N..b....
000000b0: d6db 0400 0000 0000 0200 0100 ffff ffff  ................

As we move the mouse we get a bunch of data on the screen.

So we can measure the delays between the data batches and hopefully get the actual mouse rate.

I’ve written a simple Rust program that performs a crude measurement here (unmanbearpig/hid_rate)

here is a snippet from it:

loop {
    last_time = time::Instant::now();
    let bytes_read = file.read(&mut buf)?;
    let elapsed = last_time.elapsed();
    let secs = elapsed.as_secs_f64();
    let hz = 1.0 / secs;

    println!("{:14} ns, {:>14.3} us, {:10.5} hz, {:4} bytes read",
             elapsed.as_nanos(), secs * 1000_000.0, hz, bytes_read);
}

Here we just read some data in a loop and measure how long each iteration took. From the duration it’s trivial to calculate the rate, so it’s easier to read.

Let’s run it on the device file of our mouse and move the mouse a bit.

> hid_rate /dev/input/by-id/usb-Logitech_USB-PS_2_Optical_Mouse-event-mouse

       7982385 ns,       7982.385 us,  125.27584 hz,   72 bytes read
       8002035 ns,       8002.035 us,  124.96821 hz,   72 bytes read
       7962256 ns,       7962.256 us,  125.59255 hz,   72 bytes read
       7988828 ns,       7988.828 us,  125.17481 hz,   72 bytes read
       8011032 ns,       8011.032 us,  124.82786 hz,   72 bytes read
       7966524 ns,       7966.524 us,  125.52526 hz,   72 bytes read
       7993312 ns,       7993.312 us,  125.10459 hz,   72 bytes read
       7986931 ns,       7986.931 us,  125.20454 hz,   48 bytes read
       7981237 ns,       7981.237 us,  125.29386 hz,   48 bytes read

Nice! we get 125 hz, which is the default poll rate and it’s what we expect.

USB configuration

I’m not an expert in USB, but as far as I know USB devices never send data to the host unannounced. They only respond to host’s queries. The host polls the device for new data at a certain rate to check if device has any new data to send.

Different devices have different rates they support, that supported rate is sent by the device at configuration time in the bInterval field of the descriptor. More specifically bInterval is the delay in milliseconds between requests. It means slightly different things for different devices and speeds.

Universal Serial Bus Specification:

Interval for polling endpoint for data transfers.

Expressed in frames or microframes depending on the device operating speed (i.e., either 1 millisecond or 125 μs units).

For full-/high-speed isochronous endpoints, this value must be in the range from 1 to 16. The bInterval value is used as the exponent for a 2^bInterval-1 value; e.g., a bInterval of 4 means a period of 8 (2^4-1).

For full-/low-speed interrupt endpoints, the value of this field may be from 1 to 255.

For high-speed interrupt endpoints, the bInterval value is used as the exponent for a 2^4-1 This value must be from 1 to 16.

We don’t care about High-Speed, as mice are never High Speed. So for our device a frame is just 1ms and bInterval is the number of frame and also milliseconds.

We can find the device’s configuration by lsusb -v

> lsusb -v

Bus 001 Device 007: ID 046d:c051 Logitech, Inc. G3 (MX518) Optical Mouse
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x046d Logitech, Inc.
  idProduct          0xc051 G3 (MX518) Optical Mouse
  bcdDevice           30.00
  iManufacturer           1 Logitech
  iProduct                2 USB-PS/2 Optical Mouse
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0022
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower               98mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      1 Boot Interface Subclass
      bInterfaceProtocol      2 Mouse
      iInterface              0 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      77
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              10

So our mice’s bInterval is 10 ms. Aparrently it’s rounded to the nearest power of 2 to 8 ms. So to get the polling rate we divide 1000ms (in a second) by our bInterval which is rounded to 8 and get 125 hz.

To “overclock” our mouse we need to somehow make Linux think that the device’s supported rate is 500hz, which means we need to set bInterval to 2 (ms).

Patching the kernel

First, let’s get the kernel source code, extract the archive and cd into it. This step might be different for different distributions. We need to be able to build it and run it, which I can do.

I’m using the latest stable kernel version which right now is 5.18.3.

Now let’s make sure that there isn’t a driver or a special case for this specific mouse somewhere in the kernel. To do that I’ll just grep for something that might be a name or id if this mouse.

I use ripgrep instead of grep, but it’s not important.

> cd drivers
> rg -i 'mx518'

Found nothing. How about MX followed by 3 numbers?

> rg -i 'mx\d\d\d\b'

...
input/mouse/logips2pp.c: { 61,	PS2PP_KIND_MX,					/* MX700 */
input/mouse/logips2pp.c: { 100,	PS2PP_KIND_MX,					/* MX510 */
input/mouse/logips2pp.c: { 111,  PS2PP_KIND_MX,	PS2PP_WHEEL | PS2PP_SIDE_BTN },	/* MX300 reports task button as side */
input/mouse/logips2pp.c: { 112,	PS2PP_KIND_MX,					/* MX500 */
input/mouse/logips2pp.c: { 114,	PS2PP_KIND_MX,					/* MX310 */
...

Looks interesting…

// SPDX-License-Identifier: GPL-2.0-only
/*
 * Logitech PS/2++ mouse driver
 *
 * Copyright (c) 1999-2003 Vojtech Pavlik <vojtech@suse.cz>
 * Copyright (c) 2003 Eric Wong <eric@yhbt.net>
 */

It’s a PS/2 not USB driver, so let’s ignore that.

Now let’s look for the idProduct of our mouse, which is 0xc051:

> rg 'c051'

sound/pci/hda/patch_realtek.c
11238:	SND_PCI_QUIRK(0x144d, 0xc051, "Samsung R720", ALC662_FIXUP_IDEAPAD),

lib/crypto/chacha20poly1305-selftest.c
2980:static const u8 enc_assoc051[] __initconst = {
6033:	{ enc_input051, enc_output051, enc_assoc051, enc_nonce051, enc_key051,
6034:	  sizeof(enc_input051), sizeof(enc_assoc051), sizeof(enc_nonce051) },

drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_1_d.h
463:#define mmSCRATCH_ADDR                                                          0xc051

drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_0_d.h
463:#define mmSCRATCH_ADDR                                                          0xc051

drivers/gpu/drm/amd/include/asic_reg/gca/gfx_7_0_d.h
413:#define mmSCRATCH_ADDR                                                          0xc051

drivers/gpu/drm/amd/include/asic_reg/gca/gfx_7_2_d.h
425:#define mmSCRATCH_ADDR                                                          0xc051

drivers/gpu/drm/amd/pm/powerplay/inc/polaris10_pwrvirus.h
1117:	0x04200001, 0x7e2a0004, 0xce013084, 0x90000000, 0x28340001, 0x313c0bcc, 0x9bc00010, 0x393c051f,

drivers/gpu/drm/sun4i/sun8i_vi_scaler.c
210:	0x00fc051f, 0x00fc0521, 0x00fc0621, 0x00fc0721,

tools/testing/ktest/sample.conf
1031:#   IGNORE_WARNINGS = 42f9c6b69b54946ffc0515f57d01dc7f5c0e4712 0c17ca2c7187f431d8ffc79e81addc730f33d128

Nothing related to mice or Logitech.

Now that we are somewhat sure that there is no specific driver for our mouse let’s try to find where bInterval is being set for all devices.

Searching for bInterval resulted in the following interesting code in

drivers/usb/core/config.c:

static int usb_parse_endpoint(struct device *ddev, int cfgno,
	struct usb_host_config *config, int inum, int asnum,
	struct usb_host_interface *ifp, int num_ep,
	unsigned char *buffer, int size)

...

/*
 * Fix up bInterval values outside the legal range.
 * Use 10 or 8 ms if no proper value can be guessed.
 */
i = 0;		/* i = min, j = max, n = default */
j = 255;
if (usb_endpoint_xfer_int(d)) {
	i = 1;
	switch (udev->speed) {
	case USB_SPEED_SUPER_PLUS:
	case USB_SPEED_SUPER:
	case USB_SPEED_HIGH:
		/*
		 * Many device manufacturers are using full-speed
		 * bInterval values in high-speed interrupt endpoint
		 * descriptors. Try to fix those and fall back to an
		 * 8-ms default value otherwise.
		 */
		n = fls(d->bInterval*8);
		if (n == 0)
			n = 7;	/* 8 ms = 2^(7-1) uframes */
		j = 16;

		/*
		 * Adjust bInterval for quirked devices.
		 */
		/*
		 * This quirk fixes bIntervals reported in ms.
		 */
		if (udev->quirks & USB_QUIRK_LINEAR_FRAME_INTR_BINTERVAL) {
			n = clamp(fls(d->bInterval) + 3, i, j);
			i = j = n;
		}
		/*
		 * This quirk fixes bIntervals reported in
		 * linear microframes.
		 */
		if (udev->quirks & USB_QUIRK_LINEAR_UFRAME_INTR_BINTERVAL) {
			n = clamp(fls(d->bInterval), i, j);
			i = j = n;
		}
		break;
	default:		/* USB_SPEED_FULL or _LOW */
		/*
		 * For low-speed, 10 ms is the official minimum.
		 * But some "overclocked" devices might want faster
		 * polling so we'll allow it.
		 */
		n = 10;
		break;
	}
} else if (usb_endpoint_xfer_isoc(d)) {
	i = 1;
	j = 16;
	switch (udev->speed) {
	case USB_SPEED_HIGH:
		n = 7;		/* 8 ms = 2^(7-1) uframes */
		break;
	default:		/* USB_SPEED_FULL */
		n = 4;		/* 8 ms = 2^(4-1) frames */
		break;
	}
}
if (d->bInterval < i || d->bInterval > j) {
	dev_warn(ddev, "config %d interface %d altsetting %d "
	    "endpoint 0x%X has an invalid bInterval %d, "
	    "changing to %d\n",
	    cfgno, inum, asnum,
	    d->bEndpointAddress, d->bInterval, n);
	endpoint->desc.bInterval = n;
}

/* Some buggy low-speed devices have Bulk endpoints, which is
 * explicitly forbidden by the USB spec.  In an attempt to make
 * them usable, we will try treating them as Interrupt endpoints.
 */
if (udev->speed == USB_SPEED_LOW && usb_endpoint_xfer_bulk(d)) {
	dev_warn(ddev, "config %d interface %d altsetting %d "
	    "endpoint 0x%X is Bulk; changing to Interrupt\n",
	    cfgno, inum, asnum, d->bEndpointAddress);
	endpoint->desc.bmAttributes = USB_ENDPOINT_XFER_INT;
	endpoint->desc.bInterval = 1;
	if (usb_endpoint_maxp(&endpoint->desc) > 8)
		endpoint->desc.wMaxPacketSize = cpu_to_le16(8);
}

/*
 * Validate the wMaxPacketSize field.
 * Some devices have isochronous endpoints in altsetting 0;
 * the USB-2 spec requires such endpoints to have wMaxPacketSize = 0
 * (see the end of section 5.6.3), so don't warn about them.
 */
maxp = le16_to_cpu(endpoint->desc.wMaxPacketSize);
if (maxp == 0 && !(usb_endpoint_xfer_isoc(d) && asnum == 0)) {

...

It looks like this code is setting bInterval based on device speed and other things. Probably a decent place we can insert our code into.

Judging by the comments the code here sets endpoint->desc.bInterval to override it, so let’s try doing the same.

...

} else if (usb_endpoint_xfer_isoc(d)) {
	i = 1;
	j = 16;
	switch (udev->speed) {
	case USB_SPEED_HIGH:
		n = 7;		/* 8 ms = 2^(7-1) uframes */
		break;
	default:		/* USB_SPEED_FULL */
		n = 4;		/* 8 ms = 2^(4-1) frames */
		break;
	}
}

/* unmanbearpig: MX518 mouse hack */
/* The MX518 mouse supports 500 hz poll rate, but reports only 125 hz,
 * we can override it to get the faster rate */
if (udev->descriptor.idVendor == 0x046d && udev->descriptor.idProduct == 0xc051) {
	dev_warn(ddev, "overriding MX518 bInterval to 2 (500hz); config %d interface %d\n",
	         cfgno, inum);
	n = 2;
	endpoint->desc.bInterval = n;
}
/* unmanbearpig end hack */

if (d->bInterval < i || d->bInterval > j) {
	dev_warn(ddev, "config %d interface %d altsetting %d "
	    "endpoint 0x%X has an invalid bInterval %d, "

...

We first match the idVendor and idProduct so we only affect our mouse. Then we use dev_warn to print something into dmesg so we can check in the logs that the code has actually ran, in case something goes wrong. And finally we set the n value which is used in this function and the bInterval to 2, which corresponds to 500 hz poll rate.

Now compile and install the kernel, then reboot and pray.

My kernel surprisingly worked the first time. Let’s check if the patch worked:

> dmesg | rg MX518

[    2.940385] usb 1-1.4: overriding MX518 bInterval to 2 (500hz); config 1 interface 0

Looks good! There is our log message. Let’s check if reported bInterval is changed to 2:

> lsusb -v

        bInterval              10

Huh, it’s not changed and still is 10. Let’s try measuring it anyway.

> sudo hid_rate /dev/input/by-id/usb-Logitech_USB-PS_2_Optical_Mouse-event-mouse

       1980669 ns,       1980.669 us,  504.87992 hz,   48 bytes read
       1993399 ns,       1993.399 us,  501.65571 hz,   48 bytes read
       1986994 ns,       1986.994 us,  503.27278 hz,   48 bytes read
       1975510 ns,       1975.510 us,  506.19840 hz,   48 bytes read
       1985570 ns,       1985.570 us,  503.63372 hz,   48 bytes read
       1980698 ns,       1980.698 us,  504.87252 hz,   48 bytes read
       1989573 ns,       1989.573 us,  502.62041 hz,   72 bytes read
       1972313 ns,       1972.313 us,  507.01892 hz,   48 bytes read
       1984499 ns,       1984.499 us,  503.90552 hz,   48 bytes read
       1983499 ns,       1983.499 us,  504.15957 hz,   72 bytes read
       1978091 ns,       1978.091 us,  505.53792 hz,   48 bytes read
       1983221 ns,       1983.221 us,  504.23024 hz,   48 bytes read
       1995423 ns,       1995.423 us,  501.14687 hz,   72 bytes read
       1978025 ns,       1978.025 us,  505.55478 hz,   48 bytes read
       1995992 ns,       1995.992 us,  501.00401 hz,   72 bytes read
       3980757 ns,       3980.757 us,  251.20850 hz,   72 bytes read
       3988748 ns,       3988.748 us,  250.70523 hz,   72 bytes read
       3977186 ns,       3977.186 us,  251.43405 hz,   72 bytes read
       3991401 ns,       3991.401 us,  250.53860 hz,   72 bytes read
       3987779 ns,       3987.779 us,  250.76615 hz,   72 bytes read
       3982110 ns,       3982.110 us,  251.12315 hz,   48 bytes read
       3979834 ns,       3979.834 us,  251.26676 hz,   48 bytes read
       1988891 ns,       1988.891 us,  502.79276 hz,   48 bytes read
       1984657 ns,       1984.657 us,  503.86540 hz,   48 bytes read
       1985271 ns,       1985.271 us,  503.70957 hz,   48 bytes read
       7988877 ns,       7988.877 us,  125.17404 hz,   48 bytes read

Yay! We got the 500 hz polling rate that we wanted! It means that lsusb gets its data from the original USB descriptor, not the one that we’ve written to, but the kernel actually uses our descriptor to poll the mouse.

Well, I’m not sure if it’s good or bad that it still reports 10, but it works, and it’s good enough for me.

I suspect there might be a way to make it a kernel module so that we don’t have to patch the kernel every time we update it, but I build my own kernel anyway, so it’s not a problem. I doubt many people would need this feature for that old mouse, so as long as it works for me it’s good enough.

Hope it inspires you to hack on the kernel and make your own first patch for Linux!

Also take a look at ArchWiki article on mouse polling rate for general info on mouse polling rate in Linux.