- ↔
- →
- GitLab Values | The GitLab Handbook
- How to make task list less depressing? - The Workplace Stack Exchange
- Our Values - Zed - The editor for what's next
- Shottr – Screenshot Annotation App For Mac
- crawshaw - 2025-06-08
- June 18, 2025
-
🔗 julesmons/the-augster v6.3.0: Integration with the new native task-management system. release
What's Changed
- feat: Merge v6.0.0 from next into development by @julesmons in #9
- merge: v6.0.0 from next/development into main. by @julesmons in #10
- feat: Comprehensively improved clarity and quality of directives. by @julesmons in #11
- fix: Apply certain fixes as detailed within commits. by @julesmons in #12
- release: v6.1.0, containing some patches to the re-write. by @julesmons in #13
- fix: RESOLVE CRITICAL MISTAKE OF NOT GIVING THE VERIFICATION LIST A HEADER by @julesmons in #15
- release: v6.2.1 - RESOLVE CRITICAL MISTAKE by @julesmons in #16
- feat: Add responsive-ui heuristic by @julesmons in #18
- feat: Add important upgrade information to the README, that details a… by @julesmons in #19
- Hotfix/important upgrade notes by @julesmons in #20
- feat: Integration with the new native task-management system. by @julesmons in #21
New Contributors
- @julesmons made their first contribution in #9
Full Changelog :
v6.2.1...v6.3.0
-
🔗 @trailofbits@infosec.exchange As a Go developer, do you fully understand Go's JSON/XML/YAML parsers? They mastodon
As a Go developer, do you fully understand Go's JSON/XML/YAML parsers? They are surprisingly prone to attacks with simple misconfigurations:
Three unexpected attack scenarios:
1. Marshaling private data with misconfigured tags
2. Parser differentials in a microservices architecture
3. Cross-format confusion attacks (JSON→XML)https://blog.trailofbits.com/2025/06/17/unexpected-security-footguns-in-gos- parsers/
-
🔗 @malcat@infosec.exchange If you need to identify [#malware](https://infosec.exchange/tags/malware) mastodon
If you need to identify #malware quickly, give #malcat a try: its Kesakode code identification is fast and can even work offline!
More info: https://doc.malcat.fr/analysis/kesakode.html
-
🔗 Servo Blog This month in Servo: color inputs, SVG, embedder JS, and more! rss
Two big pieces of news for images in Servo this month:
- We now display animated GIFs in all their animated glory (@rayguo17, #36286)! This work required careful architecting to integrate with existing animation mechanisms in the engine without incurring unnecessary CPU usage.
- We support loading SVG images in < img src> (@mukilan, @mrobinson, #36721).
Outreachy [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#outreachy)
We’re excited to host two Outreachy interns over the next few months! Jerens Lensun (@jerensl) will be working on improving Servo’s CI setup and other Python-focused infrastructure, while Usman Baba Yahaya (@uthmaniv) will implement support for the Network Monitor in our devtools.
They will both be blogging about their internships, and you can follow their work on Jeren’s blog and Usman’s blog.
Web content [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#web-content)
Servo’s layout implementation has historically been all-or-nothing — any change in the page, no matter how isolated, requires laying out the entire page from scratch. Fixing this limitation is known as incremental layout , and it’s a key performance optimization in all browser engines. This month we’ve landed a number of changes in this area that make some kinds of CSS changes much more efficient than a full layout (@mrobinson, @Loirooriol, #36896, #36978, #37004, #37047, #37069, #37048, #37088, #37099).
We have also made significant progress on the Trusted Types API , going from 47% of tests passing to 58% over the course of May (@TimvdLippe, #36710, #36668, #36811, #36824, #36941, #36960). Supporting this work on Trusted Types, our Content Security Policy implementation has been steadily improving, now passing 59% of automated tests (@TimvdLippe, @jdm, @simonwuelker, #36709, #36710, #36776, #36832, #36860, #36887, #36923, #36963, #36962, #36961, #36965, #37020).
We’ve enabled support for URLPattern (@simonwuelker, #36826, #37004, #37116), < input type=color> (@simonwuelker, #36992), plus several other web API features:
- TransformStream (@Taym95, @gterzian, #36739, #36905)
- setHTMLUnsafe() method on ShadowRoot (@TG199, #36240)
- scrollingElement property on Document (@JimmyDdotEXE, #35994)
- pipeThrough() method on ReadableStream (@Taym95, #36977)
- readText() method on navigator.clipboard (@Gae24, #36689)
- type property on Stylesheet (@simonwuelker, #37126)
Our layout and CSS support continues to improve. This month, we improved our page background sizing and style computation (@mrobinson, @Loirooriol, #36917, #37147), and added support for ‘wavy’ and ‘double’ in the ‘text-decoration-line’ property (@mrobinson, #37079).
HTMLVideoElement can now be used as an image source for 2D canvas APIs (@tharkum, #37135), ImageBitmap can be serialized and transferred via postMessage() (@tharkum, #37101), media elements redraw properly whenever their size changes (@tharkum, #37056), polygon image map areas are clickable (@arihant2math, #37064), < select> elements are redrawn when their contents change (@simonwuelker, #36958), and getPreferredCanvasFormat() on GPU returns platform-appropriate values (@arihant2math, #37073).
We’ve fixed bugs relating to invertible and non-invertible transforms (@Loirooriol, #36749, #37147), missing underlines on macOS (@mrobinson, #37029), and sizing issues for tables and flex containers (@stevennovaryo, @Loirooriol, #36703, #36993, #36980, #37024, #37011). We’ve also fixed a number of bugs where Servo’s behaviour did not match relevant specifications:
input
events are now fired followingkeydown
events (@yezhizhen, #37078)- unscopable objects are now writable and readable, and don’t have a prototype (@simonwuelker, #37119, #37122)
Request
headers reject more erroneous headers (@sebsebmc, #36943)- External stylesheets in documents with quirks mode are more lenient about the stylesheet’s
Content-Type
(@ghostd, @mrobinson, #28321) - the
ImageData
constructor throws better errors for unsupported arguments (@Taym95, #31398) - Attribute nodes are serialized as the empty string (@simonwuelker, #36875)
- custom element
is
values are serialized as attributes (@simonwuelker, #36888) EventSource
ignores invalid field values and treats non-200 responses codes as failures (@KiChjang, #36853, #36854)- the
premultipliedAlpha
flag for WebGL canvases premultiplies correctly (@tharkum, #36895)
Our WebDriver server implementation received a lot of attention this month! Element clicks now receive the expected button value (@longvatrong111, #36871), wheel actions are supported (@PotatoCP, #36744, #36985), and we removed the possibility of races between some input actions and other WebDriver commands (@longvatrong111, @mrobinson, #36932). We’ve also added support for passing WebDriver references to DOM objects as arguments when executing scripts (@jdm, #36673), and fixed some bugs with JS value serialization (@yezhizhen, #36908) and cancelling inputs (@yezhizhen, #37010).
We’ve begun preparatory work to integrateVello as the backend for 2D canvases (@sagudev, #36783, #36790, #36999). We’ve also landed some changes towards supporting ‘::placeholder’ pseudo-elements and fixing rendering issues with text inputs (@stevennovaryo, #37065).
Embedding [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#embedding)
The engine [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#the-engine)
Embedders can now evaluate JavaScript inside a webview and receive results asynchronously (@Narfinger, @mrobinson, #35720).
All embedders will receive default styling and interactivity for elements like inputs and media elements (@webbeef, #36803), reducing the amount of configuration required to embed the engine.
Any provided system light/dark theme will be propagated to all documents loaded inside of a webview (@mrobinson, #37132).
Servo’s developer tools integration now highlights elements in the layout inspector (@simonwuelker, #35822), and displays <!DOCTYPE> nodes correctly (@simonwuelker, #36787).
We have removed the
dom_shadowdom_enabled
preference, since the feature has been enabled by default since March 2025 (@simonwuelker, #37043).Our automated benchmarking setup is expanding, and we can now measure how long it takes to start up Servo and load the servo.org homepage on HarmonyOS (@Narfinger, #36878), which will help us identify regressions in the future.
Finally, we can now write unit tests for Servo’s embedding API (@mrobinson, #36791), which allows us to write better regression tests for shutdown-related issues (@mrobinson, #36808).
servoshell [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#servoshell)
The
--user-agent
(-u
) flag now correctly sets the User-Agent header for network requests (@PartiallyUntyped, @mrobinson, #36859).Service workers have been removed from the list of features enabled by
--enable-experimental-web-platform-features
until they provide more value (@jdm, #36867).Building servoshell with
--with-asan
now causes all C++ dependencies to be built with Address Sanitizer as well, andmach bootstrap
on Windows can now usewinget
as a fallback ifchoco
is unavailable (@jschwe, #32836).The current system light/dark theme is now queried on startup (@Legend- Master, #37128). Additionally, the screen dimensions and geometry reported by the engine are now correct on OpenHarmony (@PartiallyUntyped, @jschwe, #36915).
Performance [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#performance)
Servo is now better at evicting image data from GPU caches (@webbeef, #36956). We also reduced the memory needed to store HSTS data, saving more than 60mb by doing so (@sebsebmc, #37000, #37015).
We now measure the memory usage of sessionStorage and localStorage data (@jdm, #37053), the Public Suffix List (@sebsebmc, #37049), and system fonts (@jdm, #36834).
In addition, we’ve reduced the size of the final Servo binary by 2 MB by stripping out DOM code that should never be used outside of automated tests (@jdm, #37034).
Stability [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#stability)
We fixed a number of crashes involving animated images (@simonwuelker, #37058), media elements with an unknown duration (@tharkum, servo- media#437), canvas elements during shutdown (@mrobinson, #37182), adding a Path2D to itself (@Taym95, #36847), calculating IntersectionObserver areas (@webbeef, #36955), the childNodes() method on Node (@jdm, #36889), resizing OffscreenCanvas (@simonwuelker, #36855), querying WebGL extensions (@mrobinson, #36911), and slicing a sliced Blob (@simonwuelker, #36866).
We’ve also fixed a deadlock involving streams with very large chunks (@wusyong, #36914), and fixed a source of intermittent crashes when closing tabs or removing iframes (@jdm, #37120). Finally, we rewrote the implementation of the text property on HTMLOptionElement to avoid crashes with deeply-nested elements (@kkoyung, #37167).
Having previously noticed an unsafe pattern triggered by using JS-owned values in Rust Drop implementations (#26488), we have begun incrementally removing existing Drop impls to remove that source of unsafety (@willypuzzle, #37136).
Upgrades [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#upgrades)
We upgraded our fork of WebRender to April 2025 (@mrobinson, #36770), and upgraded our Stylo dependency to May 2025 (@Loirooriol, #36835). These changes ensure that Servo is up to date with ongoing work in Firefox, which shares these dependencies.
Donations [ __](https://servo.org/blog/2025/06/18/this-month-in-
servo/#donations)
Thanks again for your generous support! We are now receiving 4597 USD/month (−1.4% over April) in recurring donations. This helps cover the cost of our self-hosted CI runners and one of our latest Outreachy interns!
Servo is also on thanks.dev, and already 25 GitHub users (+1 over April) that depend on Servo are sponsoring us there. If you use Servo libraries like url, html5ever, selectors, or cssparser, signing up for thanks.dev could be a good way for you (or your employer) to give back to the community.
4597 USD/month
10000
As always, use of these funds will be decided transparently in the Technical Steering Committee. For more details, head to our Sponsorship page.
-
- June 17, 2025
-
🔗 organicmaps/organicmaps 2025.06.12-3-android release
Detailed release information: https://organicmaps.app/news/2025-06-15/save- planned-routes-in-june-2025-update/
• New OSM data as of June 8
• Save planned routes as tracks
• Display plant nurseries, traffic barriers, studios, firepits, ladders, cranes, and love hotels
• Qingdao metro icons
• Show azimuth when tapping on a small arrow with distance
• Fixed crashes and white-on-white text on Android 5&6
• Fixed search for Lithuanian and other languages
• Fix location arrow jumps when navigation is active
• Fixed Cancel Download button
• Improved translations from Weblate…more at omaps.org/news
See a detailed announce on our website when app updates are published in all stores.
You can get automatic app updates from GitHub using Obtainium.sha256sum:
d50462ca3f4a9eb727f97c41f86acc3e8bb153b4f5d7fe19a38f4960d95e4a55 OrganicMaps-25061203-web-release.apk
-
🔗 News Minimalist OpenAI will develop AI for US military + 4 more stories rss
In the last 4 days ChatGPT read 102549 top news stories. After removing previously covered events, there are 5 articles with a significance score over 5.9.
[5.6] OpenAI will develop AI for US military—independent.co.uk(+8)
OpenAI secured a $200 million contract with the US military to develop AI for national security, including "warfighting" applications.
The contract is part of OpenAI's new "OpenAI for Government" initiative and follows the quiet removal of a ban on military use of its AI.
The Pentagon stated the deal involves developing AI for both warfighting and administrative functions, while OpenAI emphasized the latter.
[6.3] Nuclear powers modernize arsenals, sparking a highly technological arms race —noticias.uol.com.br(Portuguese) (+30)
A report by SIPRI released Monday indicates a new, technologically advanced nuclear arms race is emerging as nuclear states modernize arsenals. This trend could reverse the post-Cold War decline in overall nuclear warhead numbers.
Key nuclear powers, particularly the U.S. and Russia, are updating existing weapons and adding new versions. China's arsenal is rapidly growing, potentially reaching 1,000 warheads within seven to eight years.
The report highlights a total of 12,241 warheads in January 2025 and notes modernization efforts across various countries. The new arms race involves advancements in space, cyberspace, and potentially artificial intelligence.
[6.2] European satellites now create artificial eclipses for corona study —apnews.com(+23)
European satellites have successfully created the first artificial solar eclipses, providing scientists with extended observation opportunities.
Two satellites, launched late last year, are flying in precise formation, with one blocking the sun while the other observes the corona. The Proba-3 mission, costing $210 million, has already produced ten successful eclipses, with the longest lasting five hours.
The mission aims to generate nearly 200 eclipses over two years, offering over 1,000 hours of totality, a significant increase compared to natural eclipses. This will allow scientists to study the sun's corona in detail.
Highly covered news with significance over 5.5
[5.8] UN forecasts worsening hunger in global hotspots — reuters.com [$] (+4)
[5.5] Johns Hopkins blood test detects cancer three years early — independent.co.uk (+2)
Thanks for reading!
— Vadim
You can create your own personal newsletter like this with premium.
-
🔗 benji.dog "I like turtles" rss
Angelo added a Turtle interpreter to his site. Here's my attempt at recreating the IndieWeb logo with it:
This is what it looks like when it's being drawn:
And finally, here's the code to do it yourself.
[](https://benji.dog/articles/i-like-turtles/#c15l1)setwidth 4 [](https://benji.dog/articles/i-like-turtles/#c15l2)penup [](https://benji.dog/articles/i-like-turtles/#c15l3)right 180 [](https://benji.dog/articles/i-like-turtles/#c15l4)forward 150 [](https://benji.dog/articles/i-like-turtles/#c15l5)right 180 [](https://benji.dog/articles/i-like-turtles/#c15l6) [](https://benji.dog/articles/i-like-turtles/#c15l7)setcolor #FF0000 [](https://benji.dog/articles/i-like-turtles/#c15l8)pendown [](https://benji.dog/articles/i-like-turtles/#c15l9)repeat 2 [ forward 60 right 90 forward 20 right 90 ] [](https://benji.dog/articles/i-like-turtles/#c15l10) [](https://benji.dog/articles/i-like-turtles/#c15l11)penup [](https://benji.dog/articles/i-like-turtles/#c15l12)right 90 [](https://benji.dog/articles/i-like-turtles/#c15l13)forward 30 [](https://benji.dog/articles/i-like-turtles/#c15l14)left 90 [](https://benji.dog/articles/i-like-turtles/#c15l15) [](https://benji.dog/articles/i-like-turtles/#c15l16)pendown [](https://benji.dog/articles/i-like-turtles/#c15l17)repeat 2 [ forward 60 right 90 forward 40 right 90 ] [](https://benji.dog/articles/i-like-turtles/#c15l18) [](https://benji.dog/articles/i-like-turtles/#c15l19)penup [](https://benji.dog/articles/i-like-turtles/#c15l20)left 90 [](https://benji.dog/articles/i-like-turtles/#c15l21)forward 30 [](https://benji.dog/articles/i-like-turtles/#c15l22)right 90 [](https://benji.dog/articles/i-like-turtles/#c15l23)forward 70 [](https://benji.dog/articles/i-like-turtles/#c15l24) [](https://benji.dog/articles/i-like-turtles/#c15l25)setcolor #ff5c00 [](https://benji.dog/articles/i-like-turtles/#c15l26)pendown [](https://benji.dog/articles/i-like-turtles/#c15l27)forward 20 [](https://benji.dog/articles/i-like-turtles/#c15l28)right 65 [](https://benji.dog/articles/i-like-turtles/#c15l29)forward 60 [](https://benji.dog/articles/i-like-turtles/#c15l30)right 65 [](https://benji.dog/articles/i-like-turtles/#c15l31)forward 20 [](https://benji.dog/articles/i-like-turtles/#c15l32)right 115 [](https://benji.dog/articles/i-like-turtles/#c15l33)forward 75 [](https://benji.dog/articles/i-like-turtles/#c15l34) [](https://benji.dog/articles/i-like-turtles/#c15l35)penup [](https://benji.dog/articles/i-like-turtles/#c15l36)right 115 [](https://benji.dog/articles/i-like-turtles/#c15l37)forward 30 [](https://benji.dog/articles/i-like-turtles/#c15l38) [](https://benji.dog/articles/i-like-turtles/#c15l39)pendown [](https://benji.dog/articles/i-like-turtles/#c15l40)forward 65 [](https://benji.dog/articles/i-like-turtles/#c15l41)right 115 [](https://benji.dog/articles/i-like-turtles/#c15l42)forward 75 [](https://benji.dog/articles/i-like-turtles/#c15l43)right 130 [](https://benji.dog/articles/i-like-turtles/#c15l44)forward 75 [](https://benji.dog/articles/i-like-turtles/#c15l45) [](https://benji.dog/articles/i-like-turtles/#c15l46)penup [](https://benji.dog/articles/i-like-turtles/#c15l47)right 115 [](https://benji.dog/articles/i-like-turtles/#c15l48)forward 90 [](https://benji.dog/articles/i-like-turtles/#c15l49) [](https://benji.dog/articles/i-like-turtles/#c15l50)setcolor #ffb100 [](https://benji.dog/articles/i-like-turtles/#c15l51)pendown [](https://benji.dog/articles/i-like-turtles/#c15l52)repeat 3 [ forward 23 right 32 ] [](https://benji.dog/articles/i-like-turtles/#c15l53)right 83 [](https://benji.dog/articles/i-like-turtles/#c15l54)forward 40 [](https://benji.dog/articles/i-like-turtles/#c15l55)left 90 [](https://benji.dog/articles/i-like-turtles/#c15l56)forward 5 [](https://benji.dog/articles/i-like-turtles/#c15l57)left 90 [](https://benji.dog/articles/i-like-turtles/#c15l58)forward 40 [](https://benji.dog/articles/i-like-turtles/#c15l59)right 83 [](https://benji.dog/articles/i-like-turtles/#c15l60)repeat 6 [ right 35 forward 23 ] [](https://benji.dog/articles/i-like-turtles/#c15l61)right 40 [](https://benji.dog/articles/i-like-turtles/#c15l62)forward 17
-
🔗 idursun/jjui v0.8.11 release
- 🎉 Added support for handling selected revisions (
space
) to rebase. For example, you can select one revision from each branch and get into rebase mode, set the source to be branch (b
), and then move the cursor tomain
, hitenter
. This will move all branches onto main. - 🎉 Added support for handling selected revisions to
squash
. Select multiple revisions and start squash (S
) and move the cursor to the destination revisions and hit enter. This will squash all selected revisions into the destination revision. - Fixed visual glitch when the revision messages included CJK encoding. #130
- Allow
quit
to be bound to the same key withcancel
. #127 #116 - Updated colours of git, bookmarks, custom commands windows to use Magenta. Previously these windows used bubbletea's default colour palette, which happened to use a shade of magenta as well. This is the first step for simplifying the colour palette of
jjui
. - Fixed a bug where
jjui --config
failed to start if the configuration file was missing.
What's Changed
- Fixup build command from
ebfae40
by @ilyagr in #120 - Fixup build command from
ebfae40
once more by @ilyagr in #123 - Allow selecting multiple revisions during
rebase
andsquash
by @idursun in #124 - Build/update dependencies by @idursun in #126
Full Changelog :
v0.8.10...v0.8.11
- 🎉 Added support for handling selected revisions (
-
- June 16, 2025
-
🔗 dagger/container-use v0.0.5 release
container-use v0.0.5
Download the pre-compiled binaries from the assets below.
Changelog
Full Changelog :
v0.0.4...v0.0.5
-
🔗 dagger/container-use v0.0.4 release
container-use v0.0.4
Download the pre-compiled binaries from the assets below.
Changelog
Full Changelog :
v0.0.3...v0.0.4
-
🔗 dagger/container-use v0.0.3 release
container-use v0.0.3
Download the pre-compiled binaries from the assets below.
Changelog
9167942
: Release to homebrew (#74) (@cwlbraa)758036a
: document how to configure codex mcp (#83) (@kpenfound)159d34d
: rules: removes glob to cursor rules (#84) (@samalba)
Full Changelog :
v0.0.2...v0.0.3
-
🔗 @HexRaysSA@infosec.exchange We're headed to REcon Montreal later this month! mastodon
We're headed to REcon Montreal later this month!
Give us a 👍 if you're going too.Here’s where you can find us:
➥ Wed–Thurs: Supporting the next generation at Blackhoodie Training
➥ Friday: Join us for the "Beyond Decompilation" panel at 5pm
➥ All weekend: Spot us around, share your feedback & snag some swag!We’re not at a booth, so keep an eye out for folks in Hex-Rays attire. 👀
OR, you can skip the hunt and schedule time with us here: https://eu1.hubs.ly/H0k_rjY0 -
🔗 Confessions of a Code Addict Making System Calls in x86-64 Assembly rss
Introduction
In the previous article, we learned to use gdb and used it to debug our crashing program. Eventually, we discovered that after executing the last instruction, the CPU didn't know the program had ended. It continued reading and executing past the end of the .text section in memory, causing a crash. So, we need some way to make our process exit or stop the execution before that happens. How can we do that?
When we ran our program using the shell command
./false
, it was the shell that invoked the fork and execve system calls. These created a new process, loaded our program into memory, and scheduled it for execution on the CPU. Similarly, to terminate our program gracefully, we need to invoke another system call that tells the kernel our process is done.This system call to exit a process is called "
exit
". When we write code in high-level languages, the runtime automatically invokes it after the main function returns. However, when writing freestanding assembly, we need to do it ourselves. For that, we need to learn how to call syscalls from assembly.In this part, we will:
-
Understand what system calls are
-
Learn how to invoke them in assembly
-
Fix our crashing program step-by-step
-
Write a second assembly program using getpid
-
Hands-on exercise: a limited version of the kill command
Recap : If you haven 't seen the previous articles in the series, here's what you have missed:
_This article is part of a paid subscriber series.
If you're enjoying the content, please consider upgrading to a paid plan to unlock the rest of this series. Paid subscribers also get discounted access to courses and books, and the rest of the archive. _Alternatively, you can purchase an ebook version of this series. (If you're already a paid subscriber, email me for a discounted link.)
Understanding System Calls
Before we learn how to invoke system calls, let's first understand why they exist.
Modern operating systems serve two roles: they manage the execution of programs on the CPU and provide safe, unified access to hardware resources like files, memory, and networks. But, application code cannot directly access these hardware features. Why not?
There are three main reasons:
-
Hardware abstraction : Devices vary widely in design and interface. The OS hides this complexity by exposing a uniform way to access them. Whether you're reading from an SSD or a magnetic disk, you use the same system call (read), and the OS handles the details.
-
Portability : Most modern OSes follow the POSIX standard, which defines a consistent set of system calls. If your application uses only POSIX-compliant syscalls, it can compile and run on any compliant OS with minimal changes.
-
Security : If user programs could directly access memory or I/O devices, they could corrupt system state or access other processes' data. System calls act as a controlled gateway; only kernel code (running in a higher privilege level) is allowed to interact directly with hardware.
This separation of privilege is enforced by the CPU. On x86, the kernel runs in ring 0 (full privilege), while user programs run in ring 3 (restricted mode). All system calls are implemented inside the kernel at ring 0. To invoke them from ring 3, user space programs need a way to trigger a transition into kernel mode using a mechanism provided by the CPU.
The protection rings in x86 architecture. Kernel runs at ring-0 level which is the highest privilege mode, while user space in ring-3 which is the least privilege mode
Invoking System Calls on x86-64
-
-
🔗 sacha chua :: living an awesome life 2025-06-16 Emacs news rss
- Upcoming events (iCal file, Org):
- M-x Research: TBA https://m-x-research.github.io/ Wed Jun 18 0800 America/Vancouver - 1000 America/Chicago - 1100 America/Toronto - 1500 Etc/GMT - 1700 Europe/Berlin - 2030 Asia/Kolkata - 2300 Asia/Singapore
- Emacs Berlin (hybrid, in English) https://emacs-berlin.org/ Wed Jun 25 0930 America/Vancouver - 1130 America/Chicago - 1230 America/Toronto - 1630 Etc/GMT - 1830 Europe/Berlin - 2200 Asia/Kolkata – Thu Jun 26 0030 Asia/Singapore
- Emacs APAC: Emacs APAC meetup (virtual) https://emacs-apac.gitlab.io/announcements/ Sat Jun 28 0130 America/Vancouver - 0330 America/Chicago - 0430 America/Toronto - 0830 Etc/GMT - 1030 Europe/Berlin - 1400 Asia/Kolkata - 1630 Asia/Singapore
- Emacs configuration:
- Challenges in rebinding Emacs to "modern" shortcuts (Irreal)
- I wrote a simple and minimal emacs config for developement (Reddit) - based on Doom Emacs
- sirjoaogoncalves/ProEmacs: A modern, fast, and beautiful Emacs configuration for developers who want power without the complexity. (Reddit)
- My Emacs Keyboard bindings (Reddit)
- Emacs Lisp:
- Marcin Borkowski: Automatically converting timestamps in Emacs
- Emacs lisp as an interface definition language · Aleksandr Petrosyan (@tusharhero@mathstodon.xyz)
- Convert UNIX file permission octet (ex: 754) to string (ex: rwxr-xr—)
- Emacs Reader development livestream, working on multi-threading-no-svg branch
- Appearance:
- Navigation:
- Dired:
- Org Mode:
- Whiteboard workflow for Org-mode Using Inkscape (Reddit)
- Showing org mode link at point in echo area
- Task management with org-roam Vol. 5: Dynamic and fast agenda
- pivot-table.el: Build org-mode pivot tables
- [BLOG] #18 [[bbb:OrgMeetup]] on Wed, May 14, 19:00 UTC+3 - Ihor Radchenko (@yantar92@fosstodon.org)
- Completion:
- Coding:
- compile error right from a term in emacs (01:12)
- Making Emacs lsp-mode work with Rust conditional features (Reddit)
- Srijan Choudhary: Generating an Emacs TAGS file for an Erlang project
- magit-emacs-Q-command - runs an uncustomized Emacs with only Magit loaded
- fj.el (forgejo client) 0.10 update: jump from fork to parent, bugfixes, add server-side sorting/ordering
- Mail, news, and chat:
- AI:
- Testers wanted for macher - project-aware multi-file editing with gptel
- Adventures in babysitting coding agents with Steve Yegge, co-author of Vibe Coding (Changelog & Friends #96) (@dotemacs@mastodon.xyz)
- goose.el – A minimal Emacs interface to Goose, the open-source AI agent
- jwiegley/gptel-prompts: Code to make it easier to manage many prompts in GPTel (Reddit)
- Announcing aider.el 0.12.0, LLM work with flycheck, better magit integration, and better file management.
- The Emacs Org-Ai Package: Three Key Features #ai #chatgpt #dalle3 (04:38)
- 022 Ollama Buddy - Prototyping autocompletion #emacs #ollama (00:38)
- Community:
- Other:
- TIL Info-hide-note-references hides inserted "see" before links in Emacs Info
- Save the Last Recorded Macro in Emacs to the Registry (03:45)
- Anand Tamariya: Plan 9 Remote File Access from Emacs (YouTube 1:06)
- early demo of SVG GUI implemented in Emacs
- Build emacs on ubuntu · GitHub
- Step-by-step instructions to build Emacs for Windows 64 bit with MSYS2 and MinGW-w64. Now `native-comp` supported. · GitHub
- setting up Emacs as an OpenRC service (and installing it on Alpine Linux) (12:19)
- Emacs development:
- emacs-devel:
- world-clock: Add user option to control order of entries
- calc: Allow strings with character codes above Latin-1
- ; * etc/NEWS: Announce support for semantic-linefeed filling.
- Add 'treesit-sexp-thing' to use instead of 'treesit-sexp-type-regexp'.
- Add simpler binds for calendar month/year navigation
- vc-next-action: Leave files unregistered if user aborts the checkin
- Improve repeat-continue property handling
- Clean up text properties in 'visual-wrap-prefix-mode'
- New packages:
- magik-company: Magik backend for company-mode (MELPA)
- mcp-server-lib: Model Context Protocol server library (MELPA)
- ordered-set: Insertion-order sets (MELPA)
Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Bluesky #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!
You can e-mail me at sacha@sachachua.com.
- Upcoming events (iCal file, Org):
-
🔗 benji.dog TRMNL Metro Transit plugin rss
I recently got a TRMNL since I wanted an e-ink display to function as a dashboard for our home calendar, picture frame, and other things. Initially I wanted to set this up using an old kindle but even after going through the process of jailbreaking it, I was unable to get this dashboard functionality working.
The TRMNL allows developers to write plugins which requires an additional purchase to unlock the Developer edition. I wasn't quite sure I wanted to commit to the whole thing but I learned that I could use trmnlp to develop plugins locally, and if successful, I could go ahead an purchase the frame and developer add-on.
The first thing I wanted to add was a plugin to get updates from Metro Transit so that I could easily check what my local bus stops next bus was. This is what it looks like:
I had done a version of this before years ago for the Pebble smart watch which used my location to find nearby stops and show me the next buses scheduled for them. I never wrote about it or released the code for it. Luckily, the Metro Transit API is very easy to work and once I figured out where that data was going, all I needed to do was write the Liquid templates to display the information in a format that would most match the UI Metro Transit uses on their bus stops.
I'm very happy how this looks and I have all sorts of ideas for more plugins to write.
If you have a TRMNL, live in the Twin Cities, and like our local public transit system as much as I do, you can install this plugin by forking the plugin and adding your stop ID. You can even do multiple stops by forking it multiple times and adding a new stop to each of them. The code is public as well if you'd like to check that out.
References
-
🔗 Rust Blog Rust compiler performance survey 2025 rss
We're launching a Rust Compiler Performance Survey.
Long compile times of Rust code are frequently being cited as one of the biggest challenges limiting the productivity of Rust developers. Rust compiler contributors are of course aware of that, and they are continuously working to improve the situation, by finding new ways of speeding up the compiler, triaging performance regressions and measuring our long-term performance improvements. Recently, we also made progress on some large changes that have been in the making for a long time, which could significantly improve compiler performance by default.
When we talk about compilation performance, it is important to note that it is not always so simple as determining how long does it take
rustc
to compile a crate. There are many diverse development workflows that might have competing trade-offs, and that can be bottlenecked by various factors, such as the integration of the compiler with the used build system.In order to better understand these workflows, we have prepared a Rust Compiler Performance Survey. This survey is focused specifically on compilation performance, which allows us to get more detailed data than what we usually get from the annual State of Rust survey. The data from this survey will help us find areas where we should focus our efforts on improving the productivity of Rust developers.
You can fill out the surveyhere.
Filling the survey should take you approximately 10 minutes, and the survey is fully anonymous. We will accept submissions until Monday, July 7th, 2025. After the survey ends, we will evaluate the results and post key insights on this blog.
We invite you to fill the survey, as your responses will help us improve Rust compilation performance. Thank you!
-
- June 15, 2025
-
🔗 jellyfin/jellyfin 10.11.0 RC2 release
🚀 Jellyfin Server 10.11.0 RC2
We are pleased to announce the second release candidate preview release of Jellyfin 10.11.0!
This is a preview release, intended for those interested in testing 10.11.0 before it's final public release. We welcome testers to help find as many bugs as we can before the final release.
As always, please ensure you stop your Jellyfin server and take a full backup before upgrading!
Important Notes & Features
Please see the WIP release notes here for now: https://notes.jellyfin.org/v10.11.0_features
PLEASE READ THOSE NOTES THOROUGHLY BEFORE UPGRADING ; current RC1 users should have a seamless upgrade. If you have any questions, please ask in our Matrix chat.
Installing
This preview release is distributed in all our traditional forms, though not automatically via our Apt repository or
latest
tag.- For all non-Docker environments, you can find the files for manual download in our repository by selecting "Stable Preview" for your OS. Note that Windows Installers (.exe) are currently missing due to build issues and will be available with RC2.
- For Docker, you can pull the
10.11.0-rc2
orpreview
tags.
What's Changed (since RC1)
- Only show log in Local network by @JPVenson in #14241
- Don't attempt to do metadata removal for dovi without fallback by @gnattu in #14240
- Use filename for single videos (non-movie/null collections) in MovieResolver by @theguymadmax in #14162
- Rework startup topic handling and reenable output to logging framework by @JPVenson in #14243
- util forward headers on startup api by @JPVenson in #14246
- Fix ExcludeItemId, ExcludeProviderIds and HasAnyProviderId filter by @JPVenson in #14249
- Pin Skiasharp version to 3.116.1 by @gnattu in #14255
- Ignore null key virtual folders by @JPVenson in #14253
- Feature/persistent watch data by @JPVenson in #14262
- fix(collection): Do not lock newly created collections by @darioackermann in #14259
- fix(Session): don't query DB if queue hasn't changed by @Skaytacium in #14244
- Update dependency z440.atl.core to 6.25.0 by @renovate in #14257
- chore/typo by @IDisposable in #14264
- Fix existing media segments not being handled on scan by @ThunderClapLP in #14218
- Fix schema name on backup by @JPVenson in #14269
- Update Microsoft to 9.0.6 by @renovate in #14274
- Update dependency dotnet-ef to 9.0.6 by @renovate in #14273
- Update dependency Polly to 8.6.0 by @renovate in #14279
- Fix UserData cleanup task and queries by @JPVenson in #14280
- Properly handle file access issues in some cases by @Shadowghost in #14272
- Update github/codeql-action action to v3.29.0 by @renovate in #14287
- Fix People Issues by @Shadowghost in #14284
- Remove appsettings.json loading component from startup server by @JPVenson in #14275
- Add explicit check for placeholder ID by @JPVenson in #14298
- Only remove image file if it exists by @Shadowghost in #14302
- Always set update action when item does not exist by @JPVenson in #14304
- Use proper scheduler that honors the parallel task limit by @JPVenson in #14281
- Update dependency z440.atl.core to 6.26.0 by @renovate in #14315
- Use square root scaling for high framerate videos' bitrate requirements by @gnattu in #14314
New Contributors
- @darioackermann made their first contribution in #14259
- @Skaytacium made their first contribution in #14244
- @ThunderClapLP made their first contribution in #14218
Full Changelog :
v10.11.0-rc1...v10.11.0-rc2
What's Changed (since 10.10.x, to RC1)
- Update issue template version from 10.9.11 to 10.10.0 by @HadrienPatte in #12882
- Auto update issue template version on new release by @HadrienPatte in #12893
- Update dependency Svg.Skia to 2.0.0.2 by @renovate in #12922
- Added + in username regex validator, Test + in username, issue #10414 by @SethPattee in #12819
- Add EpisodeExpression for anime file names by @TonyBotongChu in #12778
- Update dependency z440.atl.core to 6.7.0 by @renovate in #12943
- Update github/codeql-action action to v3.27.1 by @renovate in #12992
- Update dependency z440.atl.core to 6.8.0 by @renovate in #12994
- Update dependency AsyncKeyedLock to 7.1.3 by @renovate in #13007
- Update dotnet monorepo by @renovate in #12792
- Update CI dependencies by @renovate in #13019
- Update skiasharp monorepo by @renovate in #12986
- Update Microsoft to 8.0.11 by @renovate in #13021
- Update projects to .NET 9 by @Bond-009 in #13023
- Add dotnet9 to abi compat workflow by @crobibero in #13046
- Update Microsoft to v9 (major) by @renovate in #13022
- Fixed segment providers never presented to UI by @JPVenson in #13060
- make playlist creation private by default by @dkanada in #12853
- Update CI dependencies by @renovate in #13056
- Update dependency Microsoft.NET.Test.Sdk to 17.12.0 by @renovate in #13072
- Update dependency AsyncKeyedLock to 7.1.4 by @renovate in #13089
- Update dependency Svg.Skia to 2.0.0.4 by @renovate in #13100
- Fix typo in LibraryOptions by @nielsvanvelzen in #13097
- Fix typo in guide info endpoint comment by @1hitsong in #13117
- Updated DevContainer to Bookworm Debian by @JPVenson in #13037
- Added test for ListsingsManager.DeleteListingsProvider(). by @kennethcochran in #12793
- Move TV-PG ratings to be in line with PG rating. by @Dessyreqt in #12867
- Update dependency MimeTypes to 2.5.2 by @renovate in #11222
- Implement TaskTriggerInfoType enum by @LePips in #12783
- Removed RemoveOldPlugins configuration flag by @RealGreenDragon in #13102
- Update dependency Xunit.SkippableFact to 1.5.23 by @renovate in #13134
- Update dependency z440.atl.core to 6.9.0 by @renovate in #13141
- Update github/codeql-action action to v3.27.6 by @renovate in #13152
- Update dependency dotnet-ef to v9 by @renovate in #13024
- Determine tv image type by extension if content-type is unavailable by @theguymadmax in #13076
- Update dependency Serilog.Settings.Configuration to v9 by @renovate in #13192
- Update github/codeql-action action to v3.27.7 by @renovate in #13203
- Update github/codeql-action action to v3.27.9 by @renovate in #13215
- Migrate rulesets to .editorconf by @Shadowghost in #13195
- Update dependency Serilog.AspNetCore to v9 by @renovate in #13193
- Update actions/upload-artifact action to v4.5.0 by @renovate in #13248
- Update dependency z440.atl.core to 6.10.0 by @renovate in #13230
- move to new System.Threading.Lock type for better performance by @Ich1goSan in #13213
- Update github/codeql-action action to v3.28.0 by @renovate in #13258
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.2 by @renovate in #13263
- Update dependency z440.atl.core to 6.11.0 by @renovate in #13272
- Update dependency libse to 4.0.10 - autoclosed by @renovate in #13271
- Update actions/setup-dotnet action to v4.2.0 by @renovate in #13286
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.3 by @renovate in #13291
- Update dependency coverlet.collector to 6.0.3 by @renovate in #13304
- Update eps1lon/actions-label-merge-conflict action to v3.0.3 by @renovate in #13327
- Update dependency xunit to 2.9.3 by @renovate in #13339
- Update CI dependencies by @renovate in #13347
- Prefer ApiKey over api_key in generated URL's by @nielsvanvelzen in #13342
- Add option to disable deprecated legacy authorization options by @nielsvanvelzen in #13306
- Update dependency FsCheck.Xunit to v3 by @renovate in #13333
- Update dependency z440.atl.core to 6.12.0 by @renovate in #13353
- Update Microsoft to 9.0.1 by @renovate in #13373
- Update dependency dotnet-ef to 9.0.1 by @renovate in #13374
- Update dependency SharpFuzz to 2.2.0 by @renovate in #13354
- Add ability to remove a ChannelMapping by @Giermann in #12970
- Update dependency coverlet.collector to 6.0.4 by @renovate in #13395
- Update dependency z440.atl.core to 6.13.0 by @renovate in #13403
- Update CI dependencies by @renovate in #13400
- Fix .gzip handling and URL redirection for XML TV guide parsing by @theguymadmax in #13319
- improve documentation for ContainerHelper class by @reuterma24 in #13360
- Update github/codeql-action action to v3.28.3 by @renovate in #13408
- Update dependency FsCheck.Xunit to 3.0.1 by @renovate in #13407
- Refactor library.db into jellyfin.db and EFCore by @JPVenson in #12798
- Update github/codeql-action action to v3.28.5 by @renovate in #13410
- Remove the ability to auto port forward by @Bond-009 in #13222
- Fix spelling by @jsoref in #11103
- Fix various typos by @luzpaz in #13436
- Update dependency z440.atl.core to 6.14.0 by @renovate in #13431
- Use MediaTypeNames where possible by @Bond-009 in #13440
- Fix typos by @luzpaz in #13438
- Update CI dependencies by @renovate in #13452
- Fix spelling by @jsoref in #13444
- chore(ci): Let CI fail independently on each platform by @jsoref in #13446
- Remove useless checks and dead code by @Bond-009 in #13405
- Always await instead of directly returning Task by @Bond-009 in #12925
- Fix source typo by @luzpaz in #13453
- Update github/codeql-action action to v3.28.7 by @renovate in #13458
- Update CI dependencies by @renovate in #13460
- Removing CollectionFolders from cache when they are deleted on disk. by @sinterdev in #13315
- Update dependency z440.atl.core to 6.15.0 by @renovate in #13477
- Fix build after backports due to EFCore change by @Bond-009 in #13488
- Update dependency FsCheck.Xunit to 3.1.0 by @renovate in #13463
- Make StartDate/EndDate nullable by @gnattu in #13494
- Enable nullable for AuthorizationInfo by @Bond-009 in #13485
- Order MediaStream query by StreamIndex by @gnattu in #13506
- Remove check-backport CI action by @joshuaboniface in #13523
- Handle empty image lists gracefully in SplashscreenPostScanTask by @gnattu in #13498
- Update github/codeql-action action to v3.28.9 by @renovate in #13517
- Update dependency z440.atl.core to 6.16.0 by @renovate in #13521
- Rename CreateOrUpdateItems back to CreateItems by @Bond-009 in #13527
- Fix Search results are case-sensitive for people by @tkloy24 in #13516
- Simulate old GetItemValueNames behavior by @gnattu in #13539
- Update dependency dotnet-ef to 9.0.2 by @renovate in #13548
- Update Microsoft - autoclosed by @renovate in #13534
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.4 by @renovate in #13528
- Write only for query columns to EFCore db by @gnattu in #13542
- Fix the issue where the external audio track always defaults. by @choyakawa in #13132
- Make the JsonConverters for delimited arrays more generic by @Bond-009 in #13396
- Change BaseItemEntity ChannelId to nullable Guid by @crobibero in #13553
- Disallow incremental updates to JellyfinDbModelSnapshot by @crobibero in #13564
- Revert nullability of MediaStream.IsHearingImpaired by @crobibero in #13573
- Update dependency z440.atl.core to 6.17.0 by @renovate in #13567
- Don't allow usernames to have leading or trailing spaces by @Jxiced in #13556
- Update appleboy/ssh-action action to v1.2.1 by @renovate in #13584
- Update CI dependencies by @renovate in #13603
- Update actions/download-artifact action to v4.1.9 by @renovate in #13625
- Add support for reading and storing Recording MBIDs from file metadata by @lyarenei in #12173
- Better exception message when folders or folder items are missing by @IDisposable in #13632
- Remove deprecated GetWakeOnLanInfo endpoint by @nielsvanvelzen in #13606
- Don't use RETURNING clause with EFCore by @gnattu in #13492
- Update dependency z440.atl.core to 6.18.0 by @renovate in #13608
- Fix possible NullReferenceException in playlist warning by @l2dy in #13643
- Update CI dependencies by @renovate in #13671
- Update dependency z440.atl.core to 6.19.0 by @renovate in #13683
- Include SortName in LibraryDb migration query by @te9c in #13675
- Extract trickplay files into own subdirectory by @Shadowghost in #13406
- Include CleanName in LibraryDb migration query by @theguymadmax in #13690
- Sort embedded collections in Nfo files by @IDisposable in #9560
- Migrate to IExternalUrlProvider by @Shadowghost in #13175
- Update Microsoft to 9.0.3 by @renovate in #13702
- Update dependency dotnet-ef to 9.0.3 by @renovate in #13703
- Update dependency python to 3.13 - autoclosed by @renovate in #13701
- Fix build and tests by @Shadowghost in #13718
- Add start index to /Programs/Recommended endpoint by @Bond-009 in #13696
- Fix subnet contains check by @gnattu in #13493
- Update actions/setup-dotnet action to v4.3.1 by @renovate in #13727
- Add fast-path to getting just the SeriesPresentationUniqueKey for NextUp by @crobibero in #13687
- Fix subtitle selection to respect preferred language for forced subtitles by @timminator in #13098
- Update actions/download-artifact action to v4.2.0 by @renovate in #13734
- Update CI dependencies by @renovate in #13738
- Fix IMDb URL for People by @theguymadmax in #13724
- Fix OnPlaybackStopped task erroring out when closing a browser tab with a livestream that is transcoding by @timminator in #13226
- 2x faster library.db migration, reduced memory pressure by @ferferga in #13749
- Rename Islamic Republic of Pakistan to Pakistan by @adiled in #13752
- Remove all DB data on item removal, delete internal trickplay files by @Shadowghost in #13753
- Rework season folder parsing by @Shadowghost in #11748
- Added Setup overlay app to communicate status of startup by @JPVenson in #12880
- Add missing singleton by @Shadowghost in #13761
- Fix Invalid Item Queries by @JPVenson in #13757
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.5 by @renovate in #13759
- Disable flaky tests by @JPVenson in #13765
- Add ability to provide search pattern to GetFiles by @NooNameR in #13691
- Include PeopleBaseItemMap in GetPeople to inlcude Role and SortOrder by @Lampan-git in #13616
- Add OpenAPI spec for #12880 by @JPVenson in #13764
- Fix Cleanup task not awaiting async methods by @JPVenson in #13769
- [Feature] Database code refactor by @JPVenson in #13589
- [NOT MERGED!]Add Postgres SQL support by @JPVenson in #13451
- Update CI dependencies by @renovate in #13766
- Feature/backup on migration by @JPVenson in #13754
- Extract container, video and audio compatibility checks by @dmitrylyzo in #12678
- Change the order of the iso6392.txt file by @baka0815 in #13314
- Fix only returning one item from /Item/Latest api. by @scampower3 in #12492
- Add override for migration if library.old aready exists by @JPVenson in #13779
- Add eac3 as an audio name format by @theguymadmax in #13784
- Include UnratedType in LibraryDb migration query by @theguymadmax in #13783
- Fix consumer count off by one when closing a browser tab with a livestream that is transcoding by @timminator in #13220
- Fix validation of VAAPI/QSV render node path by @nyanmisaka in #13786
- Fix Sort by Year Bug (#12101) by @jjwarrenSEP in #13733
- Add channel queries to series by @Kevinjil in #13356
- Improve SkiaEncoder's font handling by @gnattu in #13231
- Implement limiting caches by @crobibero in #13605
- Fix for Issue #12142: Fix ExtraRuleResolver filtering out top level folders by @mcmcelro in #12170
- Fix build and tests by @crobibero in #13790
- Add profile condition to limit the number of streams by @dmitrylyzo in #13583
- Reduce allocations, simplifed code, faster implementation, included tests - StreamInfo.ToUrl by @Shadowghost in #9369
- feat: allow grouping shows into collections by @jheuel in #13236
- Fix cleanup of wrong table in migration by @JPVenson in #13796
- Enable VideoToolbox AV1 decode by @gnattu in #13194
- Library.db migration impovements by @JPVenson in #13809
- Fix StreamInfo.ToUrl by @Shadowghost in #13808
- Update dependency z440.atl.core to 6.20.0 by @renovate in #13811
- Fix MoveTrickplayFiles migration by @Shadowghost in #13807
- Fix playlist order by @quyet-v in #13730
- Rework parental ratings by @Shadowghost in #12615
- Use pattern matching for null checks by @Bond-009 in #13793
- Explicitly set default value for enums used in API models by @gnattu in #13821
- Fix Tmdb external URL generation by @Shadowghost in #13817
- Reduce Skia conversions by @ferferga in #5366
- Add Dolby Vision tests for Tizen by @dmitrylyzo in #12670
- Preserve SplashscreenLocation when updating branding config by @KGT1 in #13756
- Cleanup ItemFields by @Shadowghost in #13818
- Add missing public properties to SystemInfo response by @thornbill in #13822
- allow admin users to get Splashscreen even when it's disabled by @KGT1 in #13825
- Improve dynamic HDR metadata handling by @gnattu in #13277
- Import Keyframes into database by @Shadowghost in #13771
- Trim library names by @JPVenson in #13828
- Only remove keyframe cache dir if it exists by @Shadowghost in #13834
- Cleanup extracted files by @Shadowghost in #13760
- Make ReadInputAtNativeFramerate configurable for M3U tuner by @timminator in #13773
- Fix ArgumentNullException on playlist creation by @Bond-009 in #13837
- Use subdirectories to organize extracted data by @Shadowghost in #13838
- Fix indices and update of ItemValues by @Shadowghost in #13843
- Fix ancestors by @Shadowghost in #13827
- Send Album Artist and Artist in seperate variables when doing a lyrics search by @scampower3 in #13852
- Fix Genre type by @Shadowghost in #13862
- Update github/codeql-action action to v3.28.14 by @renovate in #13863
- Fix backup not written to correct directory by @JPVenson in #13853
- Add API support for ELRC word-based lyrics by @AlexDalas in #12941
- Fix the migration as the new constraint now uses Value as unique key by @JPVenson in #13867
- Translate the ISO-639-2/B codes to ISO-639-2/T. by @baka0815 in #13068
- Update github/codeql-action action to v3.28.15 by @renovate in #13869
- Fix seeking beyond EOF again by @nyanmisaka in #13871
- Use Guid for parentPrimaryImageItemId by @nielsvanvelzen in #13874
- Fix InheritedParentalRatingSubValue not set by @JPVenson in #13880
- Update Microsoft to 9.0.4 by @renovate in #13878
- Update dependency dotnet-ef to 9.0.4 by @renovate in #13879
- Fix negated IP addresses without subnet mask not being parsed correctly by @mapret in #13854
- Only reselect audio streams when user preference is respected by @gnattu in #13832
- Add Genre cleanup and fix cleanup filter queries by @Shadowghost in #13891
- Update dependency Svg.Skia to 2.0.0.7 by @renovate in #13897
- Update dependency FsCheck.Xunit to 3.2.0 - autoclosed by @renovate in #13898
- Add polish age ratings by @relains in #13851
- Add ServerName to startup configuration by @thornbill in #13901
- Remove the hashed password from startup users response by @thornbill in #13904
- Update dependency AsyncKeyedLock to 7.1.6 by @renovate in #13905
- Safeguard against null value trimming in tag results by @Shadowghost in #13908
- Update dependency Svg.Skia to 2.0.0.8 by @renovate in #13907
- Add DoVi Profile 5 support for Rockchip RKMPP by @nyanmisaka in #13911
- Fix Genre cleanup by @Shadowghost in #13916
- Update dependency libse to 4.0.12 by @renovate in #13928
- Fix thumbnail extraction of mpegts videos in FFmpeg 7.1+ by @nyanmisaka in #13942
- Improve video resolution filtering and classification logic by @theguymadmax in #13332
- Fix SyncPlay WebSocket OpenAPI schemas by @nielsvanvelzen in #13946
- Add port awareness to startup server by @JPVenson in #13913
- Fix OverflowException when scanning media with a very short duration by @Bond-009 in #13949
- Return SyncPlay group info after creation, add GET group endpoint by @nielsvanvelzen in #13935
- Add Api and startup check for sufficient storage capacity by @JPVenson in #13888
- Update dependency z440.atl.core to 6.21.0 by @renovate in #13967
- Remove 10.11.z EFcore migration warnings by @ferferga in #13972
- Update CI dependencies by @renovate in #13981
- Deprecate OnPlaybackXXX API operations in favor of ReportPlaybackXXX by @nielsvanvelzen in #13993
- Only run merge conflict labler action on pull requests by @felix920506 in #13378
- Fix IsInMixedFolder not being set for Extras by @tkloy24 in #13536
- Rework chapter management by @Shadowghost in #13847
- Check for path overlaps by @JPVenson in #12832
- Create directory before checking for size by @JPVenson in #13962
- improved performance of save operations by @JPVenson in #13889
- Add .gitignore style ignoring by @Shadowghost in #13906
- Optimize migrations by @Shadowghost in #13855
- Don't pass through timestamp for image extractor by @gnattu in #13999
- Update dependency z440.atl.core to 6.22.0 by @renovate in #13995
- Fix distinction queries by @Shadowghost in #14007
- Fix ItemValue query by @JPVenson in #13939
- Unified migration handling by @JPVenson in #13950
- Add new nuget packages to abi diff by @crobibero in #14016
- Fix image extractor with more strict ffmpeg requirement by @gnattu in #14013
- Fix startup logger log file order by @theguymadmax in #14044
- Cleanup Tasks and Validators by @Shadowghost in #14028
- Update github/codeql-action action to v3.28.17 by @renovate in #14046
- Fix trickplay directory path construction by @theguymadmax in #14036
- Update dependency z440.atl.core to 6.23.0 by @renovate in #14045
- Add "part" as possible album prefix for stacking by @harry-hart in #14022
- Added parental ratings for Indian content by @SoumyadipAuddy in #14050
- Add albumart as recognized filename for music artwork by @theguymadmax in #14048
- Fix the transparency issue of ASS subtitle rendering in HWA by @nyanmisaka in #14024
- Update dependency Serilog.Sinks.File to v7 by @renovate in #14017
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.6 by @renovate in #14058
- fix #14034 Readd Context provider on Initialise by @JPVenson in #14040
- Cleanup file related code by @Bond-009 in #14023
- Recognize file changes and remove data on change by @Shadowghost in #13839
- Fix cross filesystem dir moving by @gnattu in #14063
- Add tests for ManagedFileSystem.MoveDirectory by @Bond-009 in #14065
- Only consider migrations that have key set for migration.xml migration by @JPVenson in #14061
- Use VBR and MBBRC in QSV encoders for better quality by @nyanmisaka in #14079
- Cleanup external item data cleanup by @Shadowghost in #14072
- Reject invalid replaygain tag value by @gnattu in #14082
- Fix Blu-ray metadata: preserve external streams and language metadata by @theguymadmax in #14077
- added different countries parental ratings by @SoumyadipAuddy in #14069
- Update danielpalme/ReportGenerator-GitHub-Action action to v5.4.7 by @renovate in #14090
- Fix extracted data cleanup in cleanup post scan task by @Shadowghost in #14083
- Update appleboy/scp-action action to v1 by @renovate in #14012
- Update Microsoft to 9.0.5 by @renovate in #14103
- Update dependency dotnet-ef to 9.0.5 by @renovate in #14102
- Translate query by AncestorIds correctly by @allesmi in #14094
- Update dependency z440.atl.core to 6.24.0 by @renovate in #14122
- Update github/codeql-action action to v3.28.18 by @renovate in #14119
- Update dependency Microsoft.CodeAnalysis.BannedApiAnalyzers to v4 by @renovate in #14113
- Fix ArgumentNullException in TmdbExternalUrlProvider by @theguymadmax in #14130
- Update skiasharp monorepo (major) by @renovate in #13369
- Add Full system backup feature by @JPVenson in #13945
- Terminate at null char for audio tags by @gnattu in #14100
- Update dependency Microsoft.NET.Test.Sdk to 17.14.0 by @renovate in #14143
- Fix ArgumentNullException when downloading season logos by @theguymadmax in #14141
- Enable OpenCL deinterlacer for AMF on Windows when available by @nyanmisaka in #14144
- Fix collection behavior when sorting by rating or runtime by @theguymadmax in #14148
- Ensure subfolders are processed in folders containing one video file by @theguymadmax in #14140
- Make name sorting case-insensitive by @theguymadmax in #14153
- Add SVG to transparent image types by @theguymadmax in #14160
- Update dependency Svg.Skia to 3.0.3 by @renovate in #13933
- Offload 1080p+ subtitle scaling to RKRGA by @nyanmisaka in #14179
- Round RemoteImage CommunityRating to nearest tenths when sorting by @SenorSmartyPants in #13145
- Localization/iso6392.txt: Correct the Portuguese entries by @baka0815 in #14030
- Fix unittests by @shanepowell in #14202
- Update dependency Microsoft.NET.Test.Sdk to 17.14.1 by @renovate in #14211
- Update github/codeql-action action to v3.28.19 by @renovate in #14212
- Add partition helper by @JPVenson in #14039
- Add multiple options for internal locking by @JPVenson in #14047
- Backup MigrationHistory as well by @JPVenson in #14136
- Update dependency FsCheck.Xunit to 3.3.0 by @renovate in #14216
- Fix server not auto restarting by @JPVenson in #14215
- Add ExcludeItemIds filtering to UserViewBuilder.Filter by @shanepowell in #14203
- Fix missing logging of connections by disallowed IPs by @lf- in #14011
- Prune trickplay data on regenerate and scan by @Shadowghost in #14085
- Update XmlTv tests to use UTC date formats by @telans in #14163
- Add declarative backups for migrations by @JPVenson in #14135
- Allow custom plugin provided database providers to be loaded by @JPVenson in #14171
- Add cache-control: no-cache to index.html if selfhosted by @JPVenson in #14222
- Migrate all known old migrations even when not applied in migration.xml by @JPVenson in #14217
- Feature/version check in library migration by @JPVenson in #14105
- Fix DirectoryNotFoundException for backdrop folders in trickplay by @theguymadmax in #14223
- Fix metadata not refreshing in negative UTC timezones by @theguymadmax in #14225
- Add support for delete from playlist call with api key by @sharinganthief in #14154
- Fix race condition in SessionManager.GetSessionInfo by @moalhaddar in #14131
- Update issue report.yml by @felix920506 in #14099
- Fix broken chapter image placeholders when no image is present by @theguymadmax in #14230
- Fix source directory for setup template by @JPVenson in #14228
- always sort season by index number by @qiqian in #13307
- Add year parameter to TMDB series cache key by @theguymadmax in #14209
- Fix search for missing metadata overriding existing metadata by @scampower3 in #12908
New Contributors
- @HadrienPatte made their first contribution in #12882
- @SethPattee made their first contribution in #12819
- @TonyBotongChu made their first contribution in #12778
- @kennethcochran made their first contribution in #12793
- @Dessyreqt made their first contribution in #12867
- @LePips made their first contribution in #12783
- @Ich1goSan made their first contribution in #13213
- @Giermann made their first contribution in #12970
- @reuterma24 made their first contribution in #13360
- @jsoref made their first contribution in #11103
- @sinterdev made their first contribution in #13315
- @tkloy24 made their first contribution in #13516
- @choyakawa made their first contribution in #13132
- @Jxiced made their first contribution in #13556
- @te9c made their first contribution in #13675
- @timminator made their first contribution in #13098
- @adiled made their first contribution in #13752
- @NooNameR made their first contribution in #13691
- @baka0815 made their first contribution in #13314
- @jjwarrenSEP made their first contribution in #13733
- @Kevinjil made their first contribution in #13356
- @mcmcelro made their first contribution in #12170
- @jheuel made their first contribution in #13236
- @quyet-v made their first contribution in #13730
- @KGT1 made their first contribution in #13756
- @AlexDalas made their first contribution in #12941
- @mapret made their first contribution in #13854
- @relains made their first contribution in #13851
- @harry-hart made their first contribution in #14022
- @SoumyadipAuddy made their first contribution in #14050
- @allesmi made their first contribution in #14094
- @shanepowell made their first contribution in #14202
- @lf- made their first contribution in #14011
- @sharinganthief made their first contribution in #14154
- @moalhaddar made their first contribution in #14131
- @qiqian made their first contribution in #13307
Full Changelog :
v10.10.7...v10.11.0-rc1
-
🔗 Anton Zhiyanov Gist of Go: Race conditions rss
This is a chapter from my book onGo concurrency, which teaches the topic from the ground up through interactive examples.
Preventing data races with mutexes may sound easy, but dealing with race conditions is a whole other matter. Let's learn how to handle these beasts!
Race condition • Compare-and- set • Idempotence and atomicity • Locker • TryLock • Shared nothing • Keep it up
Race condition
Let's say we're keeping track of the money in users' accounts:
// Accounts - money in users' accounts. type Accounts struct { bal map[string]int mu sync.Mutex } // NewAccounts creates a new set of accounts. func NewAccounts(bal map[string]int) *Accounts { return &Accounts{bal: maps.Clone(bal)} }
We can check the balance by username or change the balance:
// Get returns the user's balance. func (a *Accounts) Get(name string) int { a.mu.Lock() defer a.mu.Unlock() return a.bal[name] } // Set changes the user's balance. func (a *Accounts) Set(name string, amount int) { a.mu.Lock() defer a.mu.Unlock() a.bal[name] = amount }
Account operations —
Get
andSet
— are concurrent-safe, thanks to the mutex.There's also a store that sells Lego sets:
// A Lego set. type LegoSet struct { name string price int }
Alice has 50 coins in her account. She wants to buy two sets: "Castle" for 40 coins and "Plants" for 20 coins:
func main() { acc := NewAccounts(map[string]int{ "alice": 50, }) castle := LegoSet{name: "Castle", price: 40} plants := LegoSet{name: "Plants", price: 20} var wg sync.WaitGroup wg.Add(2) // Alice buys a castle. go func() { defer wg.Done() balance := acc.Get("alice") if balance < castle.price { return } time.Sleep(5 * time.Millisecond) acc.Set("alice", balance-castle.price) fmt.Println("Alice bought the castle") }() // Alice buys plants. go func() { defer wg.Done() balance := acc.Get("alice") if balance < plants.price { return } time.Sleep(10 * time.Millisecond) acc.Set("alice", balance-plants.price) fmt.Println("Alice bought the plants") }() wg.Wait() balance := acc.Get("alice") fmt.Println("Alice's balance:", balance) } Alice bought the castle Alice bought the plants Alice's balance: 30
What a twist! Not only did Alice buy both sets for a total of 60 coins (even though she only had 50 coins), but she also ended up with 30 coins left! Great deal for Alice, not so great for us.
The problem is that checking and updating the balance is not an atomic operation:
// body of the second goroutine balance := acc.Get("alice") // (1) if balance < plants.price { // (2) return } time.Sleep(10 * time.Millisecond) acc.Set("alice", balance-plants.price) // (3)
At point ➊, we see a balance of 50 coins (the first goroutine hasn't done anything yet), so the check at ➋ passes. By point ➌, Alice has already bought the castle (the first goroutine has finished), so her actual balance is 10 coins. But we don't know this and still think her balance is 50 coins. So at point ➌, Alice buys the plants for 20 coins, and the balance becomes 30 coins (the "assumed" balance of 50 coins minus the 20 coins for the plants = 30 coins).
Individual actions on the balance are safe (there's no data race). However, balance reads/writes from different goroutines can get "mixed up", leading to an incorrect final balance. This situation is called a race condition.
You can't fully eliminate uncertainty in a concurrent environment. Events will happen in an unpredictable order — that's just how concurrency works. However, you can protect the system's state — in our case, the purchased sets and balance — so it stays correct no matter what order things happen in.
Let's check and update the balance in one atomic operation, protecting the entire purchase with a mutex. This way, purchases are processed strictly sequentially:
// Shared mutex. var mu sync.Mutex // Alice buys a castle. go func() { defer wg.Done() // Protect the entire purchase with a mutex. mu.Lock() defer mu.Unlock() balance := acc.Get("alice") if balance < castle.price { return } time.Sleep(5 * time.Millisecond) acc.Set("alice", balance-castle.price) fmt.Println("Alice bought the castle") }() // Alice buys plants. go func() { defer wg.Done() // Protect the entire purchase with a mutex. mu.Lock() defer mu.Unlock() balance := acc.Get("alice") if balance < plants.price { return } time.Sleep(10 * time.Millisecond) acc.Set("alice", balance-plants.price) fmt.Println("Alice bought the plants") }() Alice bought the plants Alice's balance: 30
One of the goroutines will run first, lock the mutex, check and update the balance, then unlock the mutex. Only after that will the second goroutine be able to lock the mutex and make its purchase.
We still can't be sure which purchase will happen — it depends on the order the goroutines run. But now we are certain that Alice won't buy more than she's supposed to, and the final balance will be correct:
Alice bought the castle Alice's balance: 10
Or:
Alice bought the plants Alice's balance: 30
To reiterate:
- A data race happens when multiple goroutines access shared data, and at least one of them modifies it. We need to protect the data from this kind of concurrent access.
- A race condition happens when an unpredictable order of operations leads to an incorrect system state. In a concurrent environment, we can't control the exact order things happen. Still, we need to make sure that no matter the order, the system always ends up in the correct state.
Go's race detector can find data races, but it doesn't catch race conditions. It's always up to the programmer to prevent race conditions.
Compare-and-set
Let's go back to the situation with the race condition before we added the mutex:
// Alice's balance = 50 coins. // Castle price = 40 coins. // Plants price = 20 coins. // Alice buys a castle. go func() { defer wg.Done() balance := acc.Get("alice") if balance < castle.price { return } time.Sleep(5 * time.Millisecond) acc.Set("alice", balance-castle.price) fmt.Println("Alice bought the castle") }() // Alice buys plants. go func() { defer wg.Done() balance := acc.Get("alice") if balance < plants.price { return } time.Sleep(10 * time.Millisecond) acc.Set("alice", balance-plants.price) fmt.Println("Alice bought the plants") }() Alice bought the castle Alice bought the plants Alice's balance: 30
As we discussed, the reason for the incorrect final state is that buying a set (checking and updating the balance) is not an atomic operation:
// body of the second goroutine balance := acc.Get("alice") // (1) if balance < plants.price { // (2) return } time.Sleep(10 * time.Millisecond) acc.Set("alice", balance-plants.price) // (3)
At point ➊, we see a balance of 50 coins, so the check at ➋ passes. By point ➌, Alice has already bought the castle, so her actual balance is 10 coins. But we don't know this and still think her balance is 50 coins. So at point ➌, Alice buys the plants for 20 coins, and the balance becomes 30 coins (the "assumed" balance of 50 coins minus the 20 coins for the plants = 30 coins).
To solve the problem, we can protect the entire purchase with a mutex, just like we did before. But there's another way to handle it.
We can keep two separate operations (checking and updating the balance), but instead of a regular update (set), use an atomic compare-and-set operation:
// Get returns the user's balance. func (a *Accounts) Get(name string) int { a.mu.Lock() defer a.mu.Unlock() return a.bal[name] } // CompareAndSet changes the user's balance to new // if the current value equals old. Returns false otherwise. func (a *Accounts) CompareAndSet(name string, old, new int) bool { a.mu.Lock() defer a.mu.Unlock() if a.bal[name] != old { return false } a.bal[name] = new return true }
CompareAndSet
first checks if the balance has changed compared to the old value provided by the caller (the "assumed" balance). If the balance has changed (the assumed balance doesn't match the actual balance), it doesn't set the new value and returnsfalse
. If the balance hasn't changed (the assumed balance matches the actual balance), it sets the new value and returnstrue
.Now we can safely sell Lego:
// Alice buys a castle. go func() { defer wg.Done() balance := acc.Get("alice") if balance < castle.price { return } time.Sleep(5 * time.Millisecond) if acc.CompareAndSet("alice", balance, balance-castle.price) { fmt.Println("Alice bought the castle") } }() // Alice buys plants. go func() { defer wg.Done() balance := acc.Get("alice") if balance < plants.price { return } time.Sleep(10 * time.Millisecond) if acc.CompareAndSet("alice", balance, balance-plants.price) { fmt.Println("Alice bought the plants") } }() Alice bought the castle Alice's balance: 10
We no longer use a mutex to protect the entire purchase. Individual operations from different goroutines can get mixed up, but using compare-and-set instead of a regular update protects us from the race condition. If the actual account state doesn't match what we expect (because another goroutine made a change), the update won't happen.
CAS with retry
In practice, a failed compare-and-set is often followed by a retry. In our example, we would re-read the balance with
acc.Get
and, if there is still enough money, retried the purchase withacc.CompareAndSet
:// Alice buys plants (with retries). go func() { defer wg.Done() for { // Start of retry loop. balance := acc.Get("alice") if balance < plants.price { return // Not enough money, exit loop. } // Attempt the purchase. if acc.CompareAndSet("alice", balance, balance-plants.price) { fmt.Println("Alice bought the plants") return // Success, exit loop. } // Wait a bit before trying again. time.Sleep(time.Millisecond) // It's also a good idea to limit the number of retries // (not shown here for simplicity) - you don't want // the program to get stuck in an infinite loop. } }()
This approach helps the program handle occasional goroutine conflicts over a shared resource.
Compare-and-set is often used in concurrent programming. It comes in different flavors, such as:
// CompareAndSet changes the value to new if the current value equals old. // Returns true if the value was changed. CompareAndSet(old, new any) bool // CompareAndSwap changes the value to new if the current value equals old. // Returns the old value. CompareAndSwap(old, new any) any // CompareAndDelete deletes the value if the current value equals old. // Returns true if the value was deleted. CompareAndDelete(old any) bool // etc
The idea is always the same:
- Check if the assumed (old) state matches reality.
- If it does, change the state to new.
- If not, do nothing.
Go's standard library provides compare-and-swap operations for basic types like
bool
andint64
. We'll cover these in another chapter.If compare-and-set doesn't work for your situation, you can always use a regular shared mutex instead. It can protect any sequence of operations, no matter how complex.
✎ Exercise: Concurrent map + 1 more
Practice is crucial in turning abstract knowledge into skills, making theory alone insufficient. The full version of the book contains a lot of exercises — that's why I recommend getting it.
If you are okay with just theory for now, let's continue.
Idempotence and atomicity
Idempotence means that calling an operation on an object multiple times doesn't cause any changes or errors after the first time.
Let's say we have a resource that need to be released after use:
// resource that needs to be freed after use. type resource struct { // ... } // free releases the resource. // Panics if the resource has already been released. func (r *resource) free() { // ... }
And there's a worker that frees up resources when closed:
// Worker does some work and // frees resources when closed. type Worker struct { res resource } // Work performs some work. func (w *Worker) Work() { // ... } // Close frees resources // and shuts down the worker. func (w *Worker) Close() { w.res.free() }
Everything works fine until we call
Close
twice:func main() { w := new(Worker) w.Work() w.Close() w.Close() fmt.Println("worker closed") } panic: resource is already freed
Let's see how to make
Close
idempotent so we can safely release the resources.Boolean flag
Let's add a
closed
flag to the worker and check it in theClose
method:// Worker does some work and // frees resources when closed. type Worker struct { res resource closed bool } // Close frees resources // and shuts down the worker. func (w *Worker) Close() { // Ignore repeated calls. if w.closed { // (1) return } w.res.free() // (2) w.closed = true // (3) }
Now, calling
Close
multiple times works fine:func main() { w := new(Worker) w.Close() w.Close() fmt.Println("worker closed") } worker closed
But what happens if we call
Close
simultaneously from different goroutines?func main() { var wg sync.WaitGroup wg.Add(2) w := new(Worker) go func() { defer wg.Done() w.Close() }() go func() { defer wg.Done() w.Close() }() wg.Wait() fmt.Println("worker closed") } // panic: resource is already freed panic: resource is already freed
You won't see panic here often. But it's still there, lurking in the dark.
Panic! You already know why this happens — the check for
w.closed
➊ and the following resource cleanup ➋ ➌ aren't atomic. Since goroutines run concurrently, they both pass the if statement and callw.res.free()
. Then, one of the goroutines panics.We need a structure that ensures atomicity and idempotence in a concurrent environment.
select
Why don't we use a
closed
channel instead of a boolean flag, and useselect
to close the channel only once?// Worker does some work and // frees resources when closed. type Worker struct { res resource closed chan struct{} } // NewWorker creates a new worker. // We need an explicit NewWorker constructor instead of // new(Worker), because the default value for a channel // is nil, which doesn't work for our needs. func NewWorker() *Worker { return &Worker{closed: make(chan struct{})} } // Close frees resources // and shuts down the worker. func (w *Worker) Close() { select { case <-w.closed: // Ignore repeated calls. return default: w.res.free() close(w.closed) } } worker closed
We call
Close
from two goroutines and get "worker closed". Everything works fine. Then we deploy this code to production, and a week later we get a bug report saying the app sometimes crashes with the "resource is already freed" panic. What's wrong?The thing is, select does not protect against freeing resources more than once. As we know, select is safe for concurrent use. So, if two goroutines call
Close
at the same time, they both enter the select and have to pick a case. Since theclosed
channel isn't closed yet, both goroutines choose the default case. Both callw.res.free()
, and one of them panics.The chances of this happening are pretty low. You could call
Close
999 times without any issues, but on the thousandth try, the stars will align just right, both goroutines will hit the default case, and you will get a panic.It's especially frustrating that the race detector doesn't always catch this kind of issues. In the example above, the race detector might not find anything (depends on the Go version), especially if you comment out
w.res.free()
. But the race condition is still there, and closing the channel more than once will eventually cause a panic.Select is not atomic. Choosing a select case and running its body are separate actions, not a single atomic operation. So, if the code inside the case changes shared data, select can cause a race condition. It's important to keep this in mind.
Mutex
Let's go back to the
closed
boolean flag, but this time protect it with a mutex:// Worker does some work and // frees resources when closed. type Worker struct { res resource mu sync.Mutex closed bool } // Close frees resources // and shuts down the worker. func (w *Worker) Close() { // Make the method atomic. w.mu.Lock() defer w.mu.Unlock() // Ignore repeated calls. if w.closed { return } w.res.free() w.closed = true } worker closed
The mutex stays locked for the entire scope of the
Close
method. This ensures that no matter how many goroutines callClose
simultaneously, only one can execute its body at a time.Checking the
closed
flag, freeing resources, and updating theclosed
state all happen as one atomic operation. This makesClose
idempotent, so you won't get any panics when releasing resources.A mutex (or something similar) is the only way to make sure a complex operation is atomic in a concurrent environment. Do not rely on select in these situations.
Speaking of "something similar", for this specific use case — making sure something happens exactly once — Go's standard library provides a handy
sync.Once
type. We'll cover it in another chapter.✎ Exercise: Spot the race
Practice is crucial in turning abstract knowledge into skills, making theory alone insufficient. The full version of the book contains a lot of exercises — that's why I recommend getting it.
If you are okay with just theory for now, let's continue.
Locker
Here are the mutex methods we learned about in the last chapter:
type Mutex struct { // internal state } func (m *Mutex) Lock() func (m *Mutex) Unlock() type RWMutex struct { // internal state } func (rw *RWMutex) Lock() func (rw *RWMutex) Unlock() func (rw *RWMutex) RLock() func (rw *RWMutex) RUnlock()
As you can see, both types have the same
Lock
andUnlock
methods. Go's standard library provides a common interface for them:// A Locker represents an object that // can be locked and unlocked. type Locker interface { Lock() Unlock() }
If the "locking mechanism" is defined by the client, and your code just needs to lock or unlock access to shared data, use
Locker
instead of a specific type:// ArrayList is a concurrent-safe dynamic array. type ArrayList struct { vals []any lock sync.Locker } // NewArrayList creates a new empty array. func NewArrayList(lock sync.Locker) *ArrayList { if lock == nil { panic("NewArrayList: lock cannot be nil") } return &ArrayList{vals: []any{}, lock: lock} } // Len returns the length of the array. func (al *ArrayList) Len() int { al.lock.Lock() defer al.lock.Unlock() return len(al.vals) } // Append adds an element to the array. func (al *ArrayList) Append(val any) { al.lock.Lock() defer al.lock.Unlock() al.vals = append(al.vals, val) }
This way, the client can use
Mutex
,RWMutex
, or any other implementation they prefer:func main() { var wg sync.WaitGroup var lock sync.Mutex list := NewArrayList(&lock) // Add 400 elements to the array using 4 goroutines. for i := 0; i < 4; i++ { wg.Add(1) go func() { defer wg.Done() for range 100 { list.Append(rand.IntN(100)) time.Sleep(time.Millisecond) } }() } wg.Wait() fmt.Println("list length =", list.Len()) } list length = 400
By using
sync.Locker
, you can build components that don't depend on a specific lock implementation. This lets the client decide which lock to use. While you probably won't need this very often, it's a useful feature to be aware of.TryLock
Let's say our program needs to call a legacy system, represented by the
External
type. This system is so ancient that it can handle no more than one call at a time. That's why we protect it with a mutex:// External is a client for an external system. type External struct { lock sync.Mutex } // Call calls the external system. func (e *External) Call() { e.lock.Lock() defer e.lock.Unlock() // Simulate a remote call. time.Sleep(100 * time.Millisecond) }
Now, no matter how many goroutines try to access the external system at the same time, they'll have to take turns:
func main() { var wg sync.WaitGroup ex := new(External) start := time.Now() const nCalls = 4 for range nCalls { wg.Add(1) go func() { defer wg.Done() ex.Call() fmt.Println("success") }() } wg.Wait() fmt.Printf( "%d calls took %d ms\n", nCalls, time.Since(start).Milliseconds(), ) } success success success success 4 calls took 400 ms
Everything looks good on paper. But in reality, if there are a lot of these goroutines, the external system will be constantly busy handling all these sequential calls. This could end up being too much for it. So, let's change the approach:
- If a goroutine needs to call an external system,
- and that system is already busy,
- then the goroutine shouldn't wait,
- but should immediately return an error.
We can use the
TryLock
method of a mutex to implement this logic:// External is a client for an external system. type External struct { lock sync.Mutex } // Call calls the external system. func (e *External) Call() error { if !e.lock.TryLock() { return errors.New("busy") // (1) } defer e.lock.Unlock() // Simulate a remote call. time.Sleep(100 * time.Millisecond) return nil }
TryLock
tries to lock the mutex, just like a regularLock
. But if it can't, it returnsfalse
right away instead of blocking the goroutine. This way, we can immediately return an error at ➊ instead of waiting for the system to become available.Now, out of four simultaneous calls, only one will go through. The others will get a "busy" error:
func main() { var wg sync.WaitGroup ex := new(External) start := time.Now() const nCalls = 4 for range nCalls { wg.Add(1) go func() { defer wg.Done() err := ex.Call() if err != nil { fmt.Println(err) } else { fmt.Println("success") } }() } wg.Wait() fmt.Printf( "%d calls took %d ms\n", nCalls, time.Since(start).Milliseconds(), ) } busy busy busy success 4 calls took 100 ms
According to the standard library docs,
TryLock
is rarely needed. In fact, using it might mean there's a problem with your program's design. For example, if you're callingTryLock
in a busy-wait loop ("keep trying until the resource is free") — that's usually a bad sign:for { if mutex.TryLock() { // Use the shared resource. mutex.Unlock() break } }
This code will keep one CPU core at 100% usage until the mutex is unlocked. It's much better to use a regular
Lock
so the scheduler can take the blocked goroutine off the CPU.✎ Exercise: Rate limiter
Practice is crucial in turning abstract knowledge into skills, making theory alone insufficient. The full version of the book contains a lot of exercises — that's why I recommend getting it.
If you are okay with just theory for now, let's continue.
Shared nothing
Let's go back one last time to Alice and the Lego sets we started the chapter with.
We manage user accounts:
// Accounts - money in users' accounts. type Accounts struct { bal map[string]int mu sync.Mutex } // NewAccounts creates a new set of accounts. func NewAccounts(bal map[string]int) *Accounts { return &Accounts{bal: maps.Clone(bal)} } // Get returns the user's balance. func (a *Accounts) Get(name string) int { a.mu.Lock() // (1) defer a.mu.Unlock() return a.bal[name] } // Set changes the user's balance. func (a *Accounts) Set(name string, amount int) { a.mu.Lock() // (2) defer a.mu.Unlock() a.bal[name] = amount }
And handle purchases:
acc := NewAccounts(map[string]int{ "alice": 50, }) castle := LegoSet{name: "Castle", price: 40} plants := LegoSet{name: "Plants", price: 20} // Shared mutex. var mu sync.Mutex // Alice buys a castle. go func() { defer wg.Done() // Protect the entire purchase with a mutex. mu.Lock() // (3) defer mu.Unlock() // Check and update the balance. }() // Alice buys plants. go func() { defer wg.Done() // Protect the entire purchase with a mutex. mu.Lock() // (4) defer mu.Unlock() // Check and update the balance. }()
This isn't a very complex use case — I'm sure you've seen worse. Still, we had to put in some effort:
- Protect the balance with a mutex to prevent a data race ➊ ➋.
- Protect the entire purchase operation with a mutex (or use compare-and-set) to make sure the final state is correct ➌ ➍.
We were lucky to notice and prevent the race condition during a purchase. What if we had missed it?
There's another approach to achieving safe concurrency: instead of protecting shared state when working with multiple goroutines, we can avoid shared state altogether. Channels can help us do this.
Here's the idea: we'll create a
Processor
function that accepts purchase requests through an input channel, processes them, and sends the results back through an output channel:// A purchase request. type Request struct { buyer string set LegoSet } // A purchase result. type Purchase struct { buyer string set LegoSet succeed bool balance int // balance after purchase } // Processor handles purchases. func Processor(acc map[string]int) (chan<- Request, <-chan Purchase) { // ... }
Buyer goroutines will send requests to the processor's input channel and receive results (successful or failed purchases) from the output channel:
func main() { const buyer = "Alice" acc := map[string]int{buyer: 50} wishlist := []LegoSet{ {name: "Castle", price: 40}, {name: "Plants", price: 20}, } reqs, purs := Processor(acc) // Alice buys stuff. var wg sync.WaitGroup for _, set := range wishlist { wg.Add(1) go func() { defer wg.Done() reqs <- Request{buyer: buyer, set: set} pur := <-purs if pur.succeed { fmt.Printf("%s bought the %s\n", pur.buyer, pur.set.name) fmt.Printf("%s's balance: %d\n", buyer, pur.balance) } }() } wg.Wait() } Alice bought the Plants Alice's balance: 30
This approach offers several benefits:
- Buyer goroutines send their requests and get results without worrying about how the purchase is done.
- All the buying logic is handled inside the processor goroutine.
- No need for mutexes.
All that's left is to implement the processor. How about this:
// Processor handles purchases. func Processor(acc map[string]int) (chan<- Request, <-chan Purchase) { in := make(chan Request) out := make(chan Purchase) acc = maps.Clone(acc) go func() { for { // Receive the purchase request. req := <-in // Handle the purchase. balance := acc[req.buyer] pur := Purchase{buyer: req.buyer, set: req.set, balance: balance} if balance >= req.set.price { pur.balance -= req.set.price pur.succeed = true acc[req.buyer] = pur.balance } else { pur.succeed = false } // Send the result. out <- pur } }() return in, out }
It would have been a good idea to add a way to stop the processor using context, but I decided not to do it to keep the code simple.
The processor clones the original account states and works with its own copy. This approach makes sure there is no concurrent access to the accounts, so there are no races. Of course, we should avoid running two processors at the same time, or we could end up with two different versions of the truth.
It's not always easy to structure a program in a way that avoids shared state. But if you can, it's a good option.
Keep it up
Now you know how to protect shared data (from data races) and sequences of operations (from race conditions) in a concurrent environment using mutexes. Be careful with them and always test your code thoroughly with the race detector enabled.
Use code reviews, because the race detector doesn't catch every data race and can't detect race conditions at all. Having someone else look over your code can be really helpful.
In the next chapter, we'll talk about semaphores (coming soon).
-