Blog on Hugo van Kemenade

A CLI to fight GitHub spam

Wed, 18 Feb 2026 13:35:25 +0000

`gh triage spam` #

We get a lot of spam in the CPython project .

A lot of it isn’t even slop, but mostly worthless “nothing” issues and PRs that barely fill in the issue template, or add a line of nonsense to some arbitrary file.

They’re often from new accounts with usernames like:

za9066559-wq
quanghuynh10111-png
riffocristobal579-cmd
sajjad5giot
satyamchoudhary1430-boop
SilaMey
standaell1234-maker
eedamhmd2005-ui
ksdmyanmar-lighter
experments-studios
madurangarathanayaka5-art

A new issue from a username following the pattern nameNNNN-short_suffix is a dead giveaway. I think they’re trying to farm “realistic” accounts: open a PR, open an issue, comment on something, make a fake review.

It’s easy but tedious to:

close the PR/issue as not planned
retitle to “spam”
apply the “invalid” label
remove other labels

I use the GitHub CLI gh a lot (for example, gh co NNN to check out a PR locally), and it’s straightforward to write your own Python-based extensions, so I wrote gh triage.

Install:

$ gh extension install hugovk/gh-triage
Cloning into '/Users/hugo/.local/share/gh/extensions/gh-triage'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 15 (delta 5), reused 12 (delta 2), pack-reused 0 (from 0)
Receiving objects: 100% (15/15), 5.09 KiB | 5.09 MiB/s, done.
Resolving deltas: 100% (5/5), done.
✓ Installed extension hugovk/gh-triage

Then run like gh triage spam <issue-or-pr-number-or-url>:

$ gh triage spam https://github.com/python/cpython/issues/144900
✅ Removed labels: type-bug
✅ Added labels: invalid
✅ Changed title: spam
✅ Closed

This can be used for any repo that you have permissions for: it applies the “invalid” or “spam” labels, but only if they exist in the repo.

Next step: perhaps it could print out the URL to make it easy to report the account to GitHub (usually for “Spam or inauthentic Activity”).

`gh triage unassign` #

Not spam, but another triage helper.

A less common occurrence is a rebase or merge from main or change of PR base branch that ends up bringing in lots of code changes. This often assigns the PR to dozens of people via CODEOWNERS, for example: python/cpython#142564.

Everyone’s already been pinged and subscribed to the PR, so it’s too late to help that, but we can automate unassigning them all so at least the PR is not in their “assigned to” list.

Run gh triage unassign <issue-or-pr-number-or-url> to:

remove all assignees (issues and PRs)
remove all requested reviewers (PRs only)

For example:

gh triage unassign 142564

Speeding up Pillow's open and save

Wed, 28 Jan 2026 10:29:04 +0000

Tachyon #

I tried out Tachyon, the new “high-frequency statistical sampling profiler” coming in Python 3.15, to see if we can speed up the Pillow imaging library. I started with a simple script to open an image:

import sys
from PIL import Image

im = Image.open(f"Tests/images/hopper.{sys.argv[1]}")

Then ran:

$ python3.15 -m profiling.sampling run --flamegraph /tmp/1.py png
Captured 35 samples in 0.04 seconds
Sample rate: 1,000.00 samples/sec
Error rate: 25.71
Flamegraph data: 1 root functions, total samples: 26, 169 unique strings
Flamegraph saved to: flamegraph_97927.html

Which generates this flame graph:

The whole thing took 40 milliseconds, with half in Image.py’s open(). If you visit the interactive HTML page we can see open() calls preinit(), which in turn imports GifImagePlugin, BmpImagePlugin, PngImagePlugin and JpegImagePlugin (hover over the <module> boxes to see them).

Do we really need to import all those plugins when we’re only interested in PNG?

Okay, let’s try another kind of image:

$ python3.15 -m profiling.sampling run --flamegraph /tmp/1.py webp
Captured 59 samples in 0.06 seconds
Sample rate: 1,000.00 samples/sec
Error rate: 22.03
Flamegraph data: 1 root functions, total samples: 46, 256 unique strings
Flamegraph saved to: flamegraph_98028.html

Hmm, 60 milliseconds with 80% in open() and most of that in init(). The HTML page shows it imports AvifImagePlugin, PdfImagePlugin, WebpImagePlugin, DcxImagePlugin, DdsImagePlugin and PalmImagePlugin. We also have preinit importing GifImagePlugin, BmpImagePlugin and PngImagePlugin.

Again, why import even more plugins when we only care about WebP?

Loading all the plugins? #

That’s enough profiling, let’s look at the code.

When open()ing or save()ing an image, if Pillow isn’t yet initialised, we call a preinit() function. This loads five drivers for five formats by importing their plugins: BMP, GIF, JPEG, PPM and PNG.

During import, each plugin registers its file extensions, MIME types and some methods used for opening and saving.

Then we check each of these plugins in turn to see if one will accept the image. Most of Pillow’s plugins detect an image by opening the file and checking if the first few bytes match a magic prefix. For example:

GIF starts with b"GIF87a" or b"GIF89a".
PNG starts with b"\211PNG\r\n\032\n" (reference).
JPEG starts with b"\xff\xd8\xff", where \xff\xd8 means “Start of Image”, and \xff is the start of the next marker (reference).

If none of these five match, we call init(), which imports the remaining 42 plugins. We then check each of these for a match.

This has been the case since at least PIL 1.1.1 released in 2000 (this is the oldest version I have to check). There were 33 builtin plugins then and 47 now.

Lazy loading #

This is all a bit wasteful if we only need one or two image formats during a program’s lifetime, especially for things like CLIs. Longer running programs may need a few more, but unlikely all 47.

A benefit of the plugin system is third parties can create their own plugins, but we can be more efficient with our builtins.

I opened a PR to add a mapping of file extensions to plugins. Before calling preinit() or init(), we can instead do a cheap lookup, which may save us importing, registering, and checking all those plugins.

Of course, we may have an image without an extension, or with the “wrong” extension, but that’s fine; I expect it’s rare and anyway we’ll fall back to the original preinit() -> init() flow.

After merging the PR, here’s a new flame graph for opening PNG (HTML page):

And for WebP (HTML page):

The flame graphs are scaled to the same width, but there’s far fewer boxes meaning there’s much less work now. We’re down from 40 and 60 milliseconds to 20 and 20 milliseconds.

The PR has a bunch of benchmarks which show opening a PNG (that previously loaded five plugins) is now 2.6 times faster. Opening a WebP (that previously loaded all 47 plugins), is now 14 times faster. Similarly, Saving PNG is improved by 2.2 times and WebP by 7.9 times. Success! This will be in Pillow 12.2.0.

Localising xkcd

Sat, 03 Jan 2026 14:06:51 +0000

I gave a lightning talk at a bunch of conferences in 2025 about some of the exciting new things coming in Python 3.14, including template strings.

One thing we can use t-strings for is to prevent SQL injection. The user gives you an untrusted t-string, and you can sanitise it, before using it in a safer way.

I illustrated this with xkcd 327, titled “Exploits of a Mom”, but commonly known as “Little Bobby Tables”.

I localised most of the slides for the PyCon I was at, including this comic. Here they are!

PyCon Italia #

May, Bologna

PyCon Greece #

August, Athens

PyCon Estonia #

October, Tallinn

PyCon Finland #

October, Jyväskylä

PyCon Sweden #

October, Stockholm

Thanks #

Thanks to Randall Munroe for licensing the comic under the Creative Commons Attribution-NonCommercial 2.5 License. These adaptations are therefore licensed the same way.

Finally, here’s links for 2026, I recommend them all:

PyCon Italia, 27-30 May: the CFP is open until 6th January
PyCon Estonia, 8-9 October
PyCon Greece, 12-13 October
PyCon Sweden, TBA
PyCon Finland, TBA

Replacing python-dateutil to remove six

Mon, 29 Dec 2025 16:53:00 +0000

The dateutil library is a popular and powerful Python library for dealing with dates and times.

However, it still supports Python 2.7 by depending on the six compatibility shim, and I’d prefer not to install for Python 3.10 and higher.

Here’s how I replaced three uses of its relativedelta in a couple of CLIs that didn’t really need to use it.

One #

norwegianblue was using it to calculate six months from now:

import datetime as dt

from dateutil.relativedelta import relativedelta

now = dt.datetime.now(dt.timezone.utc)
# datetime.datetime(2025, 12, 29, 15, 59, 44, 518240, tzinfo=datetime.timezone.utc)
six_months_from_now = now + relativedelta(months=+6)
# datetime.datetime(2026, 6, 29, 15, 59, 44, 518240, tzinfo=datetime.timezone.utc)

But we don’t need to be so precise here, and 180 days is good enough, using the standard library’s datetime.timedelta:

import datetime as dt

now = dt.datetime.now(dt.timezone.utc)
# datetime.datetime(2025, 12, 29, 15, 59, 44, 518240, tzinfo=datetime.timezone.utc)
six_months_from_now = now + dt.timedelta(days=180)
# datetime.datetime(2026, 6, 27, 15, 59, 44, 518240, tzinfo=datetime.timezone.utc)

Two #

pypistats was using it get the last day of a month:

import datetime as dt

first = dt.date(year, month, 1)
# datetime.date(2025, 12, 1)
last = first + relativedelta(months=1) - relativedelta(days=1)
# datetime.date(2025, 12, 31)

Instead, we can use the stdlib’s calendar.monthrange:

import calendar
import datetime as dt

last_day = calendar.monthrange(year, month)[1]
# 31
last = dt.date(year, month, last_day)
# datetime.date(2025, 12, 31)

Three #

Finally, to get last month as a yyyy-mm string:

import datetime as dt

from dateutil.relativedelta import relativedelta

today = dt.date.today()
# datetime.date(2025, 12, 29)
d = today - relativedelta(months=1)
# datetime.date(2025, 11, 29)
d.isoformat()[:7]
# '2025-11'

Instead:

import datetime as dt

today = dt.date.today()
# datetime.date(2025, 12, 29)
if today.month == 1:
 year, month = today.year - 1, 12
else:
 year, month = today.year, today.month - 1
 # 2025, 11
f"{year}-{month:02d}"
# '2025-11'

Goodbye six, and we also get slightly quicker install, import and run times.

Bonus #

I recommend Adam Johnson’s tip to import datetime as dt to avoid the ambiguity of which datetime is the module and which is the class.

Header photo: Ver Sacrum calendar by Alfred Roller

And now for something completely different

Tue, 23 Dec 2025 13:03:33 +0000

Starting in 2019, Python 3.8 and 3.9 release manager Łukasz Langa added a new section to the release notes called “And now for something completely different” with a sketch transcript from Monty Python.

For Python 3.10 and 3.11, the next release manager Pablo Galindo Salgado continued the section but included astrophysics facts.

For Python 3.12, the next RM Thomas Wouters shared poems (and took a break for 3.13).

And for Python 3.14, I’m doing all things π, pie and [mag]pie.

Here’s a collection of my different things for the first year (and a bit) of Python 3.14.

alpha 1 #

2024-10-15

π (or pi) is a mathematical constant, approximately 3.14, for the ratio of a circle’s circumference to its diameter. It is an irrational number, which means it cannot be written as a simple fraction of two integers. When written as a decimal, its digits go on forever without ever repeating a pattern.

Here’s 76 digits of π:

3.141592653589793238462643383279502884197169399375105820974944592307816406286

Piphilology is the creation of mnemonics to help remember digits of π.

In a pi-poem, or “piem”, the number of letters in a word equal the corresponding digit. This covers 9 digits, 3.14159265:

How I wish I could recollect pi easily today!

One of the most well-known covers 15 digits, 3.14159265358979:

How I want a drink, alcoholic of course, after the heavy chapters involving quantum mechanics!

Here’s a 35-word piem in the shape of a circle, 3.1415926535897932384626433832795728:

It’s a fact A ratio immutable Of circle round and width, Produces geometry’s deepest conundrum. For as the numerals stay random, No repeat lets out its presence, Yet it forever stretches forth. Nothing to eternity.

The Guinness World Record for memorising the most digits is held by Rajveer Meena, who recited 70,000 digits blindfold in 2015. The unofficial record is held by Akira Haraguchi who recited 100,000 digits in 2006.

alpha 2 #

2024-11-19

Ludolph van Ceulen (1540-1610) was a fencing and mathematics teacher in Leiden, Netherlands, and spent around 25 years calculating π (or pi), using essentially the same methods Archimedes employed some seventeen hundred years earlier.

Archimedes estimated π by calculating the circumferences of polygons that fit just inside and outside of a circle, reasoning the circumference of the circle lies between these two values. Archimedes went up to polygons with 96 sides, for a value between 3.1408 and 3.1428, which is accurate to two decimal places.

Van Ceulen used a polygon with half a billion sides. He published a 20-decimal value in his 1596 book Vanden Circkel (“On the Circle”), and later expanded it to 35 decimals:

3.14159265358979323846264338327950288

Van Ceulen’s 20 digits is more than enough precision for any conceivable practical purpose. For example, even if a printed circle was perfect down to the atomic scale, the thermal vibrations of the molecules of ink would make most of those digits physically meaningless. NASA Jet Propulsion Laboratory’s highest accuracy calculations, for interplanetary navigation, uses 15 decimals: 3.141592653589793.

At Van Ceulen’s request, his upper and lower bounds for π were engraved on his tombstone in Leiden. The tombstone was eventually lost but restored in 2000. In the Netherlands and Germany, π is sometimes referred to as the “Ludolphine number”, after Van Ceulen.

alpha 3 #

2024-12-17

A mince pie is a small, round covered tart filled with “mincemeat”, usually eaten during the Christmas season – the UK consumes some 800 million each Christmas. Mincemeat is a mixture of things like apple, dried fruits, candied peel and spices, and originally would have contained meat chopped small, but rarely nowadays. They are often served warm with brandy butter.

According to the Oxford English Dictionary, the earliest mention of Christmas mince pies is by Thomas Dekker, writing in the aftermath of the 1603 London plague, in Newes from Graues-end: Sent to Nobody (1604):

Ten thousand in London swore to feast their neighbors with nothing but plum-porredge, and mince-pyes all Christmas.

Here’s a meaty recipe from Rare and Excellent Receipts, Experienc’d and Taught by Mrs Mary Tillinghast and now Printed for the Use of her Scholars Only (1678):

XV. How to make Mince-pies.

To every pound of Meat, take two pound of beef Suet, a pound of Corrants, and a quarter of an Ounce of Cinnamon, one Nutmeg, a little beaten Mace, some beaten Colves, a little Sack & Rose-water, two large Pippins, some Orange and Lemon peel cut very thin, and shred very small, a few beaten Carraway-seeds, if you love them the Juyce of half a Lemon squez’d into this quantity of meat; for Sugar, sweeten it to your relish; then mix all these together and fill your Pie. The best meat for Pies is Neats-Tongues, or a leg of Veal; you may make them of a leg of Mutton if you please; the meat must be parboyl’d if you do not spend it presently; but if it be for present use, you may do it raw, and the Pies will be the better.

alpha 4 #

2025-01-14

In Python, you can use Greek letters as constants. For example:

from math import pi as π

def circumference(radius: float) -> float:
 return 2 * π * radius

print(circumference(6378.137)) # 40075.016685578485

alpha 5 #

2025-02-11

2025-01-29 marked the start of a new lunar year, the Year of the Snake :snake: (and the Year of Python?).

For centuries, π was often approximated as 3 in China. Some time between the years 1 and 5 CE, astronomer, librarian, mathematician and politician Liu Xin (劉歆) calculated π as 3.154.

Around 130 CE, mathematician, astronomer, and geographer Zhang Heng (張衡, 78–139) compared the celestial circle with the diameter of the earth as 736:232 to get 3.1724. He also came up with a formula for the ratio between a cube and inscribed sphere as 8:5, implying the ratio of a square’s area to an inscribed circle is √8:√5. From this, he calculated π as √10 (~3.162).

Third century mathematician Liu Hui (刘徽) came up with an algorithm for calculating π iteratively: calculate the area of a polygon inscribed in a circle, then as the number of sides of the polygon is increased, the area becomes closer to that of the circle, from which you can approximate π.

This algorithm is similar to the method used by Archimedes in the 3rd century BCE and Ludolph van Ceulen in the 16th century CE (see 3.14.0a2 release notes), but Archimedes only went up to a 96-sided polygon (96-gon). Liu Hui went up to a 192-gon to approximate π as 157/50 (3.14) and later a 3072-gon for 3.14159.

Liu Hu wrote a commentary on the book The Nine Chapters on the Mathematical Art which included his π approximations.

In the fifth century, astronomer, inventor, mathematician, politician, and writer Zu Chongzhi (祖沖之, 429–500) used Liu Hui’s algorithm to inscribe a 12,288-gon to compute π between 3.1415926 and 3.1415927, correct to seven decimal places. This was more accurate than Hellenistic calculations and wouldn’t be improved upon for 900 years.

Happy Year of the Snake!

alpha 6 #

2025-03-14

March 14 is celebrated as pi day, because 3.14 is an approximation of π. The day is observed by eating pies (savoury and/or sweet) and celebrating π. The first pi day was organised by physicist and tinkerer Larry Shaw of the San Francisco Exploratorium in 1988. It is also the International Day of Mathematics and Albert Einstein’s birthday. Let’s all eat some pie, recite some π, install and test some py, and wish a happy birthday to Albert, Loren and all the other pi day children!

alpha 7 #

2025-04-08

On Saturday, 5th April, 3.141592653589793 months of the year had elapsed.

beta 1 #

2025-05-07

The mathematical constant pi is represented by the Greek letter π and represents the ratio of a circle’s circumference to its diameter. The first person to use π as a symbol for this ratio was Welsh self-taught mathematician William Jones in 1706. He was a farmer’s son born in Llanfihangel Tre’r Beirdd on Angelsy (Ynys Môn) in 1675 and only received a basic education at a local charity school. However, the owner of his parents’ farm noticed his mathematical ability and arranged for him to move to London to work in a bank.

By age 20, he served at sea in the Royal Navy, teaching sailors mathematics and helping with the ship’s navigation. On return to London seven years later, he became a maths teacher in coffee houses and a private tutor. In 1706, Jones published Synopsis Palmariorum Matheseos which used the symbol π for the ratio of a circle’s circumference to diameter (hunt for it on pages 243 and 263 or here). Jones was also the first person to realise π is an irrational number, meaning it can be written as decimal number that goes on forever, but cannot be written as a fraction of two integers.

But why π? It’s thought Jones used the Greek letter π because it’s the first letter in perimetron or perimeter. Jones was the first to use π as our familiar ratio but wasn’t the first to use it in as part of the ratio. William Oughtred, in his 1631 Clavis Mathematicae (The Key of Mathematics), used π/δ to represent what we now call pi. His π was the circumference, not the ratio of circumference to diameter. James Gregory, in his 1668 Geometriae Pars Universalis (The Universal Part of Geometry) used π/ρ instead, where ρ is the radius, making the ratio 6.28… or τ. After Jones, Leonhard Euler had used π for 6.28…, and also p for 3.14…, before settling on and popularising π for the famous ratio.

beta 2 #

2025-05-26

In 1897, the State of Indiana almost passed a bill defining π as 3.2.

Of course, it’s not that simple.

Edwin J. Goodwin, M.D., claimed to have come up with a solution to an ancient geometrical problem called squaring the circle, first proposed in Greek mathematics. It involves trying to draw a circle and a square with the same area, using only a compass and a straight edge. It turns out to be impossible because π is transcendental (and this had been proved just 13 years earlier by Ferdinand von Lindemann), but Goodwin fudged things so the value of π was 3.2 (his writings have included at least nine different values of π: including 4, 3.236, 3.232, 3.2325… and even 9.2376…).

Goodwin had copyrighted his proof and offered it to the State of Indiana to use in their educational textbooks without paying royalties, provided they endorsed it. And so Indiana Bill No. 246 was introduced to the House on 18th January 1897. It was not understood and initially referred to the House Committee on Canals, also called the Committee on Swamp Lands. They then referred it to the Committee on Education, who duly recommended on 2nd February that “said bill do pass”. It passed its second reading on the 5th and the education chair moved that they suspend the constitutional rule that required bills to be read on three separate days. This passed 72-0, and the bill itself passed 67-0.

The bill was referred to the Senate on 10th February, had its first reading on the 11th, and was referred to the Committee on Temperance, whose chair on the 12th recommended “that said bill do pass”.

A mathematics professor, Clarence Abiathar Waldo, happened to be in the State Capitol on the day the House passed the bill and walked in during the debate to hear an ex-teacher argue:

The case is perfectly simple. If we pass this bill which establishes a new and correct value for pi , the author offers to our state without cost the use of his discovery and its free publication in our school text books, while everyone else must pay him a royalty.

Waldo ensured the senators were “properly coached”; and on the 12th, during the second reading, after an unsuccessful attempt to amend the bill it was postponed indefinitely. But not before the senators had some fun.

The Indiana News reported on the 13th:

…the bill was brought up and made fun of. The Senators made bad puns about it, ridiculed it and laughed over it. The fun lasted half an hour. Senator Hubbell said that it was not meet for the Senate, which was costing the State $250 a day, to waste its time in such frivolity. He said that in reading the leading newspapers of Chicago and the East, he found that the Indiana State Legislature had laid itself open to ridicule by the action already taken on the bill. He thought consideration of such a proposition was not dignified or worthy of the Senate. He moved the indefinite postponement of the bill, and the motion carried.

beta 3 #

2025-06-17

If you’re heading out to sea, remember the Maritime Approximation:

π mph = e knots

beta 4 #

2025-07-08

All this talk of π and yet some say π is wrong. Tau Day (June 28th, 6/28 in the US) celebrates τ as the “true circle constant”, as the ratio of a circle’s circumference to its radius, C/r = 6.283185… The Tau Manifesto declares π “a confusing and unnatural choice for the circle constant”, in part because “2π occurs with astonishing frequency throughout mathematics”.

If you wish to embrace τ the good news is PEP 628 added math.tau to Python 3.6 in 2016:

When working with radians, it is trivial to convert any given fraction of a circle to a value in radians in terms of tau. A quarter circle is tau/4, a half circle is tau/2, seven 25ths is 7*tau/25, etc. In contrast with the equivalent expressions in terms of pi (pi/2, pi, 14*pi/25), the unnecessary and needlessly confusing multiplication by two is gone.

release candidate 1 #

2025-07-22

Today, 22nd July, is Pi Approximation Day, because 22/7 is a common approximation of π and closer to π than 3.14.

22/7 is a Diophantine approximation, named after Diophantus of Alexandria (3rd century CE), which is a way of estimating a real number as a ratio of two integers. 22/7 has been known since antiquity; Archimedes (3rd century BCE) wrote the first known proof that 22/7 overestimates π by comparing 96-sided polygons to the circle it circumscribes.

Another approximation is 355/113. In Chinese mathematics, 22/7 and 355/113 are respectively known as Yuelü (约率; yuēlǜ; “approximate ratio”) and Milü (密率; mìlǜ; “close ratio”).

Happy Pi Approximation Day!

release candidate 2 #

2025-08-14

The magpie, Pica pica in Latin, is a black and white bird in the crow family, known for its chattering call.

The first-known use in English is from a 1589 poem, where magpie is spelled “magpy” and cuckoo is “cookow”:

Th[e]y fly to wood like breeding hauke, And leave old neighbours loue, They pearch themselves in syluane lodge, And soare in th’ aire aboue. There : magpy teacheth them to chat, And cookow soone doth hit them pat.

The name comes from Mag, short for Margery or Margaret (compare robin redbreast, jenny wren, and its corvid relative jackdaw); and pie, a magpie or other bird with black and white (or pied) plumage. The sea-pie (1552) is the oystercatcher, the grey pie (1678) and murdering pie (1688) is the great grey shrike. Others birds include the yellow and black pie, red-billed pie, wandering tree-pie, and river pie. The rain-pie, wood-pie and French pie are woodpeckers.

Pie on its own dates to before 1225, and comes from the Latin name for the bird, pica.

release candidate 3 #

2025-09-18

According to Pablo Galindo Salgado at PyCon Greece:

There are things that are supercool indeed, like for instance, this is one of the results that I’m more proud about. This equation over here, which you don’t need to understand, you don’t need to be scared about, but this equation here tells what is the maximum time that it takes for a ray of light to fall into a black hole. And as you can see the math is quite complicated but the answer is quite simple: it’s 2π times the mass of the black hole. So if you normalise by the mass of the black hole, the answer is 2π. And because there is nothing specific about your election of things in this formula, this formula is universal. It means it doesn’t depend on anything other than nature itself. Which means that you can use this as a definition of π. This is a valid alternative definition of the number π. It’s literally half the maximum time it takes to fall into a black hole, which is kind of crazy. So next time someone asks you what π means you can just drop this thing and impress them quite a lot. Maybe Hugo could use this information to put it into the release notes of πthon [yes, I can, thank you!].

3.14.0 (final) #

2025-10-07

Edgar Allen Poe died on 7th October 1849.

As we all recall from 3.14.0a1, piphilology is the creation of mnemonics to help memorise the digits of π, and the number of letters in each word in a pi-poem (or “piem”) successively correspond to the digits of π.

In 1995, Mike Keith, an American mathematician and author of constrained writing, retold Poe’s The Raven as a 740-word piem. Here’s the first two stanzas of Near A Raven:

Poe, E. Near a Raven

Midnights so dreary, tired and weary. Silently pondering volumes extolling all by-now obsolete lore. During my rather long nap - the weirdest tap! An ominous vibrating sound disturbing my chamber’s antedoor. “This”, I whispered quietly, “I ignore”.

Perfectly, the intellect remembers: the ghostly fires, a glittering ember. Inflamed by lightning’s outbursts, windows cast penumbras upon this floor. Sorrowful, as one mistreated, unhappy thoughts I heeded: That inimitable lesson in elegance - Lenore - Is delighting, exciting…nevermore.

3.14.1 #

2025-12-02

Seki Takakazu (関孝和; c. March 1642 – December 5, 1708) was a Japanese mathematician and samurai who laid the foundations of Japanese mathematics, later known as wasan (和算, from wa (“Japanese”) and san (“calculation”).

Seki was a contemporary of Isaac Newton and Gottfried Leibniz but worked independently. He created a new algebraic system, worked on infinitesimal calculus, and is credited with the discovery of Bernoulli numbers (before Bernoulli’s birth).

Seki also calculated π to 11 decimal places using a polygon with 131,072 sides inscribed within a circle, using an acceleration method now known as Aitken’s delta-squared process, which was rediscovered by Alexander Aitken in 1926.

Header photo: A scan of Seki Takakazu’s posthumous Katsuyō Sanpō (1712) showing calculations of π.

Steering Council results

Sat, 13 Dec 2025 14:40:12 +0000

The Python Steering Council 2026 election results are in and congratulations to the new Python Steering Council!

Welcome Savannah for the first time, and thank you to Greg Smith and Emily Morehouse for four and three years’ service each.

Three are starting their sixth terms, and four members have been or are release managers.

The chart above only covers the Steering Council years. Let’s also not forget Guido van Rossum’s BDFL years:

Steering Council election

Mon, 08 Dec 2025 19:50:58 +0000

🗳️ Update: See the results!

The Python Steering Council election is on!

This year six candidates are running for the five seats. See PEP 8107 for links to their nomination statements and PEP 13 for more on Python language governance.

I made a chart to show when the nominations arrived during the nomination period: six is fewer than in previous years, but they didn’t all wait until the last three days like in 2023 and 2024.

This year we’re also using a new voting system: with “Multi-winner Bloc STAR” you give between 0-6 stars per candidate.

If you’re in the core team, remember to vote by Friday!

Setting secrets in env vars

Thu, 27 Nov 2025 12:55:00 +0000

Sometimes you need to set environment variables with secrets, API keys or tokens, but they can be susceptible to exfiltration by malware, as seen during the recent Shai-Hulud attack.

For publishing to PyPI, it’s strongly recommended to use Trusted Publishing rather than managing long-lived tokens on your machine.

For other credentials, here’s how you can set them as env vars with 1Password CLI and direnv on macOS.

1Password #

The 1Password password manager has a command-line interface called op. Thanks to Simon Willison for his useful TIL post about it.

First install the CLI:

brew install 1password-cli

Then open 1Password > Settings > Developer > Integrate with 1Password CLI.

Then you can use the op command with your password item’s name in 1Password:

$ op item get "My secret item" --fields password
[use 'op item get xkm4wrtpvnq8hcjd3yfzs2belg --reveal' to reveal]

This obscures the actual password, and gives the item’s ID. You can use this ID:

$ op item get xkm4wrtpvnq8hcjd3yfzs2belg --fields password
[use 'op item get xkm4wrtpvnq8hcjd3yfzs2belg --reveal' to reveal]

Use --reveal to see the actual password:

$ op item get "My secret item" --fields password --reveal
my-secret-password

$ op item get xkm4wrtpvnq8hcjd3yfzs2belg --fields password --reveal
my-secret-password

Alternatively, use a secret reference. Open the item in 1Password, next to the password click ⌄ and select Copy Secret Reference then op read it:

$ op read "op://MyVault/xkm4wrtpvnq8hcjd3yfzs2belg/password"
my-secret-password

The secret reference is made up of the vault name and item ID, and op read doesn’t need --reveal.

Take your pick which one you prefer.

direnv #

direnv is a handy shell tool that can load and unload environment variables depending on your current directory.

Next, let’s set up direnv so that when you cd into a certain directory, it fetches the password from 1Password and sets it as an env var. When cding out, the env var is unloaded.

Install direnv:

brew install direnv

Then hook direnv into your shell. If you use Oh My Zsh, edit ~/.zshrc and add direnv to the plugins list:

plugins=(git gitfast gh you-should-use direnv)

Restart/reload your shell:

source ~/.zshrc

(I first got “Warning: direnv not found. Please install direnv and ensure it’s in your PATH before using this plugin” so needed to move eval "$(/opt/homebrew/bin/brew shellenv)" before source $ZSH/oh-my-zsh.sh.)

Now, in your project directory, create a file called .envrc containing an env var export that calls op:

# .envrc
export DEBUG=1
export MY_SECRET=$(op read "op://MyVault/xkm4wrtpvnq8hcjd3yfzs2belg/password")

On first run, you need to explicitly allow direnv to run for this directory. You also need to do this when editing .envrc later:

direnv allow .

If 1Password is set up with a passkey, it will now prompt to authorise:

$ cd my-project
direnv: loading ~/my-project/.envrc
direnv: export +DEBUG +MY_SECRET

Sometimes it shows a warning because of the 1Password UI prompt, but it’s okay:

$ cd my-project
direnv: loading ~/my-project/.envrc
direnv: ([/opt/homebrew/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
direnv: export +DEBUG +MY_SECRET

Then you can access the env var in your scripts:

$ echo ${DEBUG-nope}
1
$ echo ${MY_SECRET-nope}
my-secret-password

But when leaving the directory, you can no longer access it:

$ cd ..
direnv: unloading
$ echo ${DEBUG-nope}
nope
$ echo ${MY_SECRET-nope}
nope

Bonus #

1Password offers Team accounts to open-source projects as a way of giving back to the open source community.

Update #

2026-01-11:

See how Ned Batchelder explicitly sets and unsets env vars, and updates the prompt and window title to make it clear they’re active.
See how Miguel Grinberg uses process substitution to limit access to secrets from Bash scripts.

Header photo: Slaymaker Barry Co. Sprocket Locks in The Bearings (vol. XII no. 23, 2nd January 1896) in the Smithsonian Libraries and Archives, with no known copyright restrictions.

Python Core Sprint 2025

Mon, 03 Nov 2025 08:58:00 +0000

🐍🏃In September, the annual Python Core Sprint was hosted by Arm in Cambridge, UK!

The plan: put 35 core developers and 13 special guests in a room for a week, and see what they cook up.

Monday highlights #

We kicked off the first day with a round of five-word intros (mine: “three”, “dot”, “fourteen”, “release”, “manager”), lots of talks, and lots of discussion about talks:

Ken Jin Ooi - Building a JIT Community and Demo [effect] of new C API
Antonio Cuni - Tracing JITs on real code
Brett Cannon - WASI update and Precompiled binaries from python.org
Hood Chatham - Upstreaming Pyodide FFI
Russell Keith-Magee - Managing cross-platform wheel builds
Steering Council - PEP 793 and abi3/abi3t/abi4
Matthew Parkinson - Designing Deep Immutability

I did some ~~sprint~~ spring cleaning of our PyPI projects, dropping support for then-almost-EOL Python 3.9:

And because Mariatta wasn’t with us, here’s the all-important Python T-shirt census:

Menlo Park 2016 core sprint (Greg, Guido)
EuroPython 2023 (Antonio)
EuroPython 2024 volunteer (Hugo)
PyCon Canada 2016 (Brett)
PyCon US Pittsburgh Charlas (Thomas)
Cambridge 2025 core sprint (Diego)

The sprint room

Tuesday highlights #

Release day?

I’d originally planned to release Python 3.14.0rc3 on the Tuesday, but the morning was full of presentations, and the afternoon had an early departure for the social event, so I moved it to Wednesday instead.

Tania Allard gave a presentation about the different types of mentorship and how we can improve, followed by an open discussion.

Gregory P. Smith gave a demo on how we can use tools like Claude with CPython.

Tania, Jannis Leidel, Carol Willing and I discussed the User Success Workgroup and we came up with some ideas on next steps.

We ended the day with a punting tour on the river Cam and dinner at Jesus College, thank you, Arm!

And Thomas Wouters gave a fun session of his Feuding Pythonistas game (spoiler: people are wrong on the internet).

Python/tech t-shirt census:

Python 23.7, Menlo Park 20167 core sprint (Carl)
PyCon Italia 2025 (Antonio)
Python 3.14 pie (Hugo)
WHY 2025 (Thomas)
PyCon US 2025 (Diego)
PyCascades (Guido)
EuroPython 2022 (Mark)
Pythonista (Jacob)

Punting on the Cam

Wednesday highlights #

Release day?

The Steering Council asked for an extra day to decide about a possible typing revert (python/steering-council#307), so not today.

Lightning talks:

⚡ Guido van Rossum on collecting an oral history
⚡ Ee Durbin on modernising the last bits of infra and bots
⚡ Steve Dower demoed the new Windows installer
⚡ Larry Hastings on a linked_list date type
⚡ Adam Turner asked shall we close old issues
⚡ Greg shared a draft PEP for timestamps on async tracebacks

Carol, Adam, Thomas, Petr Viktorin and I discussed a number of docs topics.

I released the Python Docs Sphinx Theme with more translations.

We had a Q&A session with the Steering Council, three in-person and two joining remotely.

Jacob Coffee and I looked into upgrading the Python Insider and PSF blogs into something a little more modern.

T-shirt census:

PyCon Greece 2025 (Hugo)
PyCon Korea 2025 (Donghee)
PyLadies (Savannah, Petr)
EuroPython 2024 (Lys)
PyCon Italia 2023 (Antonio)
Cambridge 2025 core sprint (Guido)

The Python Steering Council

Thursday highlights #

Release day? Yes!

The Steering Council decided not to revert, so full steam ahead with the release.

Savannah Ostrowski, release manager for 3.16 and 3.17, shadowed to see what the process looks like (not as bad as it looks like in PEP 101).

Time for a couple of quick PRs and an interview with Pablo Galindo Salgado and Łukasz Langa on the core.py podcast, along with 29 others!

T-census:

PyCon UA 2016 (Łukasz)
Cambridge 2025 core sprint (Donghee)
Read the Docs Budapest 2014 (Petr)
Quansight Labs “Sustaining the future of Open Source” hoodie (Lys)

The 3.14.0rc3 release in progress

Friday highlights #

I went to Manchester to attend PyCon UK. Some highlights:

The conference also included sprints, and Adam and I ran the CPython sprint. We had a big table full of contributors and a few made their very first contributions, which is always rewarding for all involved!

PyCon UK sprint

Some numbers for me during the week:

Python release candidates released: 1
Issues created: 1
PRs created: 8
Issues closed: 7
PRs merged: 28
PRs closed: 1
Total issues involved with: 18
Total PRs involved with: 70
Repositories affected: 19

Thank you #

Huge thanks to Diego Russo and Arm for arranging and hosting us. The core sprint is always a highlight of the year and an incredibly productive week.

Read writeups by Diego and Antonio, and I recommend listening to Łukasz and Pablo’s core.py podcast for interviews with 18 (part one) and 12 sprinters (part two). They’re long, but it’s fascinating to hear all the different things everyone is working on.

Header photo by Arm

Three times faster with lazy imports

Sun, 19 Oct 2025 16:05:00 +0000

PEP 810 proposes “explicit lazy imports” for Python 3.15:

Lazy imports defer the loading and execution of a module until the first time the imported name is used, in contrast to ‘normal’ imports, which eagerly load and execute a module at the point of the import statement.

By allowing developers to mark individual imports as lazy with explicit syntax, Python programs can reduce startup time, memory usage, and unnecessary work. This is particularly beneficial for command-line tools, test suites, and applications with large dependency graphs.

It’s not been accepted yet, but let’s try out the reference implementation on one of my CLI tools, pypistats.

Setup #

First fetch the reference implementation. From a CPython checkout:

git remote add LazyImportsCabal https://github.com/LazyImportsCabal/cpython
git fetch LazyImportsCabal
gco lazy # see https://hugovk.dev/blog/2025/my-most-used-command-line-commands/

Because we want to install NumPy and pandas, let’s pretend to be Python 3.14 so we can use the binary wheels instead of having to build from source:

--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
 /* Version parsed out into numeric values */
 /*--start constants--*/
 #define PY_MAJOR_VERSION 3
-#define PY_MINOR_VERSION 15
+#define PY_MINOR_VERSION 14
 #define PY_MICRO_VERSION 0
 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA
 #define PY_RELEASE_SERIAL 0

 /* Version as a string */
-#define PY_VERSION "3.15.0a0"
+#define PY_VERSION "3.14.0a0"
 /*--end constants--*/

--- a/configure.ac
+++ b/configure.ac
-m4_define([PYTHON_VERSION], [3.15])
+m4_define([PYTHON_VERSION], [3.14])

Build non-debug CPython with optimisations:

GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \
 GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \
 ./configure --enable-optimizations --with-lto \
 --with-system-libmpdec --config-cache \
 --with-openssl="$(brew --prefix openssl@3)" && make -s -j8

Install NumPy and pandas:

./python.exe -m pip install numpy pandas

And then an editable install of the CLI, because we’ll also test changing the imports:

./python.exe -m pip install -e ~/github/pypistats

Let’s check the dependencies with pipdeptree:

uvx "pipdeptree[graphviz]" --python ./python.exe --packages pypistats --graph-output svg > pipdeptree.svg

pypistats has seven direct dependencies, which result in a total of 41 dependencies six layers deep, not counting NumPy and pandas:

Benchmarks #

Let’s benchmark running pypistats --help, which is meant to be quick, using hyperfine:

brew install hyperfine

Inline imports #

In the pypistats CLI, I had already applied the trick of moving heavier imports into the functions that call them (the PEP calls these “inline imports”).

Instead of the lazy keyword, I’m using the PYTHON_LAZY_IMPORTS env var here to make it easy to compare two different runs.

$ hyperfine --warmup 10 --runs 20 --export-json out.json \
 "./python.exe -m pypistats --help" \
 "PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help"
Benchmark 1: ./python.exe -m pypistats --help
 Time (mean ± σ): 46.2 ms ± 1.1 ms [User: 38.8 ms, System: 6.4 ms]
 Range (min … max): 45.1 ms … 49.6 ms 20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help
 Time (mean ± σ): 35.3 ms ± 0.5 ms [User: 29.5 ms, System: 4.8 ms]
 Range (min … max): 34.6 ms … 36.3 ms 20 runs

Summary
 PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help ran
 1.31 ± 0.04 times faster than ./python.exe -m pypistats --help

Plotted with plot_progression.py:

From 46 to 35 milliseconds, or, 1.31 times faster, not bad.

Fully lazy #

But we no longer need the inline imports trick with PEP 810!

I modified the CLI so all imports are at the top, and also removed if TYPE_CHECKING: guards. Here’s a diff.

$ hyperfine --warmup 10 --runs 20 --export-json out2.json \
 "./python.exe -m pypistats --help" \
 "PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help"
Benchmark 1: ./python.exe -m pypistats --help
 Time (mean ± σ): 104.1 ms ± 1.6 ms [User: 88.2 ms, System: 14.5 ms]
 Range (min … max): 101.9 ms … 109.5 ms 20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help
 Time (mean ± σ): 35.7 ms ± 0.5 ms [User: 29.8 ms, System: 4.8 ms]
 Range (min … max): 34.7 ms … 36.5 ms 20 runs

Summary
 PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help ran
 2.92 ± 0.06 times faster than ./python.exe -m pypistats --help

From 104 to 36 milliseconds, or 2.92 times faster, much better!

Header photo: “Lazy Man Fishing” at Cascade Locks on the Columbia River 05/1973 in the U.S. National Archives , with no known copyright restrictions.

Releasing Python 3.14.0

Sat, 11 Oct 2025 17:04:00 +0000

Prologue #

I livetooted the release of Python 3.14.0. Here it is in blogpost form!

One week #

Only one week left until the release of Python 3.14.0 final!

What are you looking forward to?

#Python #Python314
Tue, Sep 30, 2025, 15:19 EEST

Three days #

Three days until release and a bug in the Linux kernel has turned a dozen buildbots red…

It’s already been fixed in the kernel, but will take some time to bubble up. We’ll skip that test for relevant kernel versions in the meantime.

#Python #Python314
Sat, Oct 4, 2025, 16:15 EEST

Green #

And back to green!

#Python #Python314
Sun, Oct 5, 2025, 16:58 EEST

Release day! #

First off, check blockers and buildbots.

A new release-blocker appeared yesterday (because of course) but it can wait until 3.14.1.

Three deferred-blockers are also waiting until 3.14.1.

A new tier-2 buildbot failure appeared yesterday (because of course) but it had previously been offline for a month and will need some reconfiguration. Can ignore.

OK, let’s make a Python!

#Python #Python314 #release
Tue, Oct 7, 2025, 11:40 EEST

`run_release.py` #

Next up, merge and backport the final change to What’s New in Python 3.14 to declare it latest stable.

Now start run_release.py, the main release automation script, which does a bunch of pre-checks, runs blurb to create a merged changelog, bumps some numbers, and pushes a branch and tag to my fork. It’ll go upstream at the end of a successful build.

Then kick off the CI to build source zips, docs and Android binaries.

#Python #Python314 #release
Tue, Oct 7, 2025, 12:43 EEST

Installers #

(That’s actually the second CI attempt, we had to update some script arguments following an Android test runner update.)

This build takes about half an hour.

I’ve also informed the Windows and macOS release managers about the tag and they will start up installer builds.

This takes a few hours, so I’ve got time to finish up the release notes.

PEP 101 is the full process, but much is automated and we don’t need to follow it all manually.

#Python #Python314 #release
Tue, Oct 7, 2025, 12:52 EEST

Windows #

The Windows build has been started.

The jobs with profile-guided optimisation (PGO) build once, then collect a profile by running the tests, and then build again using that profile, to see how ‘real’ code executes and optimises for that.

Meanwhile, the docs+source+Android build has finished and the artifacts have been copied to where they need to go with SBOMs created.

#Python #Python314 #release
Tue, Oct 7, 2025, 13:50 EEST

macOS #

The Windows build is ready and macOS is underway.

#Python #Python314 #release
Tue, Oct 7, 2025, 15:36 EEST

Final steps #

macOS installer done, next on to the final publishing and announcing steps.

#Python #Python314 #release
Tue, Oct 7, 2025, 17:02 EEST

🚀 It’s out! #

🥧 Please install and enjoy Python 3.14!

#Python #Python314 #release
Tue, Oct 7, 2025, 17:27 EEST

Finally #

And the last few tasks: announce also on the blog & mailing lists, update the PEP & downloads landing page, fix Discourse post links, unlock the 3.14 branch for the core team to start landing PRs that didn’t need to be in the RC, and eat the pie.

A HUGE thanks to @sovtechfund Fellowship for allowing me to dedicate my time on getting this out 🎉

#Python #Python314 #release
Tue, Oct 7, 2025, 19:28 EEST

Ready prek go!

Sat, 06 Sep 2025 15:50:00 +0000

I’ve been using prek recently as a drop-in replacement for pre-commit.

It uses uv for managing Python virtual environments and dependencies, is rewritten in Rust (because of course) and uses the same .pre-commit-config.yaml as pre-commit.

Its homepage says it’s not yet production ready, but several projects like Apache Airflow and PDM are already using it. I’ve been using it for a while and reporting issues as I find them; the maintainer is quick with fixes and releases. All the hooks I need are now supported.

Getting started #

First install using one of the many methods, then:

cd my-repo
pre-commit uninstall # remove the old hooks
prek install # install the new hooks
prek run --all-files # run all lints on all files
git commit -m "my commit" # run on the changed files in a commit

Benchmarks #

prek is noticeably quicker at installing hooks.

⚡ About 10x faster than pre-commit and uses only a third of disk space.

This 10x is a comparison of installing hooks using the excellent hyperfine benchmarking tool.

Here’s my own comparison.

In the first test, I initially ran pre-commit clean or prek clean to clear their caches. I then ran each tool with run --all-files in serial on the 126 repos I have cloned right now, 84 of which have a .pre-commit-config.yaml file. Each then downloads and installs the hooks when needed, then runs the lint tools.
Second, because running the lint tools should be independent and constant time for both tools, the next test ran install-hooks instead of run --all files.

To get an idea of the amount of work for these two tests, pre-commit reported initialising environments 217 times.
Finally, let’s run hyperfine on the Pillow config, which installs hooks from 14 repos:

❯ hyperfine \
--prepare 'rm -rf ~/.cache/prek/ && rm -rf ~/.cache/pre-commit && rm -rf ~/.cache/uv' \
--setup 'prek --version && pre-commit --version' \
'prek install-hooks' \
'pre-commit install-hooks'

Results #

	`pre-commit`	`prek`	Times faster
`run --all-files`	17m13s	7m59s	2.16
`install-hooks`	10m48s	2m24s	4.50
`hyperfine`	39.841s	5.539s	7.19

The hyperfine results:

Benchmark 1: prek install-hooks
 Time (mean ± σ): 5.539 s ± 0.176 s [User: 8.973 s, System: 5.692 s]
 Range (min … max): 5.231 s … 5.834 s 10 runs

Benchmark 2: pre-commit install-hooks
 Time (mean ± σ): 39.841 s ± 2.017 s [User: 19.760 s, System: 8.203 s]
 Range (min … max): 36.930 s … 43.976 s 10 runs

Summary
 prek install-hooks ran
 7.19 ± 0.43 times faster than pre-commit install-hooks

Give it a try and give it a ⭐!

Bonus #

These are the aliases I have set for pre-commit and prek:

alias pci="pre-commit install --allow-missing-config"
alias pcu="pre-commit uninstall"
alias pca="pre-commit autoupdate --jobs 0"
alias pcr="pre-commit run --all-files"
alias pki="prek install --allow-missing-config"
alias pku="prek uninstall"
alias pka="prek autoupdate --jobs 0"
alias pkr="prek run --all-files"

Where:

install’s --allow-missing-config prevents failing with an error code when a repo has no config file
autoupdate’s --jobs 0 uses all the available threads to make it faster

Header photo: AH-1G Aircraft Maintenance Test Flight Handbook and handwritten checklist for helicopters in the San Diego Air and Space Museum Archive, with no known copyright restrictions.

EuroPython 2025: A roundup of writeups

Mon, 25 Aug 2025 13:59:00 +0000

Some out-of-context quotes:

“We can just bump the version and move on.” – Dr. Brett Cannon
“You just show up. That’s it.” – Rodrigo Girão Serrão
“If it kwargs like a dorg, it’s a dorg.” – Sebastián Ramírez
“Our job will be to put the human in.” – Paul Everitt

20 July 2025

Is AI Leaving the Python Community Behind? by Georgi Ker

21 July 2025

23 July 2025

First EuroPython by Raffaella Suardini

24 July 2025

EuroPython 2025 Recap by Cheuk Ting Ho

11 August 2025

Personal highlights of EuroPython 2025 by Rodrigo Girão Serrão

12 August 2025

EuroPython 2025: “I came for the language, but stayed for the community” by Jürgen Gmach

17 August 2025

PyLadies, PacMan, and Public Health by Gauden Galea

21 August 2025

EuroPython 2025開幕までの道のり⁠⁠、Day 1の注目セッション by Takanori Suzuki

8 September 2025

Recap EuroPython 2025 by Victor Brinkhorst

And a bunch of LinkedIn posts:

Finally, the official photos and videos should be up soon, and here are my photos.

Update #

2025-09-06: Added Gauden Galea’s writeup
2025-09-09: Added Victor Brinkhorst’s writeup

Header photo: Savannah Bailey’s keynote (CC BY-NC-SA 2.0 Hugo van Kemenade).

Run coverage on tests

Fri, 27 Jun 2025 17:41:00 +0000

I recommend running coverage on your tests.

Here’s a couple of reasons why, from the past couple of months.

Example one #

When writing tests, it’s common to copy and paste test functions, but sometimes you forget to rename the new one (see also: the Last Line Effect).

For example:

def test_get_install_to_run_with_platform(patched_installs):
 i = installs.get_install_to_run("<none>", None, "1.0-32")
 assert i["id"] == "PythonCore-1.0-32"
 assert i["executable"].match("python.exe")
 i = installs.get_install_to_run("<none>", None, "2.0-arm64")
 assert i["id"] == "PythonCore-2.0-arm64"
 assert i["executable"].match("python.exe")


def test_get_install_to_run_with_platform(patched_installs):
 i = installs.get_install_to_run("<none>", None, "1.0-32", windowed=True)
 assert i["id"] == "PythonCore-1.0-32"
 assert i["executable"].match("pythonw.exe")
 i = installs.get_install_to_run("<none>", None, "2.0-arm64", windowed=True)
 assert i["id"] == "PythonCore-2.0-arm64"
 assert i["executable"].match("pythonw.exe")

The tests pass, but the first one is never run because its name is redefined. This clearly shows up as a non-run test in the coverage report. In this case, we only need to rename one of them, and both are covered and pass.

But sometimes there’s a bug in the test which would cause it to fail, but we just don’t know because it’s not run.

Tip 1: This can also be found by Ruff’s F811 rule.

Tip 2: pytest’s parametrize is a great way to combine similar test functions with different input data.

Example two #

This is more subtle:

im = Image.new("RGB", (1, 1))
for colors in (("#f00",), ("#f00", "#0f0")):
 append_images = (Image.new("RGB", (1, 1), color) for color in colors)
 im_reloaded = roundtrip(im, save_all=True, append_images=append_images)

 assert_image_equal(im, im_reloaded)
 assert isinstance(im_reloaded, MpoImagePlugin.MpoImageFile)
 assert im_reloaded.mpinfo is not None
 assert im_reloaded.mpinfo[45056] == b"0100"

 for im_expected in append_images:
 im_reloaded.seek(im_reloaded.tell() + 1)
 assert_image_similar(im_reloaded, im_expected, 1)

It’s not so obvious when looking at the code, but Codecov highlights a problem:

The append_images generator is being consumed inside roundtrip(), so we have nothing to iterate over in the for loop – hence no coverage. The fix is to use a list instead of a generator.

Header photo: Misplaced manhole cover (CC BY-NC-SA 2.0 Hugo van Kemenade).

PEPs & Co.

Wed, 14 May 2025 15:45:00 +0000

PEPs #

Here’s Barry Warsaw on the origin of PEPs, or Python Enhancement Proposals (edited from PyBay 2017):

I like backronyms. For those who don’t know: a backronym is where you come up with the acronym first and then you come up with the thing that the acronym stands for. And I like funny sounding words, like FLUFL was one of those. When we were working for CNRI, they also ran the IETF conferences. The IETF is the Internet Engineering Task Force, and they’re the ones who come up with the RFCs. If you look at RFC 822, it defines what an email message looks like.

We got to a point, because we were at CNRI we were more intimately involved in the IETF and how they do standards and things, we observed at the time that there were so many interesting ideas coming in being proposed for Python that Guido really just didn’t have time to dive into the details of everything.

So I thought: well, we have this RFC process, let’s try to mirror some of that so that we can capture the essence of an idea in a document that would serve as a point of discussion, and that Guido could let people discuss and then come in and read the summary of the discussion.

And I was just kind of thinking: well, PEPs, that’s kind of peppy, it’s kind of a funny sounding word. I came up with the word and then I backronymed it into Python Enhancement Proposal. And then I wrote PEP 0 and PEP 1. PEP 0 was originally handwritten, and so I was the first PEP author because I came up with the name PEP.

But the really interesting thing is that you see the E.P. part used in a lot of other places, like Debian has DEPs now. There’s a lot of other communities that have these enhancement proposals so it’s kind of interesting. And then the format of the PEP was directly from that idea of the RFC’s standard.

& Co. #

Here’s a collection of enhancement proposals from different communities.

Acronym	Name
AIP	Airflow Improvement Proposals
AIP	API Improvement Proposals
APE	Astropy Proposals for Enhancement
BIP	Bitcoin Improvement Proposals
CEP	Conda Enhancement Proposals
CFEP	conda-forge’s Enhancement Proposals
DEP	Debian Enhancement Proposals
DEP	Django Enhancement Proposals
FEP	Fediverse Enhancement Proposals
GLEP	Gentoo Linux Enhancement Proposal
IPEP	IPython Enhancement Proposals
JEP	JDK Enhancement Proposals
JEP	JMESPath Enhancement Proposals
JEP	Jupyter Enhancement Proposals
KEP	Kubernetes Enhancement Proposals
NEP	NumPy Enhancement Proposals
PEEP	Pipenv Enhancement Proposals
PEP	Python Enhancement Proposals
SKIP	scikit-image proposals
SLEP	Scikit-learn enhancement proposals
SPEC	Scientific Python Ecosystem Coordination
TIP	Tcl Improvement Proposals
WEP	Write the Docs Enhancement Proposals
XEP	XMPP Extension Protocols
YTEP	yt Enhancement Proposals

Are there more? Let me know!

Header photo: Grand Grocery Co., Lincoln, Nebraska, USA (1942) by The Library of Congress, with no known copyright restrictions.

My most used command-line commands

Fri, 04 Apr 2025 16:50:00 +0000

Following Jeff Triplett’s lead, here’s a list of my most used terminal commands.

No.	Total	Command	Info
1.	1239	`rg`	ripgrep
2.	1038	`c`	aliased to `pycharm`
3.	847	`gc`	aliased to `git commit --verbose` via Oh My Zsh Git plugin
4.	559	`gco`	aliased to `git checkout`
5.	518	`git`
6.	384	`./python.exe`	Python built from source on macOS
7.	365	`cd`
8.	359	`cat`	aliased to `bat -p`
9.	336	`gs`	aliased to `scmpuff_status` via scmpuff to give numeric shortcuts for files
10.	326	`gl`	aliased to `git log` via Oh My Zsh Git plugin
11.	315	`gb`	aliased to `git branch` via Oh My Zsh Git plugin
12.	308	`p`	aliased to `python`
13.	263	`ga`	aliased to `git add` via Oh My Zsh Git plugin
14.	227	`l`	aliased to `ls -lah`
15.	214	`uv`	Python package manager
16.	203	`f`	aliased to `find . \| rg` (TODO: learn `fd`)
17.	189	`python`
18.	175	`gd`	aliased to `git diff` via Oh My Zsh Git plugin
19.	174	`rm`
20.	165	`em`	the CLI emoji keyboard
21.	164	`gh`	aliased to `GH_PAGER="less -FRX" gh`, mostly for `gh co <PR number>`, sometimes `gh cache delete --all`
22.	145	`open`
23.	120	`mv`
24.	120	`make`	for building Python, docs, devguide and PEPs
25.	117	`cp`
26.	116	`tox`
27.	116	`pypi`	see below
28.	111	`bc`	aliased to `/usr/local/bin/bcompare` Beyond Compare diff tool
29.	110	`piu`	aliased to `uv pip install --system -U`
30.	110	`pcr`	aliased to `pre-commit run --all-files`

pypi is a little function in my dotfiles for opening the PyPI page for a package, either pypi <package> or let it try and guess the name from the current directory:

pypi () {
	if [ "$1" = "" ]
	then
		local name=$(python setup.py --name)
	else
		local name=$1
	fi
	echo $name
	open https://pypi.org/project/"$name"/
}

The macOS/iTerm/Oh My Zsh command to get this list:

history | awk '{print $2}' | sort | uniq --count | sort --numeric-sort --reverse | head -30

Header photo: Close-up the typebars of a Olympia Büromaschinenwerke A.G. Erfurt typewriter (CC BY-NC-SA 2.0 Hugo van Kemenade).

Free-threaded Python on GitHub Actions

Tue, 25 Mar 2025 10:25:00 +0000

GitHub Actions now supports experimental free-threaded CPython!

There are three ways to add it to your test matrix:

actions/setup-python: t suffix
actions/setup-uv: t suffix
actions/setup-python: freethreaded variable

actions/setup-python: `t` suffix #

Using actions/setup-python, you can add the t suffix for Python versions 3.13 and higher: 3.13t and 3.14t.

This is my preferred method, we can clearly see which versions are free-threaded and it’s straightforward to test both regular and free-threaded builds.

on: [push, pull_request, workflow_dispatch]

jobs:
 build:
 runs-on: ${{ matrix.os }}
 strategy:
 fail-fast: false
 matrix:
 python-version: [
 "3.13",
 "3.13t", # add this!
 "3.14",
 "3.14t", # add this!
 ]
 os: ["windows-latest", "macos-latest", "ubuntu-latest"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true # needed for 3.14

 - run: |
 python --version --version
 python -c "import sys; print('sys._is_gil_enabled:', sys._is_gil_enabled())"
 python -c "import sysconfig; print('Py_GIL_DISABLED:', sysconfig.get_config_var('Py_GIL_DISABLED'))"

Regular builds will output something like:

Python 3.14.0a6 (main, Mar 17 2025, 02:44:29) [GCC 13.3.0]
sys._is_gil_enabled: True
Py_GIL_DISABLED: 0

And free-threaded builds will output something like:

Python 3.14.0a6 experimental free-threading build (main, Mar 17 2025, 02:44:30) [GCC 13.3.0]
sys._is_gil_enabled: False
Py_GIL_DISABLED: 1

For example: hugovk/test/actions/runs/14057185035

actions/setup-uv: `t` suffix #

Similarly, you can install uv with astral/setup-uv and use that to set up free-threaded Python using the t suffix.

on: [push, pull_request, workflow_dispatch]

jobs:
 build:
 runs-on: ${{ matrix.os }}
 strategy:
 fail-fast: false
 matrix:
 python-version: [
 "3.13",
 "3.13t", # add this!
 "3.14",
 "3.14t", # add this!
 ]
 os: ["windows-latest", "macos-latest", "ubuntu-latest"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 uses: astral-sh/setup-uv@v5 # change this!
 with:
 python-version: ${{ matrix.python-version }}
 enable-cache: false # only needed for this example with no dependencies

 - run: |
 python --version --version
 python -c "import sys; print('sys._is_gil_enabled:', sys._is_gil_enabled())"
 python -c "import sysconfig; print('Py_GIL_DISABLED:', sysconfig.get_config_var('Py_GIL_DISABLED'))"

For example: hugovk/test/actions/runs/13967959519

actions/setup-python: `freethreaded` variable #

Back to actions/setup-python, you can also set the freethreaded variable for 3.13 and higher.

on: [push, pull_request, workflow_dispatch]

jobs:
 build:
 runs-on: ${{ matrix.os }}
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.13", "3.14"]
 os: ["windows-latest", "macos-latest", "ubuntu-latest"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true # needed for 3.14
 freethreaded: true # add this!

 - run: |
 python --version --version
 python -c "import sys; print('sys._is_gil_enabled:', sys._is_gil_enabled())"
 python -c "import sysconfig; print('Py_GIL_DISABLED:', sysconfig.get_config_var('Py_GIL_DISABLED'))"

For example: hugovk/test/actions/runs/39359291708

`PYTHON_GIL=0` #

And you may want to set PYTHON_GIL=0 to force Python to keep the GIL disabled, even after importing a module that doesn’t support running without it.

See Running Python with the GIL Disabled for more info.

With the t suffix:

- name: Set PYTHON_GIL
 if: endsWith(matrix.python-version, 't')
 run: |
 echo "PYTHON_GIL=0" >> "$GITHUB_ENV"

With the freethreaded variable:

- name: Set PYTHON_GIL
 if: "${{ matrix.freethreaded }}"
 run: |
 echo "PYTHON_GIL=0" >> "$GITHUB_ENV"

Please test! #

For free-threaded Python to succeed and become the default, it’s essential there is ecosystem and community support. Library maintainers: please test it and where needed, adapt your code, and publish free-threaded wheels so others can test their code that depends on yours. Everyone else: please test your code too!

Improving licence metadata

Fri, 14 Feb 2025 15:11:00 +0000

What? #

PEP 639 defines a spec on how to document licences used in Python projects.

Instead of using a Trove classifier such as “License :: OSI Approved :: BSD License”, which is imprecise (for example, which BSD licence?), the SPDX licence expression syntax is used.

How? #

`pyproject.toml` #

Change pyproject.toml as follows.

I usually use Hatchling as a build backend, and support was added in 1.27:

 [build-system]
 build-backend = "hatchling.build"
 requires = [
 "hatch-vcs",
- "hatchling",
+ "hatchling>=1.27",
 ]

Replace the freeform license field with a valid SPDX license expression, and add license-files which points to the licence files in the repo. There’s often only one, but if you have more than one, list them all:

 [project]
 ...
-license = { text = "MIT" }
+license = "MIT"
+license-files = [ "LICENSE" ]

Optionally delete the deprecated licence classifier:

 classifiers = [
 "Development Status :: 5 - Production/Stable",
 "Intended Audience :: Developers",
- "License :: OSI Approved :: MIT License",
 "Operating System :: OS Independent",

For example, see humanize#236 and prettytable#350.

Upload #

Then make sure to use a PyPI uploader that supports this.

I recommend using Trusted Publishing which I use with pypa/gh-action-pypi-publish to deploy from GitHub Actions. I didn’t need to make any changes here, just make a release as usual.

Result #

PyPI #

PyPI shows the new metadata:

pip #

pip can also show you the metadata:

$ pip install prettytable==3.13.0
$ pip show prettytable
Name: prettytable
Version: 3.13.0
...
License-Expression: BSD-3-Clause
Location: /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages
Requires: wcwidth
Required-by: norwegianblue, pypistats

Thank you! #

A lot of work went into this. Thank you to PEP authors Philippe Ombredanne for creating the first draft in 2019, to C.A.M. Gerlach for the second draft in 2021, and especially to Karolina Surma for getting the third draft over the finish line and helping with the implementation.

And many projects were updated to support this, thanks to the maintainers and contributors of at least:

Header photo: Amelia Earhart’s 1932 pilot licence in the San Diego Air and Space Museum Archive, with no known copyright restrictions.

I'm excited to join the Sovereign Tech Fellowship

Thu, 06 Feb 2025 11:40:00 +0000

For the duration of 2025, I’m thrilled to join the Sovereign Tech Fellowship for Maintainers!

This is a pilot programme from the Sovereign Tech Agency to pay maintainers of critical open source technologies in the public interest.

I’m one of six participants in this first cohort and I’ll be working on Python, with a focus on CPython, including as release manager for Python 3.14 (under development with full release due in October 2025, supported until 2030) and the next one (development starts in May 2025 for release in October 2026, supported until 2031).

I’m delighted to work with the Sovereign Tech Agency as they really get open source. Supported by Germany’s Ministry for Economic Affairs and Climate Action, they recognise the importance of open source and its place as critical digital infrastructure in the public interest. In their own words:

The Sovereign Tech Agency supports the development, improvement, and maintenance of open digital infrastructure. Our goal is to sustainably strengthen the open source ecosystem. We focus on security, resilience, technological diversity, and the people behind the code.

We often don’t notice how much our lives depend on digital infrastructure until it stops working. But making it available, accessible and secure is key for digitalization in the public interest. The Sovereign Tech Agency invests in open digital infrastructure. It is critical for innovation and economic growth, and forms the foundation for digitalization across sectors.

In addition to CPython core development and release management, I’ll continue to maintain other Python projects such as Pillow, humanize and termcolor, and CLI utilities such as norwegianblue, pypistats and pepotron, and help out more widely in the Python ecosystem.

Tasks include:

Bug fixing, issue triaging, PR review and security improvements
Mentoring new contributors to foster community growth and diversity
Communicating updates to the wider community via blog posts and social media
Workflow, CI, and infrastructure optimisations to streamline core development tasks
Maintenance and improvement of PEP editing infrastructure and related documentation tools
Automation and dashboard creation for triage scripts to assist other contributors
Process and workflow improvements for CI and core contributor environments
Ongoing documentation enhancements, including hosting monthly Documentation Working Group meetings, improving build processes, and focusing on accessibility
Updating and modernising projects for compatibility with the latest Python versions
Implementing stronger security measures and using Trusted Publishing and digital attestations
Adding and maintaining Python support for widely used PyPI packages
Improving community-wide tooling, documentation, and accessibility

Meet my fellow Sovereign Tech Fellows, and by the way, the Sovereign Tech Agency is currently hiring, check out their open positions.

How to delay a Python release

Tue, 04 Feb 2025 18:55:00 +0000

Prologue #

This was a Twitter thread from 15th January 2022 about my first CPython bug. Eight days from report to fix to merge, not bad!

Delay #

I helped delay the release of Python 3.11.0a4! But in a good way! 😇

Python 3.11 is due out in October, but they make early alpha, beta and release candidates available for people to help test Python itself and their own code before the big release.

So I tested Pillow…

Tests #

The Pillow test suite passed with 3.11 ✅

Next I tried building the documentation with 3.11 ❌

The docs program, Sphinx, emitted a couple of warnings. Warnings are often missed because they don’t error. But luckily we use the “-W” option to turn warnings into hard errors.

Sphinx #

Maybe Sphinx isn’t ready for Python 3.11?

Rather than submitting a report with the full Pillow documentation (lots of files) I made a new, “minimal” example with just enough stuff to reproduce it.

This makes it easier to investigate what’s up.

Report 1 #

I reported this to Sphinx. The problem was that a page in a subdirectory could load an image from one directory, but not from another, further away directory.

It occurs for the Python 3.11.0a3 alpha, but not 3.7-3.10.

CPython #

A few hours later the Sphinx maintainer Takeshi said it looks like a change to a part of Python itself – os.path.normpath() – since 3.11.0a3, and as it wasn’t mentioned on the “What’s New in Python 3.11” page it could be a bug in Python.

He asked me to report it to Python.

Report 2 #

I reported it to Python with Takeshi’s even more minimal example.

Half an hour later Christian pointed out a change which may have caused this.

I tested and confirmed.

The next day Steve confirmed it was a bug and set it as a “release blocker”.

Fix #

Steve also said it will needs tests, because this bug slipped out due to a gap in testing.

I didn’t know how to fix the bug, but I could write some test cases!

neonene then took the tests and fixed the bug! In doing so they found even more bugs!

Merge #

These extra bugs also existed in earlier versions.

But it turns out path handling can get pretty complicated in places, so Steve decided to only fix my bug now to get it released and the others can be sorted later.

The fix was merged and I confirmed it also worked with Sphinx ✅

Conclusion #

And that’s about it!

It’s now fixed in 3.11.0a4; much better to find these before 3.11.0 final is released to the world in October. Along the way we found more issues to address.

Short version: test your code with 3.11 now, you may find issues in your code or in Python itself 🚀

Epilogue #

Back to 2025: Please test and delay Python 3.14 alpha – but in a good way! 😇

A surprising thing about PyPI's BigQuery data

Sun, 24 Nov 2024 20:45:31 +0000

You can get download numbers for PyPI packages (or projects) from a Google BigQuery dataset. You need a Google account and credentials, and Google gives 1 TiB of free quota per month.

Each month, I have automation to fetch the download numbers for the 8,000 most popular packages over the past 30 days, and make it available as more accessible JSON and CSV files at Top PyPI Packages. This data is widely used for research in academia and industry.

However, as more packages and releases are uploaded to PyPI, and there are more and more downloads logged, the amount of billed data increases too.

This chart shows the amount of data billed per month.

At first, I was only collecting downloads data for 4,000 packages, and it was fetched for two queries: downloads over 365 days and over 30 days. But as time passed, it started using up too much quota to download data for 365 days.

So I ditched the 365-day data, and increased the 30-day data from 4,000 to 5,000 packages. Later, I checked how much quota was being used and increased from 5,000 packages to 8,000 packages.

But then I exceeded the BigQuery monthly quota of 1 TiB fetching data for July 2024.

To fetch the missing data and investigate what’s going in, I started Google Cloud’s 90-day, $300 (€277.46) free-trial 💸

Here’s what I found!

Finding: it costs more to get data for downloads from only pip than from all installers #

I use the pypinfo client to help query BigQuery. By default, it only fetches downloads for pip.

Only pip #

This command gets one day’s download data for the top 10 packages, for pip only:

$ pypinfo --limit 10 --days 1 "" project
Served from cache: False
Data processed: 58.21 GiB
Data billed: 58.21 GiB
Estimated cost: $0.29

Results:

project	download count
boto3	37,251,744
aiobotocore	16,252,824
urllib3	16,243,278
botocore	15,687,125
requests	13,271,314
s3fs	12,865,055
s3transfer	12,014,278
fsspec	11,982,305
charset-normalizer	11,684,740
certifi	11,639,584
Total	158,892,247

All installers #

Adding the --all flag gets one day’s download data for the top 10 packages, for all installers:

$ pypinfo --all --limit 10 --days 1 "" project
Served from cache: False
Data processed: 46.63 GiB
Data billed: 46.63 GiB
Estimated cost: $0.23

project	download count
boto3	39,495,624
botocore	17,281,187
urllib3	17,225,121
aiobotocore	16,430,826
requests	14,287,965
s3fs	12,958,516
charset-normalizer	12,781,405
certifi	12,647,098
setuptools	12,608,120
idna	12,510,335
Total	168,226,197

So we can see the default pip-only costs an extra 25% data processed and data billed, and costs an extra 25% in dollars.

Unsurprisingly, the actual download counts are higher for all installers. The ranking has changed a bit, but I expect we’re still getting more-or-less the same packages in the top thousands of results.

Queries #

It sends a query like this to BigQuery for only pip:

SELECT
 file.project as project,
 COUNT(*) as download_count,
FROM `bigquery-public-data.pypi.file_downloads`
WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY)
 AND details.installer.name = "pip"
GROUP BY
 project
ORDER BY
 download_count DESC
LIMIT 10

And for all installers:

SELECT
 file.project as project,
 COUNT(*) as download_count,
FROM `bigquery-public-data.pypi.file_downloads`
WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY)
GROUP BY
 project
ORDER BY
 download_count DESC
LIMIT 10

These queries are the same, except the default has an extra AND details.installer.name = "pip" condition. It seems reasonable it would cost more to do extra filtering work.

Installers #

Let’s look at the installers:

$ pypinfo --all --limit 100 --days 1 "" installer
Served from cache: False
Data processed: 29.49 GiB
Data billed: 29.49 GiB
Estimated cost: $0.15

installer name	download count
pip	1,121,198,711
uv	117,194,833
requests	29,828,272
poetry	23,009,454
None	8,916,745
bandersnatch	6,171,555
setuptools	1,362,797
Bazel	1,280,271
Browser	1,096,328
Nexus	593,230
Homebrew	510,247
Artifactory	69,063
pdm	62,904
OS	13,108
devpi	9,530
conda	2,272
pex	194
Total	1,311,319,514

pip still by far the most popular, and unsurprising uv is up there too, with about 10% of pip’s downloads.

The others are about 25% or less of uv. A lot of them are mirroring services that we wanted to exclude before.

I think given uv’s importance, and my expectation that it will continue to take a bigger share of the pie, plus especially the extra cost for filtering by just pip, means that we should switch to fetching data for all downloaders. Plus the others don’t account for that much of the pie.

Finding: the number of packages doesn’t affect the cost #

This was the biggest surprise. Earlier I’d been increasing or decreasing the number to try and remain under quota. But it turns out it makes no difference how many packages you query!

I fetched data for just one day and all installers for different package limits: 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000. Sample query:

SELECT
 file.project as project,
 COUNT(*) as download_count,
FROM `bigquery-public-data.pypi.file_downloads`
WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -2 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY)
GROUP BY
 project
ORDER BY
 download_count DESC
LIMIT 8000

Result: Interestingly, the cost is the same for all limits (1000-8000): $0.31.

Repeating with one day but filtering for pip only:

Result: Cost increased to $0.39 but again the same for all limits.

Let’s repeat with all installers, but for 30 days, and this time query in decreasing limits, in case we were only paying for incremental changes: 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000:

Result: Again, the cost is the same regardless of package limit: $4.89 per query.

Well then, let’s repeat with the limit increasing by powers of ten, up to 1,000,000! This last one fetches data for all 531,022 packages on PyPI:

limit	projects count	estimated cost	bytes billed	bytes processed
1	1	0.20	43,447,746,560	43,447,720,943
10	10	0.20	43,447,746,560	43,447,720,943
100	100	0.20	43,447,746,560	43,447,720,943
1000	1,000	0.20	43,447,746,560	43,447,720,943
8000	8,000	0.20	43,447,746,560	43,447,720,943
10000	10,000	0.20	43,447,746,560	43,447,720,943
100000	100,000	0.20	43,447,746,560	43,447,720,943
1000000	531,022	0.20	43,447,746,560	43,447,720,943

Result: Again, same cost, whether for 1 package or 531,022 packages!

Finding: the number of days affects the cost #

No surprise. I’d earlier noticed 365 days too took much quota, and I could continue with 30 days.

Here’s the estimated cost and bytes billed (for one package, all installers) between one and 30 days (f"pypinfo --all --json --indent 0 --days {days} --limit 1 '' project"), showing a roughly linear increase:

Conclusion #

It doesn’t matter how many packages I fetch data for, I might as well fetch all and make it available to everyone, depending on the size of the data file. It will make sense to still offer a smaller file with 8,000 or so packages: often you just need a large-ish yet manageable number.
It costs more to filter for only downloads from pip, so I’ve switched to fetching data for all installers.
The number of days affects the cost, so I will need to decrease this in the future to stay within quota. For example, at some point I may need to switch from 30 to 25 days, and later from 25 to 20 days.

More details from the investigation, the scripts and data files can be found at hugovk/top-pypi-packages#36.

And let me know if you know any tricks to reduce costs!

Header photo: “The Balancing Rock, Stonehenge, Near Glen Innes, NSW” by the Royal Australian Historical Society, with no known copyright restrictions.

Speed up CI with uv ⚡

Sat, 02 Nov 2024 13:11:22 +0000

We can use uv to make linting and testing on GitHub Actions around 1.5 times as fast.

Linting #

When using pre-commit for linting:

name: Lint

on: [push, pull_request, workflow_dispatch]

env:
 FORCE_COLOR: 1

permissions:
 contents: read

jobs:
 lint:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 with:
 persist-credentials: false
 - uses: actions/setup-python@v5
 with:
 python-version: "3.x"
 cache: pip
 - uses: pre-commit/action@v3.0.1

We can replace pre-commit/action with tox-dev/action-pre-commit-uv:

 - uses: actions/setup-python@v5
 with:
 python-version: "3.x"
- cache: pip
- - uses: pre-commit/action@v3.0.1
+ - uses: tox-dev/action-pre-commit-uv@v1

name: Lint

on: [push, pull_request, workflow_dispatch]

env:
 FORCE_COLOR: 1

permissions:
 contents: read

jobs:
 lint:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 with:
 persist-credentials: false
 - uses: actions/setup-python@v5
 with:
 python-version: "3.x"
 - uses: tox-dev/action-pre-commit-uv@v1

This means uv will create virtual environments and install packages for pre-commit, which is faster for the initial seed operation when there’s no cache.

Lint comparison #

For example: python/blurb#32

	Before	After	Times faster
No cache	60s	37s	1.62
With cache	11s	11s	1.00

Testing #

When testing with tox:

name: Test

on: [push, pull_request, workflow_dispatch]

permissions:
 contents: read

env:
 FORCE_COLOR: 1

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]

 steps:
 - uses: actions/checkout@v4
 with:
 persist-credentials: false

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true
 cache: pip

 - name: Install dependencies
 run: |
 python --version
 python -m pip install -U pip
 python -m pip install -U tox

 - name: Tox tests
 run: |
 tox -e py

We can replace tox with tox-uv:

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true
- cache: pip

- - name: Install dependencies
- run: |
- python --version
- python -m pip install -U pip
- python -m pip install -U tox
+ - name: Install uv
+ uses: hynek/setup-cached-uv@v2

 - name: Tox tests
 run: |
- tox -e py
+ uvx --with tox-uv tox -e py

name: Test

on: [push, pull_request, workflow_dispatch]

permissions:
 contents: read

env:
 FORCE_COLOR: 1

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

 steps:
 - uses: actions/checkout@v4
 with:
 persist-credentials: false

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true

 - name: Install uv
 uses: hynek/setup-cached-uv@v2

 - name: Tox tests
 run: |
 uvx --with tox-uv tox -e py

tox-uv is tox plugin to replace virtualenv and pip with uv in your tox environments. We only need to install uv, and use uvx to both install tox-uv and run tox, for faster installs of tox, the virtual environment, and the dependencies within it.

Test comparison #

For example: python/blurb#32

	Before	After	Times faster
No cache	2m 0s	1m 26s	1.40
With cache	1m 58s	1m 22s	1.44

No pip with tox-uv #

One difference with uv environments compared to regular venv/virtualenv ones is that they do not have pip. This means calls to pip need replacing, for example in tox.ini:

 [testenv]
-commands_pre =
- {envpython} -m pip install -U -r requirements.txt
+deps =
+ -r requirements.txt
 pass_env =
 FORCE_COLOR
 commands =
 {envpython} -m pytest {posargs}

If you still need pip (or setuptools or wheel), add uv_seed = True to your [testenv] to inject them.

Bonus tip #

Run the new tool zizmor to find security issues in GitHub Actions.

Header photo: “Road cycling at the 1952 Helsinki Olympics” by Olympia-Kuva Oy & Helsinki City Museum, Public Domain.

Python Core Sprint 2024

Sat, 05 Oct 2024 19:39:48 +0000

🐍🏃The week before last was the annual Python Core Dev Sprint, graciously hosted by Meta in Bellevue, WA!

The idea: bring a bunch of Python core team members, triagers, and special guests to the same room for a week. It’s hugely beneficial and productive, we held many in-depth discussions that just don’t happen when we’re all remote and async, and got to work on many different things together.

The sprint room

During the week, I reviewed 39 PRs, created 15, merged 10, updated 4, and closed 2 issues.

Monday highlights #

As release manager for Python 3.14, I discussed with Brett Cannon one of his project ideas which will come after lock files, and after the next big one.

Also as RM, discussed with Russell Keith-Magee, Ned Deily, Łukasz Langa and Thomas Wouters about including official binaries for iOS and Android, which wandered into ideas about security releases.

I did some maintenance of our PyPI projects, adding PEP 740 attestations, support for the new Python 3.13 and dropping support for the very-nearly-EOL 3.8.

Tuesday highlights #

Started investigating slow doctest on 3.13+ with Alex Waygood, who on Wednesday narrowed it down to a problem with the new incremental garbage collector, which would go on to be reverted by Friday and result in Python 3.13’s Monday release to be postponed and replaced with an extra release candidate. Not ideal, but much better to discover these things before the big release.

We had a Q&A session with the Steering Council: Barry Warsaw, Emily Morehouse, Gregory P. Smith, Pablo Galindo Salgado and Thomas.

The Python Steering Council

Proofread Guido van Rossum’s STAR voting proposal for electing future steering councils.

Discussed with Eric Snow his novel method for displaying many code samples in a table, using <details> disclosures to prevent the table being too wide. Looks like a good solution!

Wednesday highlights #

I applied the finishing touches to PEP 2026 (Calendar versioning for Python) and Barry gave it a final review. Ready for submission!

Seth Larson, the PSF Security Developer-in-Residence, wasn’t at the sprint but we discussed our plan to stop providing GPG signatures for CPython and rely on SigStore instead. Expect a PEP soon!

Also not at the sprint, I recommended PSF Infrastructure Engineer Jacob Coffee as a CPython triager. Welcome aboard!

The whole room discussed including static type annotations in CPython.

We had a Q&A session with two of the three Developers-in-Residence, Łukasz and Petr Viktorin.

Q&A with Łukasz and Petr

Discussed expanding the voter pool for Steering Council elections with Mariatta, Greg and Thomas.

Larry Hastings handed out, in return for oohs and aahs, some nice P.C.D.S. 2024 stickers he generously designed and printed up for us. Thanks!

PCDS 2024 stickers by Larry

Thursday highlights #

On the 26th September, at 10:26 Bellevue time (20:26 Helsinki time), I submitted PEP 2026 to the Steering Council!🤞

Brett discussed whether we should update PEP 387 to prefer 5 year deprecations instead of 2 years.

Brandt Bucher gave us all an update on the progress of the Just-in-Time (JIT) compiler (“we went from 0% slower to 0% faster!”) and we discussed plans for Python 3.14.

Because I couldn’t attend Thursday’s Helsinki Python meetup due to being at another kind of Python meetup on the other side of the world, I gave the famous HelPy quiz to the assembled core devs. Unsurprisingly they did pretty well, but the most incorrect answer was a pleasant surprise: we’ve had ~400 not ~80 new contributors to Python 3.13!

Pablo performed card tricks!

Magic from Pablo

Meta took us out for a delicious dinner at a local fish restaurant. Thank you!

Friday highlights #

Mariatta presented ideas to Jelle Zijlstra, Petr, Russell and me about to use modern tools to create a modern, interactive tutorial.

Also during the week, continued work with Adam Turner on improving the docs.python.org build. Adam wasn’t at the sprint, so tag-teamed PR reviews overnight. After much work straddling many teams, projects and repos, we’ve got the full HTML build loop for 13 languages × 3 versions down from over 40 hours to just under 9 hours, with more improvements coming.

Made a demo of the CPython docs using the PyData Sphinx Theme.

Along with around 25 others, I was on Łukasz and Pablo’s core.py podcast.

Łukasz and Pablo in their ad-hoc podcast studio in a Meta meeting room

Itamar gave us cake for the podcast’s first birthday!

cake.py. Photo by Itamar Oren.

Thank you #

It was a hugely productive week, big thanks to Itamar Oren and Meta for organising and hosting!

See also Mariatta’s excellent blog posts, and I recommend the core.py podcast with short interviews with some 25 attendees! Łukasz and Pablo were also guests on the Changelog podcast during the sprint.

Header photo by Itamar Oren

PyCon US 2024: A roundup of writeups

Thu, 13 Jun 2024 14:41:55 +0000

If you read just one, check Kati’s thorough recap!

22nd May 2024

Echos of the People API user guide by Ned Batchelder

24th May 2024

Wagtailers spread their wings at PyCon 2024 by Meagen Voss (@vossisboss)
Flet at PyCon US 2024 by Feodor Fitsner

27th May 2024

PyCon US 2024: My First PyCon in US 🫶🏻 by Tomy Hsieh

28th May 2024

PyCon 2024 Reflection by Trey Hunner
Weeknotes: PyCon US 2024 by Simon Willison
3 key takeaways from PyCon US 2024 by Luis Juncal & Yolanda Robla

30th May 2024

pyOpenSci at PyCon US 2024 - Python packaging and community by Leah Wasser
Our Experience at PyCon US 2024 in Pittsburgh by Ngazetungue Muheue (@ngazetungue)

1st June 2024

PyCon US 2024 Recap by Kati Michel (@katherinemichel)

4th June 2024

PyCon US 2024 by Yilei Yang

13th June 2024

PyCon US 2024 as Security Developer-in-Residence by Seth Michael Larson (@sethmlarson)

14th June 2024

The Python Language Summit 2024 by Seth Michael Larson (@sethmlarson)

16th June 2024

PyCon US 2024 Highlights by David Foster

19th June 2024

From Pittsburgh to New York: A PyCon US 2024 Adventure by Monica Oyugi (@monicaoyugi)

Header photo: Downtown Pittsburgh seen between the Andy Warhol Bridge and Roberto Clemente Bridge

Help test Python 3.13!

Tue, 04 Jun 2024 13:04:03 +0000

Calling all Python library maintainers! 🐍

The Python 3.13 beta is out! 🎉

PEP 719 defines the release schedule for Python 3.13.0:

The first beta candidate came out on 8th May 2024
The first release candidate is set for 30th July 2024
And the full release is set for 1st October 2024

In his announcement, Thomas Wouters, release manager for Python 3.12 and 3.13, said:

We strongly encourage maintainers of third-party Python projects to test with 3.13 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Tuesday 2024-07-30). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.13.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.13 as possible during the beta phase.

Test with 3.13 #

It’s now time for us library maintainers to start testing our projects with 3.13. There’s two big benefits:

There have been removals and changes in Python 3.13. Testing now helps us make our code compatible and avoid any big surprises (for us and our users) at the big launch in October.
We might find bugs in Python itself! Reporting those will help get them fixed and help everyone.

How #

GitHub Actions: setup-python #

To test the latest alpha, beta or release candidate with actions/setup-python, add 3.13 and allow-prereleases: true to your workflow matrix.

For example:

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.10", "3.11", "3.12", "3.13"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true

(We can instead use 3.13-dev and omit allow-prereleases: true, but I find the above a bit neater, and when 3.13.0 final is released in October, it will continue testing with full release versions.)

GitHub Actions: deadsnakes #

For the bleeding edge, we can use deadsnakes/action to test the latest nightly build:

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.10", "3.11", "3.12", "3.13-dev"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 if: "!endsWith(matrix.python-version, '-dev')"
 uses: actions/setup-python@v5
 with:
 python-version: ${{ matrix.python-version }}

 - uses: deadsnakes/action@v3.1.0
 name: Set up Python ${{ matrix.python-version }} (deadsnakes)
 if: endsWith(matrix.python-version, '-dev')
 with:
 python-version: ${{ matrix.python-version }}

When to support 3.13? #

When should you declare support and add the Programming Language :: Python :: 3.13 Trove classifier? Some projects already have!

If you have a pure Python project, you can release now.

If you have C extensions and other projects depend on yours, a preview release with wheels will help them test and prepare. I’ve already started releasing these.

ABI breaks? #

ABI breaks during the beta are infrequent and unintentional. If they happen, you can rebuild your wheels and upload them to an existing PyPI release by adding an optional build tag to the filename:

The wheel filename is {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl.

build tag: […] Acts as a tie-breaker if two wheel file names are the same in all other respects (i.e. name, version, and other tags).

For example, this updates the filename and metadata with build number 1, and removes the original file:

python -m pip install "wheel>=0.4.0"
wheel tags --build=1 --remove sampleproject-5.0.0-cp313-cp313-macosx_10_10_x86_64.whl
# this creates a file named sampleproject-5.0.0-1-cp313-cp313-macosx_10_10_x86_64.whl

Upload it, and the new file will be used instead of the old one. See also Brett Cannon’s advice on making new wheels.

In any case, let’s start testing 3.13 now! 🚀

Printable PyCon 2024 schedule

Sat, 11 May 2024 11:01:33 +0000

Want to print out the PyCon US schedule? Paper doesn’t run out of batteries, is easy to scribble on, and stuff into a pocket (technical term: the affordances of paper).

Here’s some custom CSS and JavaScript to make it nicely printable.

Install the Styler browser extension
View a PyCon schedule page such as https://us.pycon.org/2024/schedule/talks/ and click the Styler extension’s S icon
Paste this CSS into the upper box:

body.pycon-schedule div.internal-page-header,
body.pycon-schedule div.panel-heading,
body.pycon-schedule footer,
body.pycon-schedule div.badges {
 display: none;
}

body.pycon-schedule .container {
 margin: 0;
 max-width: fit-content;
}
body.pycon-schedule main.content {
 margin: 0;
 width: 100% !important;
}
body.pycon-schedule .calendar {
 left: auto !important;
 width: 100% !important;
}
body.pycon-schedule .slot {
 padding: 5px;
}
@media print {
 a:after { content:''; }
 a[href]:after { content: none !important; }
}

Paste this JavaScript into the lower box:

$(document).ready(function () {
 if (
 window.location.pathname.match("2024/schedule/") &&
 !window.location.pathname.match("2024/schedule/presentation/")
 ) {
 $("body").addClass("pycon-schedule");
 }
});

Print!

It’ll run on any of the https://us.pycon.org/2024/schedule/* pages, but not the individual presentation pages such as https://us.pycon.org/2024/schedule/presentation/61/

It’s not perfect, the right edge is slightly cut off, but it’s more printable than the original.

Once printed, you can click the Styler icon and the x button to disable Styler for the site so you can browse the original version.

Help us test free-threaded Python without the GIL

Fri, 10 May 2024 12:18:27 +0000

Python 3.13 is due out in October 2024 and work is underway to implement experimental support for PEP 703 - Making the Global Interpreter Lock Optional in CPython.

See also Free-threaded CPython in “What’s New in Python 3.13?”

As the Steering Council noted in their acceptance of the PEP, to succeed it’s important to have community support.

Projects will need to test their code with free-threaded (aka “nogil” but don’t call it that!) Python builds to help us find bugs in CPython, and to check their code is compatible.

Here’s some ways to test.

Official installers
Build it yourself
deadsnakes
GitHub Actions?
Fedora

Official installers #

The official macOS (starting in beta 2) and Windows installers has an option to install the free-threading binaries, which also installs python3.13t alongside the regular python3.13:

Build it yourself #

Check out the CPython Git repo and build yourself using the --disable-gil configuration flag.

For example, on macOS I run:

$ GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \
 GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \
 ./configure --config-cache \
 --with-system-libmpdec \
 --with-openssl="$(brew --prefix openssl@3.0)" \
 --disable-gil && make -s -j10
$ ./python.exe -c "import sysconfig; print(sysconfig.get_config_var('Py_GIL_DISABLED'))"
1
$ ./python.exe -c "import sys; print(sys._is_gil_enabled())"
False

More details in the devguide:

https://devguide.python.org/getting-started/setup-building/

GitHub Actions: deadsnakes/action #

We can use the deadsnakes/action to test Ubuntu.

For example:

on: [push, pull_request, workflow_dispatch]

jobs:
 build:
 runs-on: ubuntu-latest
 strategy:
 matrix:
 python-version: [3.12, 3.13-dev]
 name: main
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-python@v5
 if: "!endsWith(matrix.python-version, '-dev')"
 with:
 python-version: ${{ matrix.python-version }}
 - uses: deadsnakes/action@v3.1.0
 if: endsWith(matrix.python-version, '-dev')
 with:
 python-version: ${{ matrix.python-version }}
 nogil: true
 - run: |
 python --version --version
 which python
 python -c "import sysconfig; print(sysconfig.get_config_var('Py_GIL_DISABLED'))"
 python -c "import sys; print(sys._is_gil_enabled())"

GitHub Actions: actions/setup-python #

I’ve asked for support at actions/setup-python#771, they’re looking into it 🤞

In the meantime, give it an upvote, and use deadsnakes/action ⤴️

deadsnakes: PPA #

The deadsnakes project provides Personal Package Archives of Python packaged for Ubuntu, included free-threaded builds.

For example, python3.13-nogil under python3.13 - 3.13.0~a2-1+focal1 and python3.13 - 3.13.0~a2-1+jammy1 at https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa/+packages

Fedora #

Fedora uses the python3.13t executable name as decided by the Steering Council:

$ # Install
$ sudo dnf install python3.13-freethreading
$ # Run
$ /usr/bin/python3.13t -c "import sysconfig; print(sysconfig.get_config_var('Py_GIL_DISABLED'))"
1
$ /usr/bin/python3.13t -c "import sys; print(sys._is_gil_enabled())"
False

How to check your build #

To confirm if you’re using a free-threaded build, check the double --version option (starting in beta 2):

$ # Regular build
$ python3.13 --version --version
Python 3.13.0b2 (v3.13.0b2:3a83b172af, Jun 5 2024, 12:50:24) [Clang 15.0.0 (clang-1500.3.9.4)]
$ # Free-threaded build
$ python3.13t --version --version
Python 3.13.0b2 experimental free-threading build (v3.13.0b2:3a83b172af, Jun 5 2024, 12:57:31) [Clang 15.0.0 (clang-1500.3.9.4)]

Or the Py_GIL_DISABLED macro:

$ # Regular build
$ python3.13 -c "import sysconfig; print(sysconfig.get_config_var('Py_GIL_DISABLED'))"
0
$ # Free-threaded build
$ python3.13t -c "import sysconfig; print(sysconfig.get_config_var('Py_GIL_DISABLED'))"
1

By default, the GIL is disabled for free-threaded builds, and can be re-enabled by setting the PYTHON_GIL environment variable to 1 or running Python with -X gil 1. You can check with sys._is_gil_enabled():

$ # Regular build: GIL is always enabled
$ python3.13 -c "import sys; print(sys._is_gil_enabled())"
True
$ # Free-threaded build: GIL is disabled by default
$ python3.13t -c "import sys; print(sys._is_gil_enabled())"
False
$ # Free-threaded build: re-enable with -X
$ python3.13t -X gil=1 -c "import sys; print(sys._is_gil_enabled())"
True
$ # Free-threaded build: re-enable with env var
$ PYTHON_GIL=1 python3.13t -c "import sys; print(sys._is_gil_enabled())"
True

References:

Sphinx docs: How to activate tabs for your OS

Tue, 02 Apr 2024 18:41:30 +0000

On the Python Developer’s Guide and Pillow documentation we have some pages with tabs for different operating systems:

It’s possible to add some JavaScript so that the matching tab is activated based on the visitor’s operating system.

Here’s how!

Sphinx Inline Tabs #

First add the Sphinx Inline Tabs extension to your docs' requirements.txt:

# requirements.txt
sphinx-inline-tabs>=2023.4.21

JavaScript #

Next, add activate_tab.js to your _static/ directory:

// activate_tab.js
// Based on https://stackoverflow.com/a/38241481/724176
function getOS() {
 const userAgent = window.navigator.userAgent,
 platform = window.navigator?.userAgentData?.platform || window.navigator.platform,
 macosPlatforms = ["macOS", "Macintosh", "MacIntel", "MacPPC", "Mac68K"],
 windowsPlatforms = ["Win32", "Win64", "Windows", "WinCE"],
 iosPlatforms = ["iPhone", "iPad", "iPod"];

 if (macosPlatforms.includes(platform)) {
 return "macOS";
 } else if (iosPlatforms.includes(platform)) {
 return "iOS";
 } else if (windowsPlatforms.includes(platform)) {
 return "Windows";
 } else if (/Android/.test(userAgent)) {
 return "Android";
 } else if (/Linux/.test(platform)) {
 return "Unix";
 }

 return "unknown";
}

function activateTab(tabName) {
 // Find all label elements containing the specified tab name
 const labels = document.querySelectorAll(".tab-label");

 labels.forEach((label) => {
 if (label.textContent.includes(tabName)) {
 // Find the associated input element using the 'for' attribute
 const tabInputId = label.getAttribute("for");
 const tabInput = document.getElementById(tabInputId);

 // Check if the input element exists before attempting to set the "checked" attribute
 if (tabInput) {
 // Activate the tab by setting its "checked" attribute to true
 tabInput.checked = true;
 }
 }
 });
}

`conf.py` #

Add the extension and JavaScript to your conf.py:

# conf.py
extensions = [
 "sphinx_inline_tabs",
]

html_js_files = [
 "activate_tab.js",
]

reStructuredText #

Almost there!

Add tabs to the reStructuredText files.

For example, here we have three different commands; one for Unix, one for macOS, and one for Windows:

.. tab:: Unix

 .. code-block:: shell

 ./python -m test -h

.. tab:: macOS

 .. code-block:: shell

 ./python.exe -m test -h

.. tab:: Windows

 .. code-block:: dosbatch

 .\python.bat -m test -h

Finally, add the JavaScript to the same reST page to activate the correct tab:

.. raw:: html

 <script>
 document.addEventListener('DOMContentLoaded', function() {
 activateTab(getOS());
 });
 </script>

You can see the results here. When the page loads, the browser finds the browser name, and activates the tabs with a matching name.

If you have many sets of tabs on a page, the corresponding OS tab will be activated for all. And if you click on another OS tab, all the others with the same name are activated.

Happy tabbing!

Header photo: “The Great Pyramid and the Sphinx” by National Galleries of Scotland, with no known copyright restrictions.

Tech style guides

Sat, 02 Mar 2024 15:25:54 +0000

Here’s some tech style guides:

And some other style guides:

TIL: exclude_also with coverage.py

Mon, 06 Nov 2023 20:05:55 +0000

Sometimes you have code you want to exclude from the test coverage report, because it doesn’t really make sense to test it.

For example, maybe you want to exclude:

if __name__ == "__main__":
 main()

The old advice was to add something like this to .coveragerc:

[report]
# Regexes for lines to exclude from consideration
exclude_lines =
 # Have to re-enable the standard pragma:
 pragma: no cover

 # Don't complain if non-runnable code isn't run:
 if __name__ == .__main__.:

But since coverage.py 7.2.0 (2023-02-22) you can use exclude_also instead and skip that pragma:

[report]
# Regexes for lines to exclude from consideration
exclude_also =
 # Don't complain if non-runnable code isn't run:
 if __name__ == .__main__.:

Which is:

 [report]
 # Regexes for lines to exclude from consideration
-exclude_lines =
- # Have to re-enable the standard pragma:
- pragma: no cover
-
+exclude_also =
 # Don't complain if non-runnable code isn't run:
 if __name__ == .__main__.:

Thanks #

To Brian Okken for the tip.

To Ned Batchelder for maintaining Coverage.py.

To the Library of Congress and Flickr Commons for the photo of a covered wagon.

Why are there still so many downloads for EOL Python 3.7?

Tue, 22 Aug 2023 19:25:29 +0000

Python 3.7 was first released on 2018-06-27 and recently reached end-of-life on 2023-06-27 (PEP 537).

This means it is no longer receiving security updates and you should upgrade to a newer version (at least 3.8, but preferably 3.11):

Source: Python Developer's Guide

However, if you look at download numbers from PyPI, 3.7 still accounts for a large share. 3.7 accounted for 25% of all downloads from PyPI in July 2023, compared with 27% for 3.8:

Source: pypi-tools

But why does such an old Python version have so many downloads?

All downloads #

Let’s dig into the numbers using a handy tool called pypinfo, which helps us analyse the PyPI data from Google BigQuery.

This command fetches all of yesterday’s downloads per Python version:

pypinfo --days 1 --percent --markdown "" pyversion

Python version	percent	download count
3.8	25.00%	189,678,872
3.7	23.35%	177,150,010
3.9	20.25%	153,663,903
3.10	15.52%	117,751,108
3.11	6.90%	52,381,884
3.6	6.29%	47,749,879
2.7	2.32%	17,602,650
3.5	0.23%	1,778,388
3.4	0.10%	770,135
3.12	0.03%	224,223
3.13	0.00%	3,920
3.3	0.00%	1,165
2.8	0.00%	57
3.2	0.00%	35
None	0.00%	3
Total		758,756,232

All downloads by OS #

But what happens if we check which distros are responsible for those downloads?

This command gives us the top 20:

pypinfo --days 1 --limit 20 --percent --markdown "" system distro pyversion

system name	distro name	Python version	percent	download count
Linux	Ubuntu	3.8	18.87%	128,991,062
Linux	Amazon Linux	3.7	14.44%	98,738,952
Linux	Ubuntu	3.9	12.14%	83,019,828
Linux	Ubuntu	3.10	9.66%	66,019,309
Linux	Ubuntu	3.7	5.89%	40,257,060
Linux	Debian GNU/Linux	3.8	5.70%	38,958,367
Linux	Debian GNU/Linux	3.9	5.63%	38,482,436
Linux	Debian GNU/Linux	3.7	4.25%	29,035,532
Linux	Debian GNU/Linux	3.10	4.15%	28,348,346
Linux	Debian GNU/Linux	3.6	3.14%	21,441,883
Linux	Ubuntu	3.11	3.01%	20,570,619
Linux	Debian GNU/Linux	3.11	2.72%	18,588,584
Linux	Amazon Linux	3.9	2.43%	16,593,595
Linux	CentOS Linux	3.6	1.70%	11,605,142
Linux	Amazon Linux	3.8	1.47%	10,035,087
Linux	Amazon Linux	3.10	1.46%	9,969,514
Linux	Ubuntu	2.7	1.02%	6,960,697
Linux	Amazon Linux AMI	3.6	0.79%	5,390,823
Linux	Ubuntu	3.6	0.79%	5,388,370
Windows	None	3.10	0.76%	5,227,519
Total				683,622,725

We can see Ubuntu with 3.8 is responsible for the largest share of 17%. (That’s fine, 3.8 is supported until 2024-10-14.)

The next is Amazon Linux with 3.7, responsible for a whopping 15% of all downloads!

The others responsible for 3.7 have a much lower share: Ubuntu (6%) and Debian (4%).

Tip: replace "" in the commands above with a package name to get data for just that package, for example:

pypinfo --days 1 --limit 20 --percent --markdown requests system distro pyversion

Header photo: Space shuttle Discovery landing at Edwards Air Force Base, California, 9th December, 1992 (source: NASA on The Commons)

Help test the Python 3.12 release candidate!

Fri, 26 May 2023 12:30:43 +0000

Calling all Python library maintainers! 🐍

The third and final Python 3.12 release candidate is out! 🎉

PEP 693 defines the release schedule for Python 3.12.0:

The first release candidate came out on 6th August 2023
The second ~~and final~~ release candidate came out on 6th September 2023
The third and final release candidate came out on 19th September 2023
And the full release is set for 2nd October 2023

In his announcement, Thomas Wouters, release manager for Python 3.12 and 3.13, said:

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.12 compatibilities during this phase, and where necessary publish Python 3.12 wheels on PyPI to be ready for the final release of 3.12.0. Any binary wheels built against Python 3.12.0rc3 will work with future versions of Python 3.12. As always, report any issues to the Python bug tracker.

Test with 3.12 #

It’s now time for us library maintainers to start testing our projects with 3.12. There’s two big benefits:

There have been removals and changes in Python 3.12. Testing now helps us make our code compatible and avoid any big surprises (for us and our users) at the big launch in October.
We might find bugs in Python itself! Reporting those will help get them fixed and help everyone.

How #

GitHub Actions: setup-python #

To test the latest alpha, beta or release candidate with actions/setup-python, add 3.12 and allow-prereleases: true to your workflow matrix.

For example:

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]

 steps:
 - uses: actions/checkout@v4

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v4
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true

(We can instead use 3.12-dev and omit allow-prereleases: true, but I find the above a bit neater, and when 3.12.0 final is released in October, it will continue testing with full release versions.)

GitHub Actions: deadsnakes #

For the bleeding edge, we can use deadsnakes/action to test the latest nightly build:

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.8", "3.9", "3.10", "3.11", "3.12-dev"]

 steps:
 - uses: actions/checkout@v3

 - name: Set up Python ${{ matrix.python-version }}
 if: "!endsWith(matrix.python-version, '-dev')"
 uses: actions/setup-python@v4
 with:
 python-version: ${{ matrix.python-version }}

 - uses: deadsnakes/action@v3.0.0
 name: Set up Python ${{ matrix.python-version }} (deadsnakes)
 if: endsWith(matrix.python-version, '-dev')
 with:
 python-version: ${{ matrix.python-version }}

Travis CI #

I recommend moving to another CI.

In the meantime, you can also add 3.12-dev to .travis.yml, although at the time of writing it’s pointing to 3.12.0a3+ from 2022-12-07, which is better than nothing.

language: python
python:
 - "3.8"
 - "3.9"
 - "3.10"
 - "3.11"
 - "3.12-dev"

Again, I recommend moving to another CI.

Other CIs #

Do you use other CIs? Please leave a comment if you know how to test 3.12!

When to support 3.12? #

Now is also a good time to declare support and add the Programming Language :: Python :: 3.12 Trove classifier.

Especially if you have C extensions and other projects depend on yours, a release with wheels will help them test and prepare.

ABI breaks? #

From the announcement:

There will be no ABI changes from this point forward in the 3.12 series. The intent is for the final release of 3.12.0, scheduled for Monday, 2023-10-02, to be identical to this release candidate. This really is the last chance to find critical problems in Python 3.12.

Let’s start testing and releasing for 3.12 now! 🚀

Header photo: The carpet of the Salt Palace Convention Center grand ballroom, host of PyCon 2023, with a couple of googly eyes added to make them Pythony (source)

2023-08-14: Updated for RC1
2023-09-06: Updated for RC2
2023-09-19: Updated for RC3

TIL: how to disable cron for GitHub forks

Sun, 07 May 2023 15:59:57 +0000

GitHub Actions has a useful feature to trigger workflows on a cron schedule. For example:

name: Test

on:
 push:
 pull_request:
 schedule:
 - cron: "0 6 * * *" # daily at 6am

jobs:
 test:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3

But if a contributor has enabled GitHub Actions on their fork (which I recommend: test your contributions before opening a PR), it also runs the cron on their fork. This not only uses up extra CI resources on the fork:

It also sends a regular email to the contributor when the workflow fails:

Instead, we only need to run the cron for the upstream. For example, add:

 jobs:
 test:
+ if: ${{ github.event.repository.fork == false || github.event_name != 'schedule' }}
 runs-on: ubuntu-latest

This only runs the schedule trigger for the (non-fork) upstream, and all other triggers run for both upstream and forks.

Also #

For simpler workflows that only trigger on a cron, such as stalebot or CodeQL runs, we can add a simpler condition.

For example:

name: Close stale issues

on:
 schedule:
 - cron: "10 0 * * *" # daily at 12:10am

jobs:
 stale:
 runs-on: ubuntu-latest

 steps:
 - name: "Check issues"
 uses: actions/stale@v8

We can completely disable it for forks:

 jobs:
 stale:
+ if: github.event.repository.fork == false
 runs-on: ubuntu-latest

PS #

Use single quotes in these lines, double quotes are invalid here (Unexpected symbol).

Thanks #

To Alex Waygood for the tip.

To the British Library and Flickr Commons for the illustration of a chronograph.

Update #

2025-04-28: Instead of hardcoding the upstream repo’s owner with github.repository_owner == 'octocat', use github.event.repository.fork == false. This makes it easier to diff across projects in different organisations and consistent config makes for easier maintenance.

Printable PyCon 2023 schedule

Sat, 15 Apr 2023 10:10:02 +0000

Want to print out the PyCon US 2023 schedule? Paper doesn’t run out of batteries, is easy to scribble on, and stuff into a pocket.

Here’s some custom CSS and JavaScript to make it nicely printable.

Install the Styler browser extension
View a PyCon schedule page such as https://us.pycon.org/2023/schedule/talks/ and click the Styler extension’s S icon
Paste this CSS into the upper box:

body.pycon-schedule aside.sidebar,
body.pycon-schedule div.badges,
body.pycon-schedule button.menu-button.open-menu,
body.pycon-schedule button#theme-switch,
body.pycon-schedule footer {
 display: none;
}
body.pycon-schedule {
 font-size: 14px;
}
body.pycon-schedule main {
 padding: 0 !important;
}
body.pycon-schedule .calendar {
	left: auto !important;
 width: 100% !important;
}
@media print {
 a:after { content:''; }
 a[href]:after { content: none !important; }
}

Paste this JavaScript into the lower box:

$(document).ready(function () {
 if (
 window.location.pathname.match("2023/schedule/") &&
 !window.location.pathname.match("2023/schedule/presentation/")
 ) {
 $("body").addClass("pycon-schedule");
 }
});

Print!

It’ll run on any of the https://us.pycon.org/2023/schedule/* pages, but not the individual presentation pages such as https://us.pycon.org/2023/schedule/presentation/88/

Once printed, you can click the Styler icon and the x button to disable Styler for the site so you can browse the original version.

Bits of Pluto on Mastodon

Sat, 18 Feb 2023 13:16:49 +0000

What I built #

I built a Mastodon bot that posts a a different bit of Pluto every six hours.

Category Submission #

Wacky Wildcard

App Link #

https://botsin.space/@bitsofpluto

Screenshots #

https://botsin.space/@bitsofpluto/109854013035140138

https://botsin.space/@bitsofpluto/109878078918934666

https://botsin.space/@bitsofpluto/109882325572738354

https://botsin.space/@bitsofpluto/109872416513104327

https://botsin.space/@bitsofpluto/109869584884331747

https://botsin.space/@bitsofpluto/109835610772369900

https://botsin.space/@bitsofpluto/109828532816033685

https://botsin.space/@bitsofpluto/109845519798723221

Description #

Bits of Pluto posts a different bit of Pluto to Mastodon every six hours. Each is a crop from an image by NASA’s New Horizons spacecraft.

Link to Source Code #

https://github.com/hugovk/bitsofpluto

Permissive License #

MIT

Background #

For decades, Pluto had only been a speck of light until the NASA’s Hubble Space Telescope captured it in never-before-seen detail:

Just a few years later, I followed the journey of NASA’s New Horizons spacecraft in awe as it flew past Pluto, in the far depths of our solar system.

Like many others I was especially struck by the beautiful photographs it shot, especially this large, detailed image:

As NASA describe it:

NASA’s New Horizons spacecraft captured this high-resolution enhanced color view of Pluto on July 14, 2015. The image combines blue, red and infrared images taken by the Ralph/Multispectral Visual Imaging Camera (MVIC). Pluto’s surface sports a remarkable range of subtle colors, enhanced in this view to a rainbow of pale blues, yellows, oranges, and deep reds. Many landforms have their own distinct colors, telling a complex geological and climatological story that scientists have only just begun to decode. The image resolves details and colors on scales as small as 0.8 miles (1.3 kilometers). The viewer is encouraged to zoom in on the full resolution image on a larger screen to fully appreciate the complexity of Pluto’s surface features.

I second the recommendation to zoom in on the full resolution! The detail and is astounding, especially compared to the Hubble image.

I thought it would be fascinating to chop this up and put it in a social media feed to enjoy different aspects of the detail, to punctuate the doomscrolling with a moment of wonder.

How I built it #

I built the bot in Python. Originally it posted to Twitter, but as the future of bots on Twitter is less than certain, the time was ready to port it over to Mastodon where bots are welcome.

On the way, I learned how to calculate brightness in images, how to use the Mastodon API, and how to set up a bot on Linode and (the appropriately-named!) botsin.space.

The bot does two things:

Get a bit of Pluto
Post to Mastodon

1. Get a bit of Pluto #

The bitsofpluto() function takes a single parameter, the path to the full resolution Pluto image.

The image is opened using Pillow.

We then choose a random width for the image, somewhere between 800 and 2000 pixels, with height is set to ¾ the width. This is to give a different zoom level each time. We then randomly select a window to crop. This is our potential bit of Pluto.

But we don’t stop there. The image has quite a large background of empty black (or nearly black) space. We don’t want to post that. So we sample 9 points from the image (corners, centre point, and bisecting points) and measure the brightness where 0 is black and 255 is white. If brightness is under 10, we count it as a dark point. If there are 6 or less, we save the image to a temporary directory.

Otherwise if there are too many dark points, we discard this crop, and start again from the top. I came up with these figures of 10 and 7 through trial-and-error. It’s quite nice to sometimes get an image with a corner of Pluto’s curvature against the darkness of space:

2. Post to Mastodon #

We use the Mastodon.py library to via the Mastodon API.

The toot_it() function first reads in the credentials for posting. I generated them using a helper script I wrote based upon Allison Parrish’s instructions.

Once authenticated, there’s a two-step process for posting:

Upload the image using api.media_post(), which returns a reference to the media
Create the Mastodon post, using the media reference

Hosting #

Linode #

The bot code is hosted on Linode. The nice thing about Mastodon bots is that they rarely need expensive or high-end hosting. I created a Linode using the cheapest option: for $5/month the “Nanode 1 GB” plan gives 1 GB RAM, 1 CPU and 25 GB storage, more than enough for our needs.

I chose a Ubuntu 22.04 LTS image: LTS means “long-term support”, it will be supported for 5 years, until April 2027. And I chose the Frankfurt, DE region because it’s closest to me.

After creating a user, I logged in via SSH, and cloned the bitsofpluto repo.

I ran crontab -e to schedule it to run once every six hours:

0 */6 * * * /home/botuser/bin/scheduled/0000-06-repeat.sh > /tmp/logs/0000-06-repeat.log

Where 0000-06-repeat.sh contains:

#!/bin/bash
#set -e

mkdir -p /tmp/logs/

~/github/bitsofpluto/crontask.sh > /tmp/logs/bitsofpluto.log 2>&1

Which, when the cron triggers, switches to the repo, fetches any recent changes, and then runs the bot to make a post:

python3 bitsofpluto.py --yaml ~/bin/data/bitsofpluto.yaml --no-web

botsin.space #

The bot’s Mastodon account is on Colin Mitchell’s botsin.space instance, home to many other excellent bots.

Additional Resources/Info #

I’ve made two other Mastodon bots:

@tiny_bus_stop - uses Tracery and Cheap Bots, Toot Sweet!
@FlagFacts - uses Python

Enjoy!

How to search 5,000 Python projects

Fri, 09 Dec 2022 12:44:21 +0000

Why? #

Often Python core developers think about deprecating and removing old bits of the language. But first it’s a good idea to get an idea of how much the old bits are used. Searching the 5,000 most-popular projects on PyPI is a helpful proxy to gauge community use.

How? #

Core developer Victor Stinner has written a couple of useful scripts that live in his misc repo.

Setup #

First, clone the repo somewhere, doesn’t matter where:

mkdir -p ~/github
cd ~/github/
git clone https://github.com/vstinner/misc

# Optionally for some colour in the logs:
python3 -m pip install termcolor

Then download 5,000 sdists! Again, doesn’t matter where:

$ mkdir -p ~/top-pypi/output
$ cd ~/top-pypi/output/
$ python3 ~/github/misc/cpython/download_pypi_top.py .
Download JSON from: https://hugovk.github.io/top-pypi-packages/top-pypi-packages-30-days.min.json
Project#: 5000
[1/5000] Saving to ./boto3-1.26.25.tar.gz (101.7 kB)
[2/5000] Saving to ./botocore-1.29.25.tar.gz (10426.8 kB)
[3/5000] Saving to ./urllib3-1.26.13.tar.gz (293.4 kB)
...
[4990/5000] Saving to ./meld3-2.0.1.tar.gz (35.3 kB)
[4991/5000] Saving to ./browserstack-local-1.2.4.tar.gz (6.6 kB)
Cannot find URL for project: allensdk
Cannot find URL for project: dataframe-image
[4994/5000] Saving to ./SQLAlchemy-Continuum-1.3.13.tar.gz (79.2 kB)
[4995/5000] Saving to ./sarge-0.1.7.post1.tar.gz (25.1 kB)
[4996/5000] Saving to ./confusable_homoglyphs-3.2.0.tar.gz (158.1 kB)
[4997/5000] Saving to ./YTThumb-1.4.5.tar.gz (3.0 kB)
[4998/5000] Saving to ./azure-ai-ml-1.2.0.zip (4486.3 kB)
[4999/5000] Saving to ./pythena-1.6.0.tar.gz (5.2 kB)
[5000/5000] Saving to ./interchange-2021.0.4.tar.gz (25.2 kB)
Downloaded 5000 projects in 1602.4 seconds

With colour:

⏳ This will take a bit of time. Some projects don’t have sdists, nothing to worry about, we will still end up with a good number. At the time of writing (2022-12-09), it took me just under 27 minutes to download 4,748 files, taking up 5.37 GB of space.

If you want to download fewer, specify how many:

python3 ~/github/misc/cpython/download_pypi_top.py . 100

Search #

Next, we can search all the sdists using another script. And we don’t need to extract them!

For example, configparser’s LegacyInterpolation was deprecated in Python 3.2 (released February 2011), but only in docs and without raising a DeprecationWarning.

How much is it used in the top 5k?

$ cd ~/top-pypi/output/
$ python3 ~/github/misc/cpython/search_pypi_top.py -q . "LegacyInterpolation"
./hexbytes-0.3.0.tar.gz: hexbytes-0.3.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./hexbytes-0.3.0.tar.gz: hexbytes-0.3.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./hexbytes-0.3.0.tar.gz: hexbytes-0.3.0/.tox/py39-lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./hexbytes-0.3.0.tar.gz: hexbytes-0.3.0/.tox/py39-lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./jedi-0.18.2.tar.gz: jedi-0.18.2/jedi/third_party/typeshed/stdlib/3/configparser.pyi: class LegacyInterpolation(Interpolation): ...
./mypy-0.991.tar.gz: mypy-0.991/mypy/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./mypy-0.991.tar.gz: mypy-0.991/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./eth-hash-0.5.1.tar.gz: eth-hash-0.5.1/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./eth-hash-0.5.1.tar.gz: eth-hash-0.5.1/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./eth-account-0.7.0.tar.gz: eth-account-0.7.0/.tox/lint/lib/python3.10/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./eth-account-0.7.0.tar.gz: eth-account-0.7.0/.tox/py310-lint/lib/python3.10/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./eth-utils-2.1.0.tar.gz: eth-utils-2.1.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./pytype-2022.11.29.tar.gz: pytype-2022.11.29/pytype/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./pytype-2022.11.29.tar.gz: pytype-2022.11.29/pytype/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./pylint-2.15.8.tar.gz: pylint-2.15.8/pylint/checkers/stdlib.py: "LegacyInterpolation",
./pyre-check-0.9.17.tar.gz: pyre-check-0.9.17/typeshed/stdlib/configparser.pyi: "LegacyInterpolation",
./pyre-check-0.9.17.tar.gz: pyre-check-0.9.17/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/backports/configparser/__init__.py: "LegacyInterpolation",
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/backports/configparser/__init__.py: class LegacyInterpolation(Interpolation):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/backports/configparser/__init__.py: "LegacyInterpolation has been deprecated since Python 3.2 "
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/configparser.py: LegacyInterpolation,
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/configparser.py: "LegacyInterpolation",
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: elif isinstance(self.interpolation, configparser.LegacyInterpolation):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: elif isinstance(self.interpolation, configparser.LegacyInterpolation):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: elif isinstance(self.interpolation, configparser.LegacyInterpolation):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: class ConfigParserTestCaseLegacyInterpolation(ConfigParserTestCase):
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: interpolation = configparser.LegacyInterpolation()
./configparser-5.3.0.tar.gz: configparser-5.3.0/src/test_configparser.py: configparser.LegacyInterpolation()
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/.tox/lint/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/configparser.pyi: class LegacyInterpolation(Interpolation): ...
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/configparser.pyi: class LegacyInterpolation(Interpolation): ...
./eth-rlp-0.3.0.tar.gz: eth-rlp-0.3.0/venv-erlp/lib/python3.9/site-packages/mypy/typeshed/stdlib/3/configparser.pyi: class LegacyInterpolation(Interpolation): ...
./eth_abi-3.0.1.tar.gz: eth_abi-3.0.1/.tox/lint/lib/python3.10/site-packages/mypy/typeshed/stdlib/configparser.pyi: class LegacyInterpolation(Interpolation):

Time: 0:00:17.957695
Found 32 matching lines in 12 projects

With colour:

Answer: very little, mostly backports and type stubs. This told us it’s a good candidate for removal, so a proper DeprecationWarning was added in Python 3.11 (released October 2022) and it will be removed in Python 3.13 (October 2024).

Happy searching! 🔎

Header photo: “The card index department” by Boston Public Library is licensed under CC BY 2.0.

Help test Python 3.11 beta!

Tue, 31 May 2022 17:10:00 +0000

Calling all Python library maintainers! 🐍

Python 3.11 is in beta! 🎉

PEP 664 defines the release schedule for Python 3.11.0:

The second beta came out on 31st May 2022
The first release candidate is set for 1st August 2022
And the full release is set for 3rd October 2022

In his announcement, Pablo Galindo Salgado, release manager for Python 3.10 and 3.11, said:

We strongly encourage maintainers of third-party Python projects to test with 3.11 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Monday, 2021-08-02). Our goal is have no ABI changes after beta 4 and as few code changes as possible after 3.11.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.11 as possible during the beta phase.

Test with 3.11 #

It’s now time for library maintainers to start testing 3.11 with your project. You don’t need to declare support and release for 3.11 yet, but there’s two big benefits to testing with 3.11 on CI:

There have been removals and changes in Python 3.11. Testing now will help you make your code compatible and avoid any big surprises (for you and your users) at the big launch in October.
You might find bugs in Python itself! Reporting those will help get them fixed and help everyone.

How #

GitHub Actions: setup-python #

To test the latest alpha, beta or release candidate with actions/setup-python, add 3.11-dev to your workflow matrix.

For example:

jobs:
 build:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.7", "3.8", "3.9", "3.10", "3.11-dev"]

 steps:
 - uses: actions/checkout@v3

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@v4
 with:
 python-version: ${{ matrix.python-version }}

GitHub Actions: deadsnakes #

For the bleeding edge, we can use deadsnakes/action to test the latest nightly build:

jobs:
 build:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.7", "3.8", "3.9", "3.10", "3.11-dev"]

 steps:
 - uses: actions/checkout@v3

 - name: Set up Python ${{ matrix.python-version }}
 if: "!endsWith(matrix.python-version, '-dev')"
 uses: actions/setup-python@v3
 with:
 python-version: ${{ matrix.python-version }}

 - uses: deadsnakes/action@v2.1.1
 name: Set up Python ${{ matrix.python-version }} (deadsnakes)
 if: endsWith(matrix.python-version, '-dev')
 with:
 python-version: ${{ matrix.python-version }}

Travis CI #

I recommend moving to another CI.

In the meantime, you can also add 3.11-dev to .travis.yml, although at the time of writing it’s pointing to 3.11.0a3 from 2021-12-08, which is better than nothing.

language: python
python:
 - "3.7"
 - "3.8"
 - "3.9"
 - "3.10"
 - "3.11-dev"

Again, I recommend moving to another CI.

Other CIs #

Do you use other CIs? Please leave a comment if you know how to test 3.11!

When to support 3.11? #

When should you declare support and add the Programming Language :: Python :: 3.11 Trove classifier?

First of all, make sure your tests pass on 3.11 beta. One option is waiting until 3.11.0 final is released.

Or, as mentioned above:

Our goal is have no ABI changes after beta 4 and as few code changes as possible after 3.11.0rc1, the first release candidate.

If you have a pure Python project, you could release now.

If you have C extensions, you might want to wait until the release candidate phase, although if other projects depend on yours, a preview release would help them test and prepare.

In any case, start testing 3.11 now!

Header photo: Uppland Runic Inscription 53, a 1,000 year old runestone in the old town of Stockholm (source)

Top PyPI Packages

Sun, 29 May 2022 15:53:36 +0000

Top PyPI Packages is a website that creates a monthly dump of the 5,000 most-downloaded packages from the Python Package Index (PyPI). It provides a human-readable list and a machine-readable JSON file for programmatic use.

How it’s used #

The generated data is important for the Python community: it has been cited by many academic papers, covering research on topics such as software supply-chain attacks, genetic algorithms, neural type hints and technical debt.

Websites also use the data for analysing things like community adoption of Python 3 and of wheels package files, and for automated dependency updates.

Some are listed here.

Last but not least, CPython core developers often use the data for community analysis, for example to check how widespread certain language features are when considering deprecations and removals, or API changes of Python itself.

Some examples:

https://mail.python.org/archives/search?q=top-pypi-packages

How it works #

It runs on the cheapest DigitalOcean Droplet to query a Google BigQuery dataset from the Python Software Foundation, and builds the JSON file from that. The data is automatically committed from the droplet back to the GitHub repository, and GitHub Actions is used to automatically tag and create a release, which then creates a Digital Object Identifier (DOI) at Zenodo to help make it citable for researchers.

In more detail #

A “Droplet” is the name DigitalOcean uses for virtual machines, which is basically a Linux server. I’m using the cheapest $5/month running Ubuntu 20.04 (LTS) plus $1/month for automated backups. (In July 2022, I’ll switch to the new $4/month Droplet.)

On the first of the month, it runs a cron job:

13 11 1 * * ( eval "$(ssh-agent -s)"; ssh-add ~/.ssh/id_rsa-top-pypi-packages; /home/botuser/github/top-pypi-packages/top-pypi-packages.sh ) > /tmp/top-pypi-packages.log 2>&1

This script runs some others to call pypinfo:

/home/botuser/.local/bin/pypinfo --json --indent 0 --limit 5000 --days 30 "" project > top-pypi-packages-30-days.json

PyPI streams data about downloads to Google BigQuery which are accessible as a public dataset. Google provides a free amount of queries per month, and I’ve been adjusting the amount of data fetched to stay within the free quota (e.g. changing from the top 5k packages to 4k; it used to get data for top packages over the past 30 days and 365 days, but now only for 30 days; and bumping back up to 5k packages).

pypinfo is a handy command-line interface (CLI) to access this BigQuery data and dump it to a JSON file.

Another handy CLI tool called jq minifies the JSON data:

jq -c . < top-pypi-packages-30-days.json > top-pypi-packages-30-days.min.json

These are then committed back to the repo, tagged using CalVer (e.g. 2022-05) and pushed. GitHub Actions creates a release from the tag.

This then creates a Digital Object Identifier (DOI) at Zenodo to help make it citable for researchers.

The website, on GitHub Pages, reads in the generated JSON file and shows the top 100 (or 1,000 or 5,000) packages in human-readable form. It’s based on Python Wheels, which in nice circular fashion, uses the JSON data from this project.

Thanks #

Thanks to PyPI and Google BigQuery for the data; pypinfo and jq for the tools; Python Wheels for making their code open source; and DigitalOcean for sponsoring this project’s hosting. Visit https://do.co/oss-sponsorship to see if your project is eligible.

Header photo: “PACKAGES” by marc falardeau is licensed under CC BY 2.0.

The Python 3.1 problem

Wed, 19 May 2021 14:08:31 +0000

Or, a variation on the Norway problem #

Short version: put quotes around version numbers in YAML.

The Norway problem #

The Norway problem is when you put this in YAML:

countries:
 - GB
 - IE
 - FR
 - DE
 - NO

But get this out:

>>> import yaml
>>> with open("countries.yml") as f:
... yaml.safe_load(f)
...
{'countries': ['GB', 'IE', 'FR', 'DE', False]}

:scream:

The Norway fix #

Use quotes:

countries:
 - "GB"
 - "IE"
 - "FR"
 - "DE"
 - "NO"

>>> with open("countries.yml") as f:
... yaml.safe_load(f)
...
{'countries': ['GB', 'IE', 'FR', 'DE', 'NO']}

🇳🇴 ✅

The Python 3.1 problem #

A similar problem will affect the Python community in October 2021, when Python 3.10 comes out.

Why?

When 3.10 is added to YAML, for example in CI test matrix config, it’s interpreted as a float. This:

python-version: [3.6, 3.7, 3.8, 3.9, 3.10, pypy3]

Turns into this:

>>> import yaml
>>> with open("versions.yml") as f:
... yaml.safe_load(f)
...
{'python-version': [3.6, 3.7, 3.8, 3.9, 3.1, 'pypy3']}

CI failed! It’s not 2009! Python 3.1 not found!

:scream:

Relatedly, 3.10-dev without quotes works because it’s interpreted as a string. But when deleting -dev, 3.10 is interpreted as a float.

The Python 3.10 fix #

Version numbers are strings, not floats. Use quotes:

python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "pypy3"]

>>> import yaml
>>> with open("versions.yml") as f:
... yaml.safe_load(f)
...
{'python-version': ['3.6', '3.7', '3.8', '3.9', '3.10', 'pypy3']}

🐍 ✅

Python version share over time, 6

Wed, 01 Jan 2020 16:36:25 +0000

January 2016 — December 2019 #

To celebrate the end of life of Python 2 on 1st January 2020, here’s some statistics showing how much different Python versions have been used over four years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and December 2019:

pip #

The package installer

six #

Python 2 and 3 compatibility library

NumPy #

Scientific computing library

pytest #

Testing framework

pandas #

Data analysis toolkit

Coverage.py #

Code coverage testing

Pillow #

Imaging library

Django #

Web framework

Matplotlib #

2D plotting library

Flake8 #

Linter

Pylint #

Linter

TensorFlow #

Machine learning library

pylast #

Interface to Last.fm

How #

Statistics were collected using pypi-trends.py, a wrapper around pypinfo and pypistats to fetch all monthly downloads from the PyPI database on Google BigQuery and save them as JSON files. Data was downloaded over several days as getting all months uses up a lot of free BigQuery quota. Then jsons2csv.py plots a chart using matplotlib. Raw JSON data is in the repo.

Python version share over time, 5

Mon, 04 Nov 2019 14:40:48 +0000

January 2016 — October 2019 #

To celebrate the release of Python 3.8.0 on 14th October 2019, and with less than two months left for Python 2, here’s some statistics showing how much different Python versions have been used over nearly four years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and October 2019:

pip #

The package installer

six #

Python 2 and 3 compatibility library

NumPy #

Scientific computing library

pytest #

Testing framework

pandas #

Data analysis toolkit

Coverage.py #

Code coverage testing

Pillow #

Imaging library

Django #

Web framework

Matplotlib #

2D plotting library

Flake8 #

Linter

Pylint #

Linter

pylast #

Interface to Last.fm

How #

Python version share over time, 4

Thu, 04 Apr 2019 14:58:18 +0000

January 2016 — March 2019 #

To celebrate the release of Python 3.7.3 on 25th March 2019, and with under nine months left for Python 2, here’s some statistics showing how much different Python versions have been used over the past three years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and March 2019:

pip #

The package installer

six #

Python 2 and 3 compatibility library

NumPy #

Scientific computing library

pytest #

Testing framework

Coverage.py #

Code coverage testing

Pillow #

Imaging library

Django #

Web framework

Matplotlib #

2D plotting library

Flake8 #

Linter

Pylint #

Linter

pylast #

Interface to Last.fm

How #

Statistics were collected using pypi-trends.py, a wrapper around pypinfo to fetch all monthly downloads from the PyPI database on Google BigQuery and save them as JSON files. Data was downloaded over several days as getting all months uses up a lot of free BigQuery quota. Then jsons2csv.py plots a chart using matplotlib. Raw JSON data is in the repo.

Python version share over time, 3

Tue, 01 Jan 2019 14:40:48 +0000

January 2016 — December 2018 #

To celebrate the release of Python 3.7.2 on Christmas Eve 2018, and with under a year left for Python 2, here’s some statistics showing how much different Python versions have been used over the past three years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and December 2018:

For the NumPy scientific computing library:

For the pytest testing framework:

For the Pillow imaging library:

For the Django web framework:

For the matplotlib 2D plotting library:

For the Pylint linter:

And for the pylast interface to Last.fm:

How #

Statistics were collected using pypi-trends.py a wrapper around pypinfo to fetch all monthly downloads from the PyPI database on Google BigQuery and save them as JSON files. Data was downloaded over several days as getting all months uses up a lot of free BigQuery quota. Then jsons2csv.py plots a chart using matplotlib. Raw JSON data is in the repo.

Words of the year from Twitter, 2018

Mon, 31 Dec 2018 14:03:57 +0000

Three bots have been collecting words from Twitter for the past year.

Short version:

In 2018, people on Twitter talked about the words stan, toxic, peoplekind, councel, beclowned, caucasity, kakistocracy, catastrofuck, shithole, bombogenesis, chucklefuck, trumpfuckery, clackwanker, wankpuffin, dipshittery, fucksicle, and fuckwangled.

Long version:

They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favouriteword"

Combining and comparing the logs of all three, let's see the top words of 2018. Bold means a word wasn't in that chart in 2017.

Combined output

Total in 2018: 81,695

justice (980)
irregardless (876)
soon (696)
no (684)
love (636)
mines (571)
homophobia (506)
moist (491)
bae (434)
clout (365)
cunt (364)
alot (354)
nut (333)
ain't (329)
cock (324)
forever (305)
hurted (291)
yolo (280)
sexy (271)
impossible (269)

lovihatibot	nixibot	favibot
2018 total: 41,092	2018 total: 33,078	2018 total: 7,525
soon (688) no (528) homophobia (494) moist (483) bae (420) clout (356) nut (333) cunt (332) cock (319) yolo (277) sexy (240) bitch (227) sorry (210) queer (205) stan (197) coochie (190) fuck (186) fate (182) diet (167) toxic (159)	justice (975) irregardless (869) mines (565) love (501) alot (352) ain't (324) hurted (289) forever (259) impossible (249) conversate (230) loyalty (224) anyways (209) finna (154) stupider (153) t (149) usa (125) marriage (111) peoplekind (105) councel (105) funner (104)	no (68) beclowned (59) caucasity (35) whore (32) kakistocracy (30) feckless (25) cockwomble (24) yikes (22) bitch (20) shithole (19) catastrofuck (17) kissy (16) bombogenesis (15) oof (15) cunt (15) chucklefuck (14) trumpfuckery (14) wankpuffin (12) dipshittery (11) fuck (11)

Here's top 10 charts for each phrase from each bot.

@lovihatbot

I love the word X	I hate the word X
cunt (245) fuck (173) fate (167) bitch (147) play (145) moist (99) choice (68) whore (55) cock (47) queer (47)	soon (648) homophobia (492) no (489) bae (417) moist (384) clout (351) nut (330) yolo (277) cock (272) sexy (234)

@nixibot

X is not a word	X isn't a word	X ain't a word
irregardless (738) mines (535) love (437) forever (259) hurted (255) impossible (241) alot (241) conversate (196) loyalty (162) anyways (156)	justice (964) irregardless (125) ain't (112) alot (109) stupider (59) anyways (51) just (51) happeh (49) love (49) usa (46)	ain't (150) loyalty (23) funk (19) love (15) unlonely (15) is (13) ignorantest (8) finna (6) irregardless (6) shit (5)

@favibot

X is my new favorite word	X is my new favourite word
no (59) beclowned (54) caucasity (27) kakistocracy (26) whore (25) feckless (21) yikes (20) shithole (18) cockwomble (15) bombogenesis (13)	catastrofuck (14) cockwomble (8) caucasity (8) kissy (8) clackwanker (7) no (7) wankpuffin (7) fucksicle (7) fuckwangled (7) bitch (6)

Previous years

See the words from 2017, 2016, 2015, 2014 and 2013.

2019

For words causing a reaction in 2019, follow @lovihatibot, @nixibot and @favibot on Twitter.

Python version share over time, 2

Sat, 03 Nov 2018 14:40:48 +0000

January 2016 — October 2018 #

To celebrate the release of Python 3.7.1 on 20th October 2018, here’s some statistics showing how much different Python versions have been used over the past two and five-sixths years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and October 2018:

pip installs from PyPI over time, by Python version

For the NumPy scientific computing library:

Numpy's pip installs from PyPI over time, by Python version

For the Pillow imaging library:

Pillow's pip installs from PyPI over time, by Python version

For the Django web framework:

pylast's pip installs from PyPI over time, by Python version

And for the pylast interface to Last.fm:

Pillow's pip installs from PyPI over time, by Python version

How #

Python version share over time, 1

Thu, 05 Jul 2018 14:40:48 +0000

January 2016 — June 2018 #

To celebrate the release of Python 3.7 on 27th June 2018, here’s some statistics showing how much different Python versions have been used over the past two and a half years.

Here’s the pip installs for all packages from the Python Package Index (PyPI), between January 2016 and June 2018:

pip installs from PyPI over time, by Python version

And for the Pillow imaging library:

Pillow's pip installs from PyPI over time, by Python version

How #

Statistics were collected using pypi-trends.py, a wrapper around pypinfo to fetch all monthly downloads from the PyPI database on Google BigQuery and save them as JSON files. Data was downloaded over three or four days as getting all months uses up a lot of free BigQuery quota. Then jsons2csv.py converts them into a single CSV file for chart-wrangling in Excel. Raw CSV and JSON data is in the repo.

Words of the year from Twitter, 2017

Sat, 30 Dec 2017 20:00:10 +0000

Three bots have been collecting words from Twitter for the past year.

Short version:

In 2017, people on Twitter talked about the words broflake, caucacity, cockwomble, covfefe, dotard, douchecanoe, dracarys, shitgibbon, shooketh, twatwaffle, and woke.

Long version:

They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favouriteword"

Combining and comparing the logs of all three, let's see the top words of 2017. Bold means a word wasn't in that chart in 2016.

Combined output

Total in 2017: 120,412

forever (2,828)
mines (1,687)
no (1,161)
love (1,092)
justice (1,072)
ain't (1,038)
irregardless (1,027)
bae (1,002)
moist (891)
homophobia (856)
soon (811)
loyalty (594)
conversate (593)
finna (566)
bigly (551)
impossible (526)
cunt (500)
alot (495)
sorry (449)
nut (441)

lovihatibot	nixibot	favibot
2017 total: 55,063	2017 total: 50,050	2017 total: 9,314
bae (950) no (941) moist (880) homophobia (854) soon (791) cunt (441) nut (435) lit (398) sexy (392) sorry (377) bitch (345) cock (339) yolo (298) panties (292) shook (267) woke (262) love (249) clout (248) daddy (242) hate (239)	forever (2,699) mines (1,666) justice (1,072) ain't (1,017) irregardless (1,014) love (841) loyalty (587) conversate (586) bigly (522) impossible (501) alot (493) finna (463) anyways (385) hurted (271) stupider (251) covfefe (246) funner (245) marriage (203) worser (202) stupidest (183)	mania (255) dotard (133) covfefe (102) no (95) yikes (48) cockwomble (35) shooketh (34) cunt (33) serendipity (30) caucacity (25) bitch (24) shitgibbon (23) dracarys (22) broflake (20) wow (19) twatwaffle (19) totality (17) headass (17) thrussy (16) douchecanoe (16)

Here's top 10 charts for each phrase from each bot.

@lovihatbot

I love the word X	I hate the word X
cunt (293) fuck (218) fate (197) bitch (178) moist (143) play (128) spooky (71) love (66) choice (62) free (60)	bae (932) no (893) homophobia (851) soon (770) moist (737) nut (431) sexy (380) lit (379) sorry (370) cock (305)

@nixibot

X is not a word	X isn't a word	X ain't a word
forever (2,683) mines (1,153) irregardless (765) love (598) impossible (454) loyalty (398) conversate (392) bigly (347) alot (307) finna (288)	justice (1,057) mines (466) ain't (412) irregardless (242) love (223) alot (185) conversate (184) anyways (176) bigly (171) cant (149)	ain't (390) finna (58) loyalty (53) mines (47) ignorantest (26) is (22) love (20) shit (17) just (15) worser (11)

@favibot

X is my new favorite word	X is my new favourite word
mania (255) dotard (110) no (87) covfefe (80) yikes (40) cunt (25) serendipity (23) shooketh (22) caucacity (22) cockwomble (20)	dotard (20) covfefe (18) cockwomble (15) shooketh (11) no (7) nonce (7) muggy (6) wow (6) serendipity (6) yikes (5)

Previous years

See the words from 2016, 2015, 2014 and 2013,

2018

For words causing a reaction in 2018, follow @lovihatibot, @nixibot and @favibot on Twitter.

Words of the year from Twitter, 2016

Thu, 05 Jan 2017 16:41:29 +0000

Three bots have been collecting words from Twitter for the past year.

They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favourite/fave word"

Combining and comparing the logs of all three, let's see the top words of 2016. Bold means a word wasn't in that chart in 2015.

Combined output

Total in 2016: 168,241

mines (4,060)
bae (3,369)
no (2,582)
forever (2,143)
lit (2,005)
moist (1,805)
love (1,790)
bigly (1,488)
conversate (1,358)
ain't (1,339)
irregardless (1,204)
homophobia (1,186)
justice (1,138)
loyalty (985)
soon (886)
cunt (869)
impossible (787)
sorry (781)
family (766)
finna (674)

lovihatibot	nixibot	favibot
2016 total: 85,828	2016 total: 71,609	2016 total: 10,807
bae (3,242) no (2,209) lit (1,934) moist (1,788) homophobia (1,182) soon (851) cunt (798) sorry (692) panties (575) sexy (524) bitch (516) nut (509) slay (496) daddy (495) cock (462) baby (432) babe (430) pussy (425) hate (415) love (402)	mines (4,035) forever (2,003) love (1,387) conversate (1,345) bigly (1,341) ain't (1,296) irregardless (1,186) justice (1,134) loyalty (961) impossible (752) family (732) anyways (547) finna (526) 4ever (508) alot (448) funner (445) worser (421) stupider (367) hurted (319) marriage (301)	no (109) bigly (102) yikes (91) shook (42) cunt (41) braggadocious (37) shenanigans (34) bragadocious (32) wow (31) malarkey (30) bitch (30) headass (29) lit (28) savage (26) tragic (25) bougie (25) petty (22) cockwomble (22) finesse (21) soon (20)

Here's top 10 charts for each phrase from each bot.

@lovihatbot

I love the word X	I hate the word X
cunt (475) delicious (315) fuck (307) moist (242) fate (219) bitch (216) play (127) baby (114) babe (109) lit (94)	bae (3,208) no (2,135) lit (1,840) moist (1,546) homophobia (1,181) soon (808) sorry (689) panties (551) sexy (507) nut (494)

@nixibot

X is not a word	X isn't a word	X ain't a word
mines (2,679) forever (1,981) love (943) irregardless (890) conversate (880) bigly (837) impossible (678) loyalty (636) 4ever (507) finna (325)	mines (1,283) justice (1,124) family (721) ain't (528) bigly (473) conversate (441) love (391) irregardless (281) anyways (270) stupider (259)	ain't (509) loyalty (121) mines (73) finna (63) ignorantest (56) love (53) impossible (32) bigly (31) is (28) conversate (24)

@favibot

X is my new favorite word	X is my new favourite word	X is my new fave word
no (98) bigly (93) yikes (65) shook (32) cunt (29) braggadocious (28) bragadocious (28) shenanigans (27) wow (27) malarkey (26)	cockwomble (19) yikes (16) dampfnudel (11) cunt (10) savage (10) no (9) bigly (9) wankpuffin (8) braggadocious (7) shenanigans (7)	yikes (10) shook (5) tragic (5) soon (4) nunty (4) yo (3) bitch (3) wow (3) malarkey (3) ganern (2)

Changes from past years

Let's see how mentions have changed for some previous years' words.

People talking about words ‒ especially new favourite words, or loved words, or hated words, or words they've decided don't exist ‒ may be a good indicator of brand new words or at least words with new currency. Both increases and decreases may reflect an underlying change in use, and decreases may reflect an acceptance of the words.

The words from 2013 and 2014 have all more or less tailed off.

From 2015, lit is still being talked about. It's commonly used as "lit af", or "lit as fuck", for example in this tweet:

My Year is starting off lit af👌🏼 ...but is gonna be TD by Monday morning

Something, I'm not sure what, happened in June 2015 that caused its use to explode on Twitter:

Fam (from family, for those closest to you but not necessarily family), has been around a long time, peaked in January 2015 and has tailed off.

Similarly, slay ("killed it. succeeded in something amazing", "something you tell someone when they look sexy as f***") has been around for a while and was talked about in 2016 as well.

New in 2016

2016's words have been dominated by those from the US presidential elections, such as braggadocious (and bragadocious, "boastful or arrogant"), shenanigans ("secret or dishonest activity or maneuvering") and malarkey ("meaningless talk; nonsense"), but most notably by bigly (or "big league"):

Another with a very recent peak is headass:

2017

For words causing a reaction in 2017, follow @lovihatibot, @nixibot and @favibot on Twitter.

#woty15 nominations

Fri, 08 Jan 2016 21:34:16 +0000

Here’s some quick charts of some of the words nominated for American Dialect Society 2015 Words of the Year PDF), data taken from a corpus of words talked about on Twitter (read more about them here), as collected by a trio of bots. Notable troughs are generally when the bots were offline.

squad

fuckboy

lit

fleek

From my 2015 nominations: #

Lit, fleek: see above.

fam

slay

And for my 2013 nominations see here.

Words of the year from Twitter, 2015

Sat, 02 Jan 2016 11:46:01 +0000

Three bots have been collecting words from Twitter for the past year.

They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favourite/fave word"

Combining and comparing the logs of all three, let's see the top words of 2015. Bold means a word wasn't in that chart in 2014.

Combined output

2015 total: 219,918

bae (9,513)
mines (4,282)
no (4,239)
love (2,353)
moist (2,345)
lit (2,184)
loyalty (1,984)
ain't (1,725)
conversate (1,664)
fleek (1,525)
justice (1,470)
soon (1,420)
impossible (1,306)
homophobia (1,276)
cunt (1,258)
year (1,219)
forever (1,214)
fam (1,149)
sorry (1,114)
panties (1,060)

lovihatibot	nixibot	favibot
2015 total: 125,867	2015 total: 81,960	2015 total: 13,976
bae (9,151) no (4,004) moist (2,341) lit (2,107) soon (1,409) homophobia (1,278) fleek (1,274) cunt (1,182) fam (1,077) panties (1,069) sorry (1,014) fuckboy (941) bitch (880) sexy (860) babe (815) slay (771) thot (758) pussy (706) fuck (698) love (672)	mines (4,299) loyalty (2,000) love (1,702) ain't (1,678) conversate (1,661) justice (1,472) impossible (1,283) year (1,226) irregardless (1,027) forever (953) funner (717) worser (641) bestest (517) marriage (514) anyways (507) alot (496) hurted (443) stupider (442) conversating (426) bae (413)	rendezvous (165) no (119) lit (97) savage (83) yikes (82) fuckboy (81) cunt (60) fam (50) twat (42) sensational (37) bitch (37) fuck (37) fuckery (29) tragic (29) irrelevant (27) wow (26) kerfuffle (23) groovy (22) rad (22) hoe (21)

Here's top 10 charts for each phrase from each bot.

@lovihatbot

I love the word X	I hate the word X
cunt (1,253) fuck (1,075) bae (1,029) thot (700) bitch (661) babe (497) woman (323) moist (309) delicious (295) rad (269)	bae (22,926) no (6,274) thot (6,155) moist (2,967) homophobia (2,052) soon (1,700) panties (1,671) selfie (1,506) sorry (1,446) sexy (1,281)

@nixibot

X is not a word	X isn't a word	X ain't a word
mines (2,682) loyalty (1,622) impossible (1,159) love (1,154) conversate (1,101) forever (909) irregardless (774) year (689) marriage (422) worser (355)	mines (1,483) justice (1,454) ain't (722) year (530) conversate (523) love (475) funner (423) bestest (342) stupider (303) worser (254)	ain't (611) loyalty (152) ignorantest (115) mines (79) love (60) impossible (51) just (43) conversate (29) worser (27) bestest (26)

@favibot

X is my new favorite word	X is my new favourite word	X is my new fave word
no (109) rendezvous (97) lit (83) yikes (64) savage (61) fuckboy (54) cunt (44) fam (43) fuck (31) sensational (29)	rendezvous (60) fuckboy (22) savage (18) cockwomble (16) cunt (14) yikes (13) twat (11) lit (11) sassy (9) hunty (8)	nondindwa (8) rendezvous (8) wow (7) yikes (5) twat (5) tragic (5) fuckboy (5) problematic (4) dope (4) savage (4)

Change from 2013

Let's see how mentions have changed for some of 2013's words.

Bae (babe) emerged in April 2013, peaked at 3,880 in May 2014 and whilst still popular, has tailed off to 437 by December 2015.

Similarly, thot ("that hoe over there") emerged in March 2013 and grew steadily to a peak of 1,400 uses in March 2014 and dropped off to 28 by December 2015.

Selfie (a shared photo taken with a phone) had a couple of mentions in April 2013, and peaked in March 2014 with 373, ending the year with 18 in December 2015.

Ratchet (a diva) peaked with 130 in March 2014, dropping to 4 by December 2015. All but *ratchet* still show up in the charts.

Changes from 2014

From 2014:

Fleek, or more fully on fleek, means on point, on the mark, stylish, amazing or impeccable. First mentioned in August 2014, it's dropped off to 14 by December 2015.

Bruh (bro, brother), not a new word but with just a few mentions throughout 2013, picked up in March 2014, and peaked in June/July 2014, but went down to 16 by December 2015 and doesn't make the top charts.

Fuckboy still charts for favibot, but peaked in March 2015 with 202 and down to 37 by year end.

Both hella and twizzle have dropped off the charts, the latter being a one-off from the 2014 winter Olympics.

New in 2015

Here's a few new to these charts.

Lit is the surprise new entry. It's commonly used as "lit af", or "lit as fuck", for example in this tweet:

My Year is starting off lit af👌🏼 ...but is gonna be TD by Monday morning

Something, I'm not sure what, happened in June 2015 that caused its use to explode on Twitter:

Fam (from family, for those closest to you but not necessarily family), has been around a long time, became popular towards the end of 2014 and is still talked about.

Similarly, slay ("killed it. succeeded in something amazing", "something you tell someone when they look sexy as f***") has been around for a while and grew towards the end of 2014, peaking in April 2015.

2016

For words causing a reaction in 2016, follow @lovihatibot, @nixibot and @favibot on Twitter.

Cutthroat verb-nouns

Tue, 26 May 2015 12:54:38 +0000

David-Antoine Williams writes:

What is the difference between a catch-all and a catch-phrase? Both are compounds formed as Verb+Noun, but in catch-all, the noun is the direct object of the verb, whereas in catch-phrase it is the subject. That is, a catch-all is something that catches all things, whereas a catch-phrase is not something that catches phrases – it is a phrase that catches something. Get it?

Recently there has been some discussion of catch-all type compounds, which Brianne Hughes has named “cutthroat compounds,” after one of the more suggestive of these. Apparently they’re rare, because they violate a general tendency for compounds in English to put the ‘head’ (e.g. phrase) on the right (‘right-headedness’). Compare F. ouvre-bouteille to E. bottle-opener (not open-bottle), which follows the most common English productive pattern, Object-Verb+er. If catch-all had followed the normal pattern, we’d be talking about an all-catcher, as we talk about dog-catchers and wind-catchers.

D-AW went on to write a script that unlurked some cutthroats, searching for verbs with left-headed combinations recorded in the entry. I decided to write a script to try another approach -- look for hyphenated nouns where the first part can a verb and the second a noun.

A WordNet list of 117,953 nouns was reduced to just 3,937 hyphenated words, then further reduced to 916 verb-nouns (i.e. a single hyphen, no spaces) via the Wordnik API, then manually whittled down. Of these potential cutthroats, the following aren't yet on Brianne's list:

be-all
cease-fire
counter-revolution
counter-sabotage
cross-classification
cross-division
cross-eye
cross-purpose
cross-question
cross-stitch
dangle-berry
dash-pot
do-good
drop-leaf
end-all
fuss-budget
knock-knee
make-work
squint-eye
sweep-second

I've been generous with some of these, some might originally be adjective-noun but can also be seen as verb-noun: it can be argued counter-revolutions are things that counters revolutions, cross-questions are things (also questions) that cross other questions (there are more counter- and cross- words like this). Some might just be wrong, but here they are.

Here's the ones it found that are already on Brianne's list:

break-axe
cure-all
do-nothing
drop-seed
know-all
make-peace
rest-harrow
save-all
shove-ha'penny, shove-halfpenny
shut-eye
spend-all

And in case I missed any, the full list of 916 verb-nouns is here alongside the Python script.

Clusterstuff

Tue, 26 May 2015 12:37:48 +0000

Strong Language, the swearing blog, recently wrote about Clusterboinks and clusterfornications: The children of clusterfuck.

Three of my Twitter bots have been collecting words from Twitter since 2013. They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favourite/fave word"

Here's the clusterfuck variants they've found:

clustercoitus
clustercuss
clusterduck
clusterfarf
clusterfark
clusterfoops
clusterfox
clusterfucked
clusterfuckery
clusterfudge
clustermindfuck
clusterwhoops

Manwords

Tue, 20 Jan 2015 12:31:58 +0000

EngLangBlog offered some man-words and asked for more.

Three bots have been collecting words from Twitter since 2013. They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favourite/fave word"

In alphabetical order, here's all the words they've found starting with man:

man
man's
manaconda
manage
manageable
management
manager
managerial
managering
managerish
managership
manaita
manaka
manana
manarchism
manarchist
manarchists
manbaby
manband
manboat
manboob
manbun
mancake
mancandy
mancave
mancer
manchacha
manchester
manchild
mand
mandals
mandate
mandated
mandation
mandatory
mandelfie
mandem
mandemz
manderstand
mandingo
manditorium
mandolized
mandy
mane
manegment
maneurysms
maneuver
manfant
manfetti
manfidence
manflu
manfriend
mang
manga
mangaks
manganèse
mangape
mangela
mangerines
mangia
mangificent
mangify
mangina
mangirrrling
mangled
mango
mangos
mangosteen
mangrove
manguish
mangy
mangyari
manhi
manhole
manhood
manhunt
mania
maniac
maniacal
maniacally
manic
manically
manicle
manicorn
manier
maniest
manifest
manifestate
manifestating
manifestation
manifesto
manigga
manikan
manikin
manila
manilla
manimal
manimony
maninism
manipul8te
manipulable
manipulate
manipulated
manipulation
manipulative
manipulatives
manipulize
manish
manism
mank
manke
mankey
mankind
mankles
manky
manlet
manliest
manlihood
manlike
manliner
manliness
manly
manlyer
manlyness
mannaman
mannequin
manner
mannerable
mannerful
mannerible
mannerism
mannerisms
manners
mannish
mannitol
manniversary
mannn
mannnnn
manny
manophobe
manosphere
manpad
manpain
manparts
manpussy
manraj
mans
manscaping
manservant
mansion
manslaughter
manslut
mansome
mansperience
mansplain
mansplained
mansplaining
manspread
manstealer
manstripper
manstruating
mant
mantastic
mantecado
manter
manther
manties
mantique
mantitties
mantlers
mantra
mantrum
manufactroversy
manufactured
manufacturers
manureisms
manveries
manwhore
many
manyer
manyuchi
manzelish
manziel
manzielish
manzzzzzz

Breakdowns by bot

"man[...] is my favourite/favorite/fave word"

man
manaconda
manaka
manarchism
manarchist
manboat
mancake
mancandy
mancer
manchacha
manchild
mandals
mandate
mandelfie
mandem
manderstand
mandingo
mandolized
mandy
mane
maneurysms
manfant
manfetti
mang
manga
manganèse
mangela
mangerines
mangia
mangify
mangina
mangirrrling
mango
mangosteen
manguish
manhole
maniac
maniacal
manic
manicorn
manifest
manifesto
manigga
manimal
manipulative
mankey
manly
mannaman
manniversary
mannn
mannnnn
manophobe
manosphere
manparts
manraj
mans
manscaping
mansperience
mansplain
mansplained
mansplaining
manspread
manstealer
manstripper
manstruating
mantastic
manter
manther
manties
mantique
mantrum
manufactroversy
manureisms
manveries
many

"I love the word man[...]"

man
manarchists
manbaby
mancake
mandate
mandatory
mandingo
mane
mang
mangina
mango
mangrove
mangy
mangyari
manhi
manhole
manhunt
maniac
maniacal
maniacally
manifest
manifesto
manila
manilla
manipul8te
manipulate
manipulated
manky
manlet
mannequin
manner
mannnnn
manpussy
manservant
manslaughter
mansplain
mansplained
mantecado
mantitties
mantra
manyer
manyuchi
manzzzzzz

"I hate the word man[...]"

man
manage
management
manager
manband
manboob
manbun
mancave
manchester
manchild
mandals
mandate
mandated
mandatory
mandem
mandemz
mane
manflu
manfriend
mang
manga
mangled
mango
manhole
manhunt
mania
manifest
manifestation
manifesto
manikin
manila
manilla
manipulate
mankind
manky
manliner
manly
manners
mannish
mannitol
manpain
manscaping
mansion
manslaughter
mansplain
mansplaining
manufactured
manufacturers
manwhore
many
manziel

"man[...] isn't/is not/ain't a word"

man
man's
manageable
management
manager
managerial
managering
managerish
managership
manaita
manana
manband
manbun
mancave
mand
mandate
mandation
manditorium
mane
manegment
maneuver
manfidence
mang
manga
mangaks
mangape
mangificent
mangina
mango
mangos
manhood
maniac
manic
manically
manicle
manier
maniest
manifestate
manifestating
manikan
manimony
maninism
manipulable
manipulated
manipulation
manipulatives
manipulize
manish
manism
mank
manke
mankles
manky
manliest
manlihood
manlike
manliness
manly
manlyer
manlyness
mannerable
mannerful
mannerible
mannerism
mannerisms
mannish
manny
manophobe
manosphere
manpad
mans
manslut
mansome
mansplain
mansplained
mansplaining
mant
mantlers
manwhore
many
manzelish
manzielish

Words of the year from Twitter, 2014

Tue, 06 Jan 2015 20:19:48 +0000

Three bots have been collecting words from Twitter for the past year.

They've looked for certain sentences and extracted the X.

@lovihatibot -- "I love/hate the word X"
@nixibot -- "X is not/isn't/ain't a word"
@favibot -- "X is my new favorite/favourite/fave word"

Combining and comparing the logs of all three, let's see what the top words of 2014 are.

Combined output

2014 total: 333,449

bae (25,689)
thot (7,787)
no (6,888)
mines (5,038)
love (3,662)
moist (3,305)
impossible (3,092)
ain't (2,960)
loved (2,954)
cunt (2,646)
conversate (2,175)
homophobia (2,064)
loyalty (1,982)
soon (1,767)
panties (1,741)
sorry (1,617)
selfie (1,613)
bitch (1,585)
justice (1,542)
babe (1,531)

lovihatibot	nixibot	favibot
2014 total: 192,388	2014 total: 118,095	2014 total: 22,974
bae (23,955) thot (6,855) no (6,469) moist (3,276) cunt (2,487) homophobia (2,053) panties (1,738) soon (1,728) selfie (1,538) babe (1,494) bitch (1,454) sorry (1,454) sexy (1,318) fuck (1,175) swag (1,139) love (1,098) pussy (1,065) bruh (1,061) fleek (1,008) baby (860)	mines (5,002) impossible (3,031) loved (2,942) ain't (2,812) love (2,560) conversate (2,154) loyalty (1,907) bae (1,585) justice (1,536) funner (1,496) irregardless (1,209) forever (1,122) worser (994) anyways (983) stupider (856) bestest (799) alot (753) thot (711) finna (707) marriage (683)	thot (224) no (194) bruh (176) bae (151) twizzle (130) fuckboy (121) cunt (115) hella (106) rad (92) twat (84) sassy (83) bitch (74) fuck (64) irrelevant (64) slunt (58) wow (57) dingus (52) slay (49) swanky (49) yikes (47)

Here's top 10 charts for each phrase from each bot.

@lovihatbot

I love the word X	I hate the word X
cunt (1,253) fuck (1,075) bae (1,029) thot (700) bitch (661) babe (497) woman (323) moist (309) delicious (295) rad (269)	bae (22,926) no (6,274) thot (6,155) moist (2,967) homophobia (2,052) soon (1,700) panties (1,671) selfie (1,506) sorry (1,446) sexy (1,281)

@nixibot

X is not a word	X isn't a word	X ain't a word
mines (3,174) impossible (2,769) love (1,554) loved (1,517) conversate (1,395) loyalty (1,208) bae (1,127) forever (1,078) irregardless (922) anyways (602)	mines (1,745) justice (1,529) loved (1,425) ain't (1,100) funner (888) love (878) conversate (715) stupider (589) bestest (538) bae (443)	ain't (1,131) loyalty (380) ignorantest (314) love (128) mines (83) impossible (75) perfect (54) worser (45) just (44) conversate (44)

@favibot

X is my new favorite word	X is my new favourite word	X is my new fave word
thot (187) no (177) bruh (145) twizzle (123) bae (111) fuckboy (96) cunt (82) hella (79) rad (65) bitch (61)	sassy (45) bae (30) cunt (30) thot (26) quiche (23) bruh (21) rad (20) hella (20) twat (18) bitchachos (17)	thot (11) bae (10) bruh (10) fuckboy (9) rad (7) sassy (7) bitchachos (7) puta (7) hella (7) literally (6)

Change from 2013

Let's see how mentions have changed for some of 2013's words.

Bae (babe) emerged in April 2013, peaked at 3,880 in May 2014 and tailed off to 1,602 by December. This is the stronger of the two. Neal Whitman charts its ascent and its use throughout 2014 whilst James Hamblin laments its death since brands have adopted it.

Similarly, thot ("that hoe over there") emerged in March 2013 and grew steadily to a peak of 1,400 uses in March 2014 and dropped off to 162 by December.

Selfie (a shared photo taken with one's own phone) had a couple of mentions in April 2013, and peaked in March 2014 with 373, ending the year with 86 in December.

Ratchet (a diva) peaked with 130 in March 2014, dropping to 20 by December.

New in 2014

Here's a few new to these charts.

Fleek, or more fully on fleek, means on point, on the mark, stylish, amazing or impeccable. First mentioned in August 2014, it's still riding the peak. First used on Vine by Peaches Monroe in June 2014, it was later popularised by Ariana Grande, who sang the Vine on MTV in August.

Bruh (bro, brother), not a new word but with just a few mentions throughout 2013, picked up in March 2014, and peaked in June/July 2014. Use is likely spurred on by its use in a Vine clip from May 2014 and in copied Vines:

CallHimBzar took the clip, and a friend of his named Headgraphix, added a disappointed “bruh” as Farmer collapsed. The “bruh” is meant to imply, “bruh, you’re a 6’7, 220-lb guy, and you beat the shit out of your girlfriend, quit acting like a bitch.” The Vine quickly blew up, and after seeing all the excitement surrounding the Vine, Headgraphix and CallHimBzar started the #BruhMovement, encouraging others to make their own Bruh Vines. And Bruh Vine they did… to the tune of thousands of Bruh vines.

Likewise, fuckboy isn't new, but has seen increasing mentions. Know Your Meme says:

“Fuckboy” used as a pejorative toward men who are perceived as oversexed or disrespectful toward women. On Tumblr, the term has also spawned a derivative phrase “en garde (English: on your guard), fuckboy," which is often accompanied with images of skeletons in a similar vein to 2Spooky.

Intensifier hella saw some use in 2013, but it increased in 2014 peaking between February and August.

Finally, twizzle was a one-off coinciding with the Sochi 2014 Winter Olympics. Virtually every mention was in February 2014 nearly all as a "favorite word" (with US spelling).

FanSided confirms the US connection to the Olympics:

If you have been watching figure skating throughout the 2014 Winter Olympics, it is impossible not to notice the term “twizzle” that is constantly being dropped by Tara Lipinksi and Johnny Weir throughout the Sochi Games.

2015

For words causing a reaction in 2015, follow @lovihatibot, @nixibot and @favibot on Twitter.

Twitter's new favourite words

Fri, 25 Oct 2013 10:22:54 +0000

I wrote a Python script that searches Twitter for tweets containing “[X] is my new favourite word”. It then takes all those new favourite words, logs them, lowercases them, and adds them to a list of words on Wordnik. And, from this week, it also tweets them and makes word clouds.

It also does the same for “[X] is my new favorite word” and “[X] is my new fave word” so you can see some geographic variation.

It’s been going six months since 24th February 2013, runs nearly every day, and has found some 18,360 new favourite words.

Here’s top tens for the whole set and each subset.

The whole data set contains 18,360 words and 10,018 unique words.

The top 10 words are:

no (151)
cunt (120)
sassy (99)
bitch (95)
ratchet (87)
fab (86)
twat (73)
cheeky (72)
faggot (66)
fuck (62)

“New favorite words” contains 10,299 words and 5,964 unique words.

The top 10 words are:

no (129)
cunt (75)
bitch (73)
ratchet (55)
thot (53)
fuck (47)
twat (41)
sassy (38)
fuckery (36)
whatever (35)

“New favourite words” contains 6,623 words and 4,505 unique words.

The top 10 words are:

fab (50)
sassy (50)
cunt (41)
cretin (36)
cheeky (34)
faggot (33)
fabulous (32)
twat (29)
ratchet (24)
serendipity (20)

“New fave words” contains 1,436 words and 1,162 unique words.

The top 10 words are:

fab (19)
sassy (11)
ratchet (8)
cheeky (7)
swaggot (7)
cretin (7)
coont (6)
bastard (6)
dope (6)
dude (5)

What does this tell us? I don’t know, but whilst the top ten contains lots of swearing, around half (or about 2/3 for favourites) of each list contains unique words. Browsing the lists shows lots of neologisms (and neoswearisms).

The word lists on Wordnik:
https://www.wordnik.com/lists/twitter-favourites
https://www.wordnik.com/lists/twitter-favorites
https://www.wordnik.com/lists/twitter-faves

Follow along:
https://twitter.com/favibot

See word clouds:
https://www.flickr.com/photos/hugovk/sets/72157636928894765/

See the script:
https://github.com/hugovk/word-tools/blob/master/new_favourite_words.py

Inspired by:
https://www.wordnik.com/lists/outcasts

How to toggle playing/pausing of Spotify using the Pause key in Ubuntu

Fri, 13 Jan 2012 13:05:26 +0000

According to Mabishu:

After a lot of requests from Linux users, Spotify developers have integrated D-Bus support in version 0.4.8.282. So, what this means is simply and awesome! Now Linux developers could use this programmatic interface to interact with Spotify from other apps.

In other words, now is quite simple to send «play», «pause», «move next/previous song» events to Spotify and with this get Spotify fully integrate into our desktop.

To toggle playing and pausing from the terminal, run:

dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause

To hook it to the Pause/Break keyboard key:

Select System -> Preferences -> Keyboard Shortcuts
Click Add
Enter any Name: Play or pause Spotify
Paste in the Command: dbus-send --print-reply --dest=org.mpris.MediaPlayer2.spotify /org/mpris/MediaPlayer2 org.mpris.MediaPlayer2.Player.PlayPause
Click Apply
Click Disabled on the right so it changes to New Shortcut…
Press the Pause/Break key
Click Close
Listen to music in Spotify and press the Pause/Break key to pause or play the music.

See the Mabishu post to see how to check the other commands you can hook up to other keys.

(Tested with Ubuntu 10.10 and Spotify for Linux preview 0.6.291)

Symbian errors, an illustrated catalogue of errors

Thu, 05 Feb 2009 12:14:33 +0000

KErrNone	0
KErrNotFound	-1	Unable to find the specified object
KErrGeneral	-2	General (unspecified) error
KErrCancel	-3	The operation was cancelled
KErrNoMemory	-4	Not enough memory
KErrNotSupported	-5	The operation requested is not supported
KErrArgument	-6	Bad request
KErrTotalLossOfPrecision	-7	Total loss of precision
KErrBadHandle	-8	Bad object
KErrOverflow	-9	Overflow
KErrUnderflow	-10	Underflow
KErrAlreadyExists	-11	Already exists
KErrPathNotFound	-12	Unable to find the specified folder
KErrDied	-13	Closed
KErrInUse	-14	The specified object is currently in use by another program
KErrServerTerminated	-15	Server has closed
KErrServerBusy	-16	Server busy
KErrCompletion	-17	Completion error
KErrNotReady	-18	Not ready
KErrUnknown	-19	Unknown error
KErrCorrupt	-20	Corrupt
KErrAccessDenied	-21	Access denied
KErrLocked	-22	Locked
KErrWrite	-23	Failed to write
KErrDisMounted	-24	Wrong disk present
KErrEof	-25	Unexpected end of file
KErrDiskFull	-26	Disk full
KErrBadDriver	-27	Bad device driver
KErrBadName	-28	Bad name
KErrCommsLineFail	-29	Comms line failed
KErrCommsFrame	-30	Comms frame error
KErrCommsOverrun	-31	Comms overrun error
KErrCommsParity	-32	Comms parity error
KErrTimedOut	-33	Timed out
KErrCouldNotConnect	-34	Failed to connect
KErrCouldNotDisconnect	-35	Failed to disconnect
KErrDisconnected	-36	Disconnected
KErrBadLibraryEntryPoint	-37	Bad library entry point
KErrBadDescriptor	-38	Bad descriptor
KErrAbort	-39	Interrupted
KErrTooBig	-40	Too big
KErrDivideByZero	-41	Divide by zero
KErrBadPower	-42	Batteries too low
KErrDirFull	-43	Folder full
KErrHardwareNotAvailable	-44
KErrSessionClosed	-45
KErrPermissionDenied	-46

Original table of errors nabbed from http://newlc.com/Symbian-OS-Error-Codes.html

Booze It!

Mon, 16 Apr 2007 18:04:49 +0000

Another amazing application for S60 phones! (But not S60 3rd Ed.) Like all the best applications it has a hard-coded exchange rate and a rubbish UI!

Booze It! is the perfect tool for those who like the odd drink or two. It has two modes: Home and Away.

Away blurb #

Ever been in a foreign pub drinking half a litre of beer (sometimes in a pint glass) and wondering what it would cost in proper money? Well, it’s easy enough to convert from euros to pounds, either in your head or with your phone’s converter app. But that only gives the price of a half litre when what you really want to know is how much a full British pint would set you back. Well, this is the app for you!

Keys:

Type the price of the foreign draught with the number keys
Up key increases the digit at the ‘imaginary cursor’
Down key decreases the digit at the ‘imaginary cursor’
Left/right keys move the ‘imaginary cursor’
Send/green key: Send an SMS to tell your friends about your beer, you may edit before sending
Press * to toggle between Home and Away modes

Menu:

Glass size: To change the size of the foreign draught. Select either preset sizes or enter your own size.
Change rate: View the current exchange rate, or enter your own, with either euro or pound as the base. Note: the exchange rate isn’t stored and will be reset to the default when you exit…
Send via SMS: Send an SMS to tell your friends about your beer, you may edit before sending
Send via Bluetooth: Share this amazing application with a friend!

Home blurb #

Picture the scene: you’re in an off licence or supermarket, choosing some booze for a party. It’s easy to see which is the cheapest, but which gives the most alcohol for your hard earned pound? You have to account for number of cans or bottles, volume in each, percentage, and price. Simply consult Booze It! to help Choose Your Booze!

The home mode currently has a very dodgy UI. Sorry. You have to imagine a cursor, and use the left and right keys to move the cursor to the digit you want to change. You then press up or down to increase or decrease the digit at the cursor.

Keys:

Up key increases the digit at the ‘imaginary cursor’
Down key decreases the digit at the ‘imaginary cursor’
Left/right keys move the ‘imaginary cursor’
Send/green key: Send a booze report via SMS, you may edit before sending
Press * to toggle between Home and Away modes

Menu:

Send via SMS: Send a booze report, you may edit before sending
Send via Bluetooth: Share this amazing application with a friend!

See more S60 stuff here.

Gmail on Series 60

Mon, 13 Jun 2005 10:35:00 +0000

These settings work for sending on the Nokia 6680, but not receiving… I used to be able to receive as well, but at least sending works and means I can use the “Send as email” option in things like Lifeblog, and use Opera or the phone’s built-in Web app to check received email at gmail.com.

Series 60 Big Clock

Mon, 30 May 2005 21:40:06 +0000

And here’s a freeware big clock for your Series 60 phone :)

Morse Texter

Mon, 30 May 2005 17:07:09 +0000

After reading this and this and this I decided to knock up this ~~dodgy prototype~~ amazing application for S60 v2.x phones (but not v1.x or v3.x – check your version here. Tap in some Morse code, and then send it as an SMS. Keys are:

1, 4, 7, or * - Dot
2, 5, 8, or 0 - Space
3, 6, 9, or # - Dash
Left arrow - Dot
OK key - Space
Right arrow - Dash
C key - Delete last dot, dash or letter
Call/green key - Send as SMS

For a gap between letters, press space once. For a gap between words, press space a second time.

Here’s the source (Google Code, GitHub), do whatever you want with it. Some ideas:

Add short and long key press handling on the OK key for dot and dash input, with proper timing to add character and word spaces
Centre the Morse characters and make them bigger/bolder
More ideas

. / -. / .— / — / -.– / -.-.–

Blog on Hugo van Kemenade

A CLI to fight GitHub spam

gh triage spam #

gh triage unassign #

See also #

Speeding up Pillow's open and save

Tachyon #

Loading all the plugins? #

Lazy loading #

See also #

Localising xkcd

PyCon Italia #

PyCon Greece #

PyCon Estonia #

PyCon Finland #

PyCon Sweden #

Thanks #

Replacing python-dateutil to remove six

One #

Two #

Three #

Bonus #

And now for something completely different

alpha 1 #

alpha 2 #

alpha 3 #

alpha 4 #

alpha 5 #

alpha 6 #

alpha 7 #

beta 1 #

beta 2 #

beta 3 #

beta 4 #

release candidate 1 #

release candidate 2 #

release candidate 3 #

3.14.0 (final) #

3.14.1 #

Steering Council results

Steering Council election

Setting secrets in env vars

1Password #

direnv #

Bonus #

Update #

Python Core Sprint 2025

Monday highlights #

Tuesday highlights #

Wednesday highlights #

Thursday highlights #

Friday highlights #

Thank you #

Three times faster with lazy imports

Setup #

Benchmarks #

Inline imports #

Fully lazy #

Releasing Python 3.14.0

Prologue #

One week #

Three days #

Green #

Release day! #

run_release.py #

Installers #

Windows #

macOS #

Final steps #

🚀 It’s out! #

Finally #

Ready prek go!

Getting started #

Benchmarks #

Results #

Bonus #

EuroPython 2025: A roundup of writeups

Update #

Run coverage on tests

Example one #

`gh triage spam` #

`gh triage unassign` #

`run_release.py` #

actions/setup-python: `t` suffix #

actions/setup-uv: `t` suffix #

actions/setup-python: `freethreaded` variable #

`PYTHON_GIL=0` #

`pyproject.toml` #