Package installer fails with multibyte characters in the readme

[rthrash]

Bug report

Summary

On sites without a multibyte database, the installer fails to download or install manually uploaded Extras that have multibyte characters in the readme.

Step to reproduce

1. Set up a 3.x (and I suspect 2.x, untested) site without a multibyte-compatible DB
2. Try to install modAI 0.11 or other package with emojis in the readme

Observed behavior

Could not download and create transport package with signature: modai-0.11.0-pl

Expected behavior

The Extra should download and install. There are workarounds—see below—but it would be nice for devs to be able to use any character in their readme files.

Environment

non-UTF8mb4 databases

Possible Solution

There may be a possilbe solution outlined here, along with a great explanation of the challenges involved above that anchor link. Hat tip to @jaygilmore for finding that.

Workarounds

@gpsietzema, who originally pointed this out, suggested a workaround on databases that can support it:

So when i change to utf8mb4 + utf8mb4_unicode_ci it works

You can manually convert emojis to html-entieis, e.g., 🚀 → 🚀. On Mac, Raycast can copy the unicode values of any emoji character using ⌘⇧U in their emoji picker, e.g., U+1F680 for this example. You then would swap the U+ with &#x and add a semicolon at the end ; of each unicode string—if multiple, strip out any spaces between them.