diff options
author | Amir Sarabadani <ladsgroup@gmail.com> | 2023-06-08 14:20:51 +0200 |
---|---|---|
committer | Ladsgroup <Ladsgroup@gmail.com> | 2023-06-13 11:06:28 +0000 |
commit | b8e25022d842dbceb44377b139a5fb66867a7d5a (patch) | |
tree | 126e705925d199afbf32c4e009dede42ef87d460 | |
parent | d5a48eeb47a236f1a0eef56f9993608c0eddc834 (diff) |
moveToExternal: Also check for utf8 encoding before trying to convertwmf/1.41.0-wmf.12
While most rows in production use 'utf-8' to flag content being UTF-8,
we have lots of rows flagged with 'utf8':
mysql:research@s3-analytics-replica.eqiad.wmnet [dawiki]> select old_flags, count(*) from text group by old_flags limit 50;
+---------------------+----------+
| old_flags | count(*) |
+---------------------+----------+
| error | 2 |
| external,gzip | 49 |
| external,object | 36 |
| external,utf-8 | 1614469 |
| external,utf8 | 336780 |
| gzip,utf-8,external | 1094 |
| utf-8,gzip,external | 9458083 |
+---------------------+----------+
7 rows in set (26.038 sec)
This would confuse the script to try to reencode it again which possibly
could lead to all sorts of errors
Change-Id: I9b4a38538199c9954cfed51cdd2bba8b0f6cb953
(cherry picked from commit 4dd3850bebdd972001a007f1bcc5a127d3d4dc4f)
-rw-r--r-- | maintenance/storage/moveToExternal.php | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/maintenance/storage/moveToExternal.php b/maintenance/storage/moveToExternal.php index 520d55ab18ae..470af0cd203a 100644 --- a/maintenance/storage/moveToExternal.php +++ b/maintenance/storage/moveToExternal.php @@ -232,6 +232,7 @@ class MoveToExternal extends Maintenance { private function resolveLegacyEncoding( $text, $flags ) { if ( $this->legacyEncoding !== null && !in_array( 'utf-8', $flags ) + && !in_array( 'utf8', $flags ) ) { AtEase::suppressWarnings(); $text = iconv( $this->legacyEncoding, 'UTF-8//IGNORE', $text ); |